Dot Env Files: The Silent Killer

Mar 16, 2018

Laravel’s .env files are fascinating things. They seem so innocent on the surface, but underneath the friendly façade they are vicious beasts capable of bringing your entire application to its knees. In this post we’ll look at some of the ways mismanaged .env files can trip up even the most seasoned developer, and introduce a simple and powerful technique for avoiding these pitfalls.

Sit back, and let me tell you a tale from the trenches about a surly developer, played by you, navigating the hills and valleys of a career in web development.

The Problem(s)

Example 1: It Works On My Machine

Your teammate Robin pings you to code review a new feature before pushing it to QA. You promptly get your local environment up to speed by running git pull the-feature-branch, composer install, and npm run dev. Your first course of action is a preventative phpunit run to make sure the basics are covered, but you’re immediately met with a red console. You decide to open up the browser to test the feature in real life and encounter various 500s and 404s. Peeved that your teammate wasted your time and didn’t run the unit tests, you write an annoyingly formal and condescending message in your team’s public Slack channel: “Hey Robin, the tests are failing on your feature branch. In the future, make sure you run the full test suite before submitting a PR - thank you”. She politely responds “Hmmm… they’re passing on my machine 🤔”.

Frustrated, but a little worried that you sounded the alarm too soon, you dig deeper into the error logs and find them filled with ugly [2018-02-28 16:21:36] local.ERROR: cURL error 3: <url> malformed errors. Twenty minutes later you trace the problem back to—you guessed it—a missing .env variable for a newly-introduced API endpoint. Hoping to salvage any dignity you have left and regain the high ground, you search the branch’s .env.example file hoping to find the variable missing. However, Robin diligently added the new .env variable. Slightly embarrassed, you ping her back with an overly casual response and rob her of a well-deserved apology “ah ok, figured it out - thanks!”

Just another day at the office…

Problem: Pulling down, or checking out branches and forgetting to check for newly added .env.example variables.

Solution: A command that runs automatically on git pull or git checkout and warns you of newly added .env.example variables missing from your .env file.

Example 2: I Definitely Added That… Right?

Now it’s your turn to ping Robin for a code review. Robin, who is much more even-tempered than you, DMs you, asking: “I can’t seem to get your PR working in my local environment; mind if we hop on a screen share quick?” You both join and, because of your collective brainpower, quickly identify she is missing a .env variable. Because checking the .env.example file is part of her personal code-review process, she knows you forgot to add this variable. She promptly adds the new variable and pushes the change up to the repo, assuring you she does the same thing herself all the time. You are again ashamed and embarrassed, but thankful for her mercy. You promise yourself next time you’ll remember.

Problem: Pushing new code and forgetting to add a new .env variable to .env.example

Solution: A command that runs automatically on git push, preventing the push from completing if a .env variable is missing from your .env.example file.

Example 3: ROLL BACK!

All the reviews are finished, the code is in the QA environment, the product people have finished user acceptance testing, and the deployment ticket is ready to go! The ticket instructs DevOps to merge the QA branch with main. They execute the deployment and by this time, your hands are behind your head, your feet up on the desk, and you’re casually watching the Jenkins build complete as smoothly as expected. The build completes, you go about your day, when someone pings you: It’s customer service! Customers are complaining they can’t access the app. Screenshots of Whoops, something went wrong errors are flooding into the support portal. The sky is falling; people are screaming and running around the office; papers are flying all over the place. You are now running through the chaos towards DevOps, yelling in slow motion: “Rooollllll baaaacckkkk!!!”

After the code rolls back and the dust settles, you pull up the logs. Your palm immediately contacts your face. [2018-02-28 16:21:36] local.ERROR: cURL error 3: <url> malformed. You know you’ve been beaten, once again, by the dreaded .env file.

Problem: Deploying a branch and not adding a .env variable to the production environment before the deploy.

Solution: A command that runs in CI during a build that catches .env.example variables missing from the environment’s .env file, and returns an exit code of 1, stopping the build before deployment.

An ideal solution to our problem

A command that can check for .env.example variables missing from .env — this will cover the scenarios described in Examples 1 and 3
A command that checks for variables found in .env but missing from .env.example — this will cover Example 2
A way to hook these commands into the various Git commands and CI builds

There’s a Package For That

Julien Tant put out a package called Laravel Env Sync that does everything we need right out of the box. Thank you, Julien!

Let’s install it and run the commands to get started.

composer require jtant/laravel-env-sync

Thanks to auto-registering of Service Providers, which was introduced in Laravel 5.5, we are ready to go, just like that! The two commands we will be using are:

php artisan env:check — alerts you to variables in .env.example that are missing from your .env.
php artisan env:check --reverse — does the opposite, looking for variables in your .env that aren't present in .env.example.

Note: there are other commands made available by this package: env:diff & env:sync. Feel free to use them, but make sure you aren’t relying on them to stop a build, as they don’t return exit codes.

Hooking the package into Composer and Git

Fortunately, both Git and Composer make it pretty easy to hook into various actions. Let’s take a look at what’s involved.

Composer hooks

In your composer.json file, there is a scripts entry that comes shipped with Laravel:

"scripts": {
    "post-root-package-install": [...],
    "post-create-project-cmd": [...],
    "post-autoload-dump": [...]
}

We need to add a post-install-cmd hook; any command under this hook will run after a composer install. Because composer install is typically run during a deployment and after switching branches, this will cover most of our needs.

Also, because composer.json is under version control, adding this command will cover your entire team.

Here is the command added to the proper composer hook:

"scripts": {
    "post-root-package-install": [...],
    "post-create-project-cmd": [...],
    "post-autoload-dump": [...],
    "post-install-cmd": [
        "php artisan env:check"
    ]
}

Boom! You are officially protected against deploying with a missing .env variable. Now, let’s take this one step further and integrate this functionality deeper into our workflow.

Git hooks

Git hooks are little bash scripts that run before or after common Git commands. A few of the available hooks include pre-commit, pre-push, pre-rebase, post-update, and post-checkout.

These hook files can be configured on a per-project basis and are stored in the directory your-project/.git/hooks

Before we walk through creating our hooks, I want to make a quick note about Git hooks.

Versioning Git hooks

Git hooks are powerful tools, but because the .git directory isn’t under version control, their usefulness is limited. Most times I’ve wanted to use them, they would’ve been much more powerful if they were enforced across a team.

There are a couple interesting ways to version control Git hooks, but here is my preference:

Create a new directory at the root of your project called githooks
Run the following command: git config core.hooksPath githooks

Note: You will want to add this command to your README under your setup instructions, or better yet, add it to a bin/setup script that onboards new developers quickly and consistently.

Creating our Git hooks

For our purposes, we will need to set up the following hooks: post-checkout & pre-push.

Let’s start with pre-push. This file will be executed when you run the git push command. If the script returns with an exit code of 1 (a generic failure exit code), the push will be prevented.

pre-push

To create this hook, add a file to your new githooks directory called pre-push and populate it with the following:

#!/bin/sh
 
# This git hook makes sure all your .env vars are in .env.example before pushing up.
 
php artisan env:check --reverse
 
exit $?

Important Note: Git hook files must have executable permissions in order to run and can be extremely difficult to debug without this knowledge. To make the hook executable, run the following command: chmod 777 .githooks/pre-push

Great, no longer will you forget to add that sneaky .env.example variable. Now let’s tackle the post-checkout hook.

post-checkout

I often don’t remember to run composer install after checking out a new branch. To remove this point of failure (my brain), let’s set up our post-checkout hook to run composer install automatically after every branch switch.

Create a githooks/post-checkout file, and populate it with the following:

#!/bin/sh
 
# This hook is called with the following parameters:
#
# $3 - A flag indicating whether the checkout was a branch or file checkout
 
isFileCheckout=0
isBranchCheckout=1
 
if [ $3 -eq $isBranchCheckout ]
then
    composer install
fi

You’ll notice a little bit of fanciness here because, otherwise, this hook will also run during a file checkout, which is overkill.

Let’s not forget to make the file executable: chmod 777 .githooks/post-checkout

Now that we’ve set up these Git hooks, our workflow is automatically covered against all .env / .env.example derps. Aaah, what a good feeling.

Wrapping up

So there you have it. Countless hours (at least for me) saved.

I’m surprised I’ve made it this far as a programmer without having identified .env files as the huge liability they are. When you consider the bad things that can happen when your .env file is misconfigured, relying on only your memory to keep your .env file in shape is risky. Once it’s all handled automatically, though, you should feel much more at peace with your deployments, and you'll appreciate all the time you’ll save debugging .env-related issues.

Be sure to follow Julien Tant and thank him for his hard work making that killer package!