Configuration for Containerized Web Apis

Sometimes, configuration is hard. And sometimes, configuration is hard to understand. In the case of this post, configuration works just awesome. The problem was me, I did not understand how it worked. Now I do, so let’s go and share.

TL;DR Environment variables can map to appsettings, and when using a hierarchy, use a double underscore for environment variables instead of a colon.

Configuration done right

When you’re developing a .NET Web API, configuration is fairly simple. If you use one of the project templates that come with Visual Studio, it will create a file called appsettings.json. This file allows you to configure your App, even using a hierarchy when desired. Take a look at the following example:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "AllowedHosts": "*",
  "Azure": {
    "StorageAccount": "azurestorageaccount"
  }
}

You can see a nice hierarchy for logging options and my Azure configuration, and a straightforward text configuration value for AllowedHosts.

Be careful though… Usually, this file will be included in your source control system. This means that secrets should never end up in this file because if they do, the secrets are considered to be compromised.

"Azure": {
    "StorageAccount": "azurestorageaccount",
    "StorageAccountSecret": "2142r84ybqw2e3r7hb" // <- COMPROMISED
}

I will tell you how to work with secrets later, but to better understand this, let’s first find out how .NET allows multiple configuration sources to come to a configuration for your software system.

Stacking configuration

One very nice feature of ASP.NET is that it can stack configuration sources on a pile. The combination of all the configuration sources will end up being your App’s configuration. The order in which you stack those configuration options is important, because when two configuration sources contain the same key (with a different value or not), the value of a preceding source, will overwrite the previous value.

Working with secrets

You must never store secrets in your appsettings.json file, luckily there is lovely tooling to allow you to configure secrets locally without pushing them to source control. When you use Visual Studio, you can right-click your ASP.NET project and click ‘Manage User Secrets’. This opens a new file called secrets.json. This is where you can safely store secrets because this file is stored outside your source control system.

If you don’t use Visual Studio, you can use the dotnet CLI to generate the secrets file for you. Navigate to the folder of your project and open a command window. There, type dotnet user-secrets init. The secrets file is now generated for you. The file can be found here: %APPDATA%\Microsoft\UserSecrets\<user_secrets_id>\secrets.json. You can review your project file (your-project.csproj) to find the user_secrets_id which is usually a GUID.

By default, both the appsettings.json and the secrets.json are on the configuration stack (unless you changed this manually). This means that you don’t have to do anything to make the values in secrets.json overwrite the values of appsettings.json.

So a configuration (appsettings.json) file that looks like so:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "AllowedHosts": "*",
  "Azure": {
    "StorageAccount": "my storage account",
    "StorageAccountSecret": "enter your secret value here"
  }
}

in combination with a user secrets in a secrets.json file that looks like so:

{
  "Azure": {
    "StorageAccount": "myazurestorageaccount",
    "StorageAccountSecret": "214y12n457ewrhf87wber7n24rh7"
  }
}

Will end up in a single configuration containing a configuration with storage account name myazurestorageaccount and secret 214y12n457ewrhf87wber7n24rh7.

Mapping hierarchy to a single string

This Azure object configured above is a great subject for the IOptions pattern, but this pattern is outside the scope of this post. When you want to retrieve a value from configuration, you can use the IConfiguration object to fetch a value by its name:

public ClassConstructor(IConfiguration config)
{
    var configurationValue = config["AllowedHosts"];
}

When you use a hierarchy as I did in the configuration example above, each time you go a level deeper, type a colon. So if I would like to retrieve the name of my storage account from the configuration:

public ClassConstructor(IConfiguration config)
{
    var configurationValue = config["Azure:StorageAccount"];
}

And now, how about containers?

What I didn’t know, is that environment variables also take part in this configuration stack. This is very useful because when you containerize your app, you cannot change the configuration through a file. Instead, containers rely on environment variables. And you are able to change environment variables.

Now especially when you use this IOptions pattern that I mentioned earlier, for example, the environment variables were very confusing to me. I thought I needed a check to see if the system is running from a container or not. Depending on that check, fetch values from the app configuration or environment variables.

In reality, environment variables are on the stack of configuration options and will be loaded by a configuration provider. I did not realize this because I like to make these hierarchies in configuration files to structure them in a more readable way. And when you have such a structure, this colon comes into play. Unfortunately, a colon is not allowed for the name of an environment variable so I was stuck there.

The Hidden Gem

It turns out, that if you name your environment variables similar like you would retrieve them as shown earlier: config["Azure:StorageAccount"] and replace the colon with two underscores, so Azure__StorageAccount, then the environment variable can be retrieved in the exact same way as App Settings: config["Azure:StorageAccount"]. So basically, the double underscore for environment variable names are replaced with colons and can thereby also overwrite values from appsettings.json at startup.

Starting to work with containers, this configuration trick was very new to me and took me some time to find out. I have worked with .NET for a couple of years now and I knew the configuration options were awesome. I simply did not know it was this clever.