How to configure and provide secrets management securely in Azure Container Apps
In the previous post, we provisioned an Azure Container Apps environment and deployed the ASP.NET sample application into a Container App.
Let’s take it a step further use a .NET Core application and demonstrate how we can inject configuration and provide secrets securely in Azure.
This post assumes you have worked through the previous post:
https://www.playtimesolutions.com.au/how-to-deploy-a-sample-container-apps-application
The source code is available here:
https://github.com/playtimesolutions/ContainerAppsDemo
Step 1 – Local configuration settings
1.1. Inspect the application configuration
Our application will use hard-coded app settings unless we specifically override them.
src/dotnet-api/appsettings.json
{
"Settings":{
"ConfigValue": "Defined in appsettings.json"
}
}
Program.cs
The following API endpoint will log the application’s configuration to the console. This is an insecure thing to do, so we’re only doing this here you can conveniently see how the application is configured.
app.MapGet("/config", () =>
{
foreach (var k in builder.Configuration
.AsEnumerable()
.OrderBy(k => k.Key))
{
Log.Information($"{k.Key}: {k.Value}");
}
});
1.2. Use the Environment Variables Configuration Provider
dotnet run
curl http://localhost:8080/config
You’ll see the application’s configuration in the console, which includes all of the app settings and environment variables in the process.
..
Settings:
Settings:ConfigValue: Defined in appsettings.json
..
We can override these values at runtime in various ways.
A common way to override app settings at runtime is to use the EnvironmentVariablesConfigurationProvider. This provider is baked into .NET Core applications, so we can leverage this behavior without having to write any additional code.
See: https://learn.microsoft.com/en-us/dotnet/core/extensions/configuration-providers
Add the following environment variable into your shell and re-run the application.
export Settings__ConfigValue="Defined in an environment variable"
dotnet run
curl http://localhost:8080/config
The configuration provider has overridden the hard-coded app setting with the environment variable.
..
Settings:
Settings:ConfigValue: Defined in an environment variable
..
By following this approach, an application’s configuration can be easily externalized and driven from the environment instead of being hard-coded.
See: https://12factor.net/config
Step 2 – Deploy the application to Container Apps to Azure and inject settings
2.1. Build the container image
The following code creates a container image and pushes the image to the Azure Container Registry.
cd src/dotnet-api
# tag the image
TAG=$(date '+%Y%m%d%H%M%S')
IMAGE_NAME="dotnet-api:$TAG"
CONTAINER_IMAGE="$CONTAINER_REGISTRY_URL/$IMAGE_NAME"
# build dotnet-api container image
docker build -t dotnet-api:latest .
docker tag dotnet-api:latest $CONTAINER_IMAGE
# login
az acr login --name $CONTAINER_REGISTRY_NAME
# push
docker push $CONTAINER_IMAGE
2.2. Deploy the container app.
Deploy the container image.
az deployment group create \
--resource-group $RESOURCE_GROUP \
--name $DEPLOYMENT_NAME \
--template-file devops/deploy.bicep \
--parameters "configValue=Defined in a Container App environment variable" \
--parameters "containerRegistryName=$CONTAINER_REGISTRY_NAME" \
--parameters "environmentName=$ENVIRONMENT_NAME" \
--parameters "keyVaultName=$KEYVAULT_NAME" \
--parameters "imageName=$IMAGE_NAME" \
--query properties.outputs.fullyQualifiedDomainName.value
Store the fullyQualifiedDomainName in a variable for use later
URL="the_fdqn"
src/dotnet-api/devops/deploy.bicep
The bicep definition for the container app should look familiar – it’s very similar to the example in the previous post. However, this time we will define a variable in the container’s environment via a CLI-provided parameter.
...
param configValue string;
...
containers: [
{
...
env: [
...
...
{
name: 'Settings__ConfigValue'
value: configValue
}
...
...
]
}
]
...
...
To view the logs from the Container App, load the /config endpoint in the browser to produce the logs, then issue the following command to view the logs.
curl $URL/config
az containerapp logs show --name dotnet-api --resource-group $RESOURCE_GROUP
You’ll see the following included in the logs.
..
Settings:
Settings:ConfigValue: Defined in a Container App environment variable
..
With this approach, we can build a single container image with the default configuration and override settings on a case-by-case basis in order to customize the application for each environment (e.g. dev/test, prod).
Step 3 – Configure Secrets in Azure
Our objectives for secure secrets management are as follows:
- Use a key vault to store secrets
- Do not allow access to secrets unless it is specifically granted (least privilege)
- Do not expose secrets via configuration files or environment variables
Our secrets configuration approach will leverage the same configuration provider pattern as environment variables, however this time we have to jump through some additional hoops in order to secure the solution.
3.1. Setup a managed identity for the container app
In order to securely access the key vault, we need to set up a managed identity and associate it with the container app.
...
resource containerAppIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2018-11-30' = {
name: '${containerAppName}-identity'
location: location
}
resource containerApp 'Microsoft.App/containerApps@2022-03-01' = {
name: containerAppName
location: location
identity: {
type: 'UserAssigned'
userAssignedIdentities: {
'${containerAppIdentity.id}' : {}
}
}
...
The next step is to grant access to the key vault’s managed identity for ‘get’ and ‘list’ operations, so the application can retrieve the secrets at runtime.
...
resource keyVault 'Microsoft.KeyVault/vaults@2021-11-01-preview' existing = {
name: keyVaultName
}
resource keyVaultAccessPolicy 'Microsoft.KeyVault/vaults/accessPolicies@2019-09-01' = {
name: 'add'
parent: keyVault
properties: {
accessPolicies: [
{
tenantId: subscription().tenantId
objectId: containerAppIdentity.properties.principalId
permissions: {
secrets: [
'get'
'list'
]
}
}
]
}
}
...
This is an example of least privilege.
See: https://en.wikipedia.org/wiki/Principle_of_least_privilege#Details
Now, create a secret in the key vault that our application can consume.
resource runtimeSecret 'Microsoft.KeyVault/vaults/secrets@2021-11-01-preview' = {
name: 'dotnet-api-Settings--SecretValue'
parent: keyVault
properties: {
attributes: {
enabled: true
}
value: 'Defined in a key vault secret'
}
}
3.2. Create additional environment variables
The following environment variables are required to drive the process.
...
param keyVaultName string
...
containers: [
{
...
env: [
...
...
{
name: 'ASPNETCORE_ENVIRONMENT'
value: 'Azure'
}
{
name: 'AzureADManagedIdentityClientId'
value: containerAppIdentity.properties.clientId
}
{
name: 'AzureKeyVaultUri'
value: 'https://${keyVaultName}.vault.azure.net'
}
...
...
]
}
]
...
...
3.3. Wire up the KeyVault
Using the AzureKeyVaultConfigurationExtensions, we can extract values from a specified key vault and securely load them into the IConfiguraiton.
The extension works as follows:
- If
ASPNETCORE_ENVIRONMENTis Azure, then proceed. - Retrieve the key vault URL and the managed identity from the environment.
- Add the key vault integration, and specify that the only secrets required are those prefixed with
dotnet-apias per the secret we created in step 3.1
public static class Extensions
{
public static IConfigurationBuilder AddKeyVaultSecrets(this IConfigurationBuilder builder, string prefix)
{
var settings = builder.Build();
var environment = settings["ASPNETCORE_ENVIRONMENT"];
if (environment == "Azure")
{
var keyVaultEndpoint = settings["AzureKeyVaultUri"];
var azureADManagedIdentityClientId = settings["AzureADManagedIdentityClientId"];
var credentials = new DefaultAzureCredential(new DefaultAzureCredentialOptions
{
ManagedIdentityClientId = azureADManagedIdentityClientId
});
builder.AddAzureKeyVault(new Uri(keyVaultEndpoint), credentials,
new AzureKeyVaultConfigurationOptions
{
Manager = new PrefixKeyVaultSecretManager(prefix)
});
}
return builder;
}
private class PrefixKeyVaultSecretManager : KeyVaultSecretManager
{
private readonly string _prefix;
public PrefixKeyVaultSecretManager(string prefix) => _prefix = $"{prefix}-";
public override bool Load(SecretProperties properties) => properties.Name.StartsWith(_prefix);
public override string GetKey(KeyVaultSecret secret)
=> secret.Name[_prefix.Length..].Replace("--", ConfigurationPath.KeyDelimiter);
}
}
Include this code on startup.
builder.Configuration.AddKeyVaultSecrets("dotnet-api");
Ensure the application has been deployed to azure and call the /config endpoint as before, observe that we have the secret value available in the logs – it has been loaded into the application process’s memory in the IConfiguration during startup.
..
..
Settings:
Settings:SecretValue: Defined in a key vault secret
..
..
We don’t want secrets residing in the environment variables for the container – doing so would be a security vulnerability if a malicious actor gained access to the container and was able to extract the environment.
In the Azure portal we can extract the container’s environment through a terminal session.
Run the following command
env
You’ll see that Settings__SecretValue is not exposed in the container’s environment. We’ve met our three objectives for secure secret handling.
Conclusion
In this post, we’ve seen a common way to manage configuration settings and secrets in containerized .NET Core applications.
Overriding secrets with a least-privilege approach while not exposing them to malicious actors can be achieved fairly easily with the approach outlined above.
Stay tuned for the next post in the series where we’ll demonstrate how to lock down your environment even further with the use of VNET and Private Endpoints.
Playtime Solutions’
AKS Best Practice Guide
A 23 page best-practice checklist, leveraging Playtime Solutions’ hands-on experience in designing, developing and delivering enterprise-grade application. This guide assists IT and DevOps professionals in creating an enterprise-grade Kubernetes environment in Microsoft AKS.
