Integrating Azure Key Vault with Azure DevOps Pipelines

Integrating Azure Key Vault with Azure DevOps Pipelines

Azure Key Vault is a cloud service provided by Microsoft Azure that allows you to securely store and manage sensitive information such as secrets, keys, and certificates. In the context of Azure DevOps Pipelines, integrating Azure Key Vault provides a secure way to manage and access sensitive configuration data like API keys, database connection strings, passwords, and certificates during the pipeline execution process.

In this guide, we’ll walk through how to integrate Azure Key Vault with Azure Pipelines and securely use secrets from Azure Key Vault in your build and release pipelines.

Overview of the Integration Process

To use Azure Key Vault with Azure Pipelines, there are two primary steps:

1. Grant access to Azure Key Vault: Azure DevOps needs to have permission to access secrets from Azure Key Vault.

2. Reference secrets from Azure Key Vault in the pipeline: Use pipeline tasks to retrieve secrets stored in Azure Key Vault during the pipeline run.

Step 1: Grant Azure DevOps Access to Azure Key Vault

Before your Azure Pipeline can access the Key Vault secrets, you must ensure Azure DevOps has the appropriate permissions to access the Key Vault.

1. Create a Service Connection for Azure Key Vault

To allow Azure DevOps to access Key Vault, you need to create an Azure service connection with the right permissions.

Steps:

1. In Azure DevOps, go to your project.

2. Navigate to Project Settings > Service connections.

3. Click on New service connection and select Azure Resource Manager.

4. Select Service principal (automatic) to create the connection automatically.

5. Provide your subscription details and ensure you grant access to the resource group containing your Key Vault.

Once created, you’ll see this service connection under Service connections.

2. Set Access Policies in Azure Key Vault

After creating the service connection, you need to ensure that the correct permissions are set on the Azure Key Vault.

Steps:

1. Navigate to your Azure Key Vault in the Azure portal.

2. Go to Access policies under the Settings section.

3. Click + Add Access Policy.

4. In the Select principal field, search for your Azure DevOps service connection or the service principal that was automatically created by Azure DevOps.

5. Select the Get and List permissions for Secrets (and Certificates if necessary).

6. Click Add, then Save to apply the policy.

This will allow Azure DevOps to read secrets from Azure Key Vault during the pipeline execution.

Step 2: Reference Azure Key Vault Secrets in Azure Pipelines

Now that Azure DevOps has the necessary permissions to access the Azure Key Vault, you can retrieve secrets from it in your pipelines.

There are multiple ways to access Key Vault secrets in Azure Pipelines, including using the Azure Key Vault task or via pipeline variables.

1. Using the Azure Key Vault Task in YAML Pipelines

To integrate Azure Key Vault with an Azure Pipelines YAML file, you can use the AzureKeyVault task.

Example: Azure Key Vault task in YAML pipeline

xxxxxxxxxx
13
1
trigger:
2
- main
3
pool:
4
  vmImage: 'ubuntu-latest'
5
steps:
6
- task: AzureKeyVault@2
7
  inputs:
8
    azureSubscription: '<Azure Service Connection>'
9
    KeyVaultName: '<Your Key Vault Name>'
10
    SecretsFilter: ''  # This fetches all secrets, or you can specify specific secret names like 'MySecret'
11
- script: |
12
    echo $(MySecret)  # Reference the secret retrieved from Key Vault
13
  displayName: 'Display the secret value'

Explanation:

1. azureSubscription: The name of the Azure service connection you created earlier.

2. KeyVaultName: The name of your Azure Key Vault instance.

3. SecretsFilter: This specifies which secrets to retrieve. You can use ` to get all secrets or specify a comma-separated list of secrets (e.g.,MySecret, APIKey`).

The secrets retrieved from the Key Vault will be automatically injected as environment variables into your pipeline. You can reference these secrets using the $(<SecretName>) syntax.

2. Using Azure Key Vault in Classic Release Pipelines

If you are using Classic Release Pipelines (GUI-based), you can use the Azure Key Vault task.

Steps to use Azure Key Vault in Classic Release Pipelines:

1. Go to the Pipelines section in Azure DevOps.

2. Select the Releases tab and create a new release pipeline.

3. In the release pipeline, add a new Azure Key Vault task:

   • Azure Key Vault name: Enter your Key Vault name.

   • Secrets filter: Specify the secrets to fetch (e.g., MySecret, or “ to fetch all).

After this task, you can add another task to use the fetched secrets (for example, a PowerShell task or a script task) where the secrets are available as environment variables.

3. Using Secrets as Pipeline Variables

You can also inject Azure Key Vault secrets into pipeline variables, which can be accessed throughout the pipeline.

Example: Injecting secrets into pipeline variables:

xxxxxxxxxx
15
1
trigger:
2
- main
3
pool:
4
  vmImage: 'ubuntu-latest'
5
variables:
6
  - group: MyVariableGroup  # Optionally link variable groups for more complex variable management.
7
steps:
8
- task: AzureKeyVault@2
9
  inputs:
10
    azureSubscription: '<Azure Service Connection>'
11
    KeyVaultName: '<Your Key Vault Name>'
12
    SecretsFilter: 'MySecret'
13
- script: |
14
    echo $(MySecret)  # Use the secret as a pipeline variable
15
  displayName: 'Use secret as variable'

In this case, the $(MySecret) variable will hold the value of the secret stored in Azure Key Vault. You can then use this variable throughout the pipeline.

4. Using Azure Key Vault in Variable Groups (for Reusability)

For larger teams or applications that need to share secrets across multiple pipelines, you can create Variable Groups in Azure DevOps and link them to Azure Key Vault.

Steps to Create and Link a Variable Group:

1. Go to Pipelines > Library > + Variable group.

2. Add a new variable group, give it a name, and link the secrets stored in Azure Key Vault.

When creating or editing a pipeline, you can reference this variable group, and the secrets stored in Key Vault will be automatically available as pipeline variables.

Best Practices When Integrating Azure Key Vault with Azure Pipelines

1. Least Privilege Access: Always follow the principle of least privilege when granting access. Only allow the Azure DevOps service connection or service principal to have the minimal necessary permissions (e.g., Get and List permissions for secrets).

2. Avoid Logging Sensitive Data: Be cautious when using sensitive data in logs. Avoid printing sensitive secrets directly to the console or logs (e.g., using echo for secrets).

3. Use Secure Files for Certificates: If you store certificates in Azure Key Vault, you can use Azure DevOps secure files to manage these securely and use them in pipelines.

4. Use Managed Identity (if applicable): For better security, consider using a Managed Identity (if running on Azure-hosted agents) instead of a service principal. This avoids the need to manage secrets for your service connection.

5. Securely Manage Secrets: Rotate secrets in Azure Key Vault regularly and ensure that secrets are revoked when no longer needed.

Summary

Integrating Azure Key Vault with Azure Pipelines provides a secure and efficient way to manage sensitive configuration data and secrets during your CI/CD workflows. By using the AzureKeyVault task, you can fetch secrets directly from Key Vault and use them as pipeline variables, all while ensuring that sensitive data never appears in the pipeline logs or source code.

With proper access control and best practices, this integration ensures that your sensitive data is always stored securely, making your DevOps processes safer and more efficient.