Azure Pipeline Template — Avoid Redundancy and Enable Reusability

.NET Pipeline in Azure DevOps using Pipeline Templates

In this blog, we will be creating a CI Pipeline for .NET application in Azure DevOps. We will also add the steps for scanning the code for security vulnerabilities in the pipeline as part of the security of applications.

Azure DevOps Pipelines

Azure DevOps Pipelines allows us to create a CI/CD Pipelines that can build, test and deploy the applications. Azure Pipelines gives us the feature to automate the entire CI/CD process.

Azure DevOps has traditional release pipelines that are UI based and every action is based on drag and drop. This approach has some advantages as it is UI based, so is easy to configure.

Recently, Azure introduced YAML Pipelines that followed Pipelines as Code approach. This approach allowed us to create pipelines via code. It’s a declarative approach where we declare each component of a pipeline in a YAML file.

Advantages of YAML Pipeline

1. YAML Pipeline is based on Pipeline as code that gives us all the benefits of “something as code” strategy.
2. Based on DRY Principle. Write once and reference at many places.
3. Version controlled so each changes in the pipeline code is tracked.
4. Easy to Migrate. Just copy the pipeline yaml and re-use it in other projects/repos/teams.
5. Provides security against accidental deletion. If pipeline gets removed due to any reasons, then bringing a new pipeline based on YAML can be done within a minute.

Pipeline Templates in Azure DevOps

Templates helps us in reusing a piece of code multiple times at different places. If we have multiple pipelines that have common steps and task, then we can create a pipeline template in Azure DevOps and re-use the template to create the pipelines.

.NET CI Template

In this blog, we we will create an Azure Pipeline template that will build and test a .NET application.

The .NET CI Pipeline will consist of following steps-

1. Restore NuGet packages
2. Build
3. Run Unit Tests
4. Calculate code coverage
5. Generate coverage report
6. Fail the build if code coverage is below certain threshold.
7. Package artifact
8. Publish artifacts

The code scanning process will consist of two steps-

1. Whitesource Scan- To check for security vulnerabilities in packages and libraries.
2. Credential Scanner (CredScan)- To check for any credentials in the source code.

Template Code

.NET CI Pipeline Template

In the above code, we have-

1. Declared two parameter that is location of Solution file and buildConfiguration. We need to supply these values when referencing this template.
2. We have set the code coverage threshold at 80 (line 67). You can configure this number according to your requirement.
3. Apart from executing .NET build and test steps, we are also importing templates for whitesource scan(line 86) and template for running credential scanner(line 88).

Credential Scanner

Credential Scanner is a part of Microsoft Security Code Analysis platform and is used to scan the code for any credentials/secrets/certificates present in the code.

Below is the template for running credential scanner. The yaml template has 4 tasks-

1. credscan- For scanning the code
2. SdtReport- To generate the scan report
3. PublishSecurityAnalysisLogs- To store the generated logs
4. PostAnalysis- To break the build if a credential/secrets is detected in repo.

We need to run this scan as part of the CI Process. So this template is getting referenced in the above .NET CI pipeline template at line 88.

Template to run credential scanner in pipelines

Whitesource Code Scanner

WhiteSource is a platform designed to automate open source security and compliance processes. It can scan for vulnerabilities in third party packages/applications. It generates a report which shows the list of high/medium/low vulnerabilities present in application.

We need to run the whitesource as part of the CI Process. So this template is getting referenced in .NET CI pipeline template at line 86.

Template to run Whitesource scan

Pipeline Variables in Template File

We can also define the variables that are used in templates in a separate variable template file. In the above templates, you can see that some tasks have condition parameter. Before executing the tasks, the condition is evaluated. If that condition is satisfied, then only the task is executed, otherwise it is skipped.

You can see that in conditions parameter, we are referencing variables . This is coming from a separate template file that contains all the variables.

We have created a template file that contains the variables being used in CI Template.

Template containing Variables

In the above examples, apart from static variables, we are also referring to variable groups of Azure DevOps. As shown above, have created 3 variable groups that gets referenced according to the branch that will trigger the pipeline.

Creating a Pipeline with these Templates

Now, we have the required the templates to create a CI Pipeline now. This pipeline will reference the standard templates that we have created in above steps.

CI YAML Pipeline

As shown in above code, we are doing the following things-

1. Triggering this Pipeline for develop, staging and main branch(line 1–6).
2. Modifying the description of a pipeline run. This feature is useful when you want custom description for each pipeline run(line-8).
3. Declaring variables in pipeline by importing the variable template file (line 10–11). We have already created a variable template file above.
4. Defining a BuildandPackage stage that references the .NET CI template that we created above (line 13–20).
5. In BuildandPackage step, we are also specifying the parameter values that is required by the .NET CI template (line 9–10).

Defining the Architecture

Above image defines the process that is happening in running the pipelines. 1) We have an actual yaml CI pipeline file that is referencing the variable from a template and the .NET CI steps from another template file.

2) And then the .NET CI template is referencing the whitesource and credscan templates as a step in pipeline.

Deployment Of Code

Till now, we have seen the CI part of the pipeline where we are building, testing and scanning the code. We can also deploy the code by writing a template that deploys the code on a particular service.

For example- If you are deploying on Azure Webapp, then you can write a deployment template for Azure Webapps. Similarly, if its azure functions then one can also write a separate template that deploys on Azure Functions.

Once you write the deployment template, one can reference the template in the main pipeline file.

Conclusion

1. In this blog, we went through the features and functionalities of YAML based pipelines in Azure DevOps.
2. Converting the YAML Pipelines into templates can help us to reduce redundancy by reusing the templates.
3. We can reference multiple templates in a single YAML pipeline.
4. We can write a deploy template and reference in the same pipeline.