Integrate Security Audit with GitLab Pipelines

You can integrate API Security Audit in GitLab Pipelines through a custom job REST API Static Security Testing.

You must have an account in 42Crunch Platform that the job in GitLab Pipelines can use to access Security Audit.

For more details on GitLab Pipelines, see GitLab CI/CD documentation.

Create an API token for the job

You must add an API token that the job uses to authenticate to Security Audit.

1. Log in to 42Crunch Platform, and click next to your username.
2. Select Tokens, and click Create new token.
3. Enter a unique and descriptive name for the token, such as Security Audit token.
4. Make sure the token type is API token, and in token access rights, select API Security Audit, List resources, and Delete resources.

   

5. Click Generate token.
6. Copy the token value, you will need it when you configure REST API Static Security Testing.

   

Add a protected CI/CD variable for the API token

Before you add the job to your GitLab pipeline, you must add the API token you created as a protected environment variable in your GitLab project. You must be a project maintainer to be able to create project variables.

1. Log in to GitLab, go to the project you want, and click Settings > CI/CD.
2. Under Variables, click Add variable.
3. Set the following:
   • Name: X42C_API_TOKEN (the job expects to find a token with this precise name)
   • Value: The value of the API token you created

   

4. Make sure the type is Variable and choose the environments where the you want to use it.
5. Make sure that Protect variable and Mask variable are both selected to avoid any risk for the token leaking, and click Add variable.
   

You have now created a protected CI/CD variable that the pipeline job can use to authenticate to Security Audit. You can use the same variable in multiple projects. For more details, see GitLab CI/CD variables.

Add the job to your GitLab pipeline

To run the job, you must add it to your GitLab pipeline.

1. In GitLab, go to the project you want, and click CI/CD > Editor to open Pipeline Editor.
2. Add the latest image version of REST API Static Security Testing and the shell script that the Runner in GitLab executes:

   audit:
     stage: test
     image:
       name: 42crunch/gitlab-api-security-audit:v3
     script: /audit/audit.sh

   Note that the bit denoting the version (here v3) changes with each major version of the plugin.

3. If you want, add the minimum audit score that the audited OpenAPI definitions must get from the audit for the action to succeed. If any API definitions score lower than the minimum score you set for X42C_MIN_SCORE, the job fails. The default is 75:

   audit:
     stage: test
     image:
       name: 42crunch/gitlab-api-security-audit:v3
     variables:
       X42C_MIN_SCORE: 51
     script: /audit/audit.sh

   The fail-on criteria you set in the CI/CD plugin, such as the minimum score, are independent from the acceptance criteria defined in security quality gates (SQGs). This means that your CI/CD build can fail either because the criteria of the plugin are not met, the criteria of a SQG are not met, or both.

4. If you are an enterprise customer not accessing 42Crunch Platform at https://platform.42crunch.com, add X42C_PLATFORM_URL and enter your platform URL. This step is optional and most users do not have to do this. If you are not sure what your platform URL is, contact our support.

   audit:
     stage: test
     image:
       name: 42crunch/gitlab-api-security-audit:v3
     variables:
       X42C_MIN_SCORE: 51
       X42C_PLATFORM_URL: <your platform URL here>
     script: /audit/audit.sh

5. By default, the level of detail in the logs that the job produces is INFO. If you want more or less detail, you can use X42C_LOG_LEVEL to specify the level of detail. The possible values are FATAL, ERROR, WARN, INFO, DEBUG):

   audit:
     stage: test
     image:
       name: 42crunch/gitlab-api-security-audit:v3
     variables:
       X42C_MIN_SCORE: 51
       X42C_LOG_LEVEL: DEBUG
     script: /audit/audit.sh

6. If you want to automatically share any new API collections that the job creates with other users in your organization, add the variable X42C_SHARE_EVERYONE. The variable has two possible values:
   • READ_ONLY: Other users have read-only access
   • READ_WRITE: Other users have read and write access

   If you do not define X42C_SHARE_EVERYONE at all, the API collections are not shared and remain private to you.

   audit:
     stage: test
     image:
       name: 42crunch/gitlab-api-security-audit:v3
     variables:
       X42C_MIN_SCORE: 51
       X42C_LOG_LEVEL: DEBUG
       X42C_SHARE_EVERYONE: READ_ONLY
     script: /audit/audit.sh

   You can also change the sharing of the API collections later. For more details, see Sharing APIs and access level.

7. Click Commit changes to save your changes to the pipeline. To test the job, run your pipeline.

The job will either succeed or fail depending on the minimum score. The summary of the run in the pipeline reports provides you further details on how the job went.

The job uploads all discovered OpenAPI definitions to the specified API collection in 42Crunch PlatformBy default when running on branch, the plugin uses the naming convention <shortened-source-control-uri> Branch:<branch-name> for the created API collection, for example, 42Crunch/sample Branch:sample.

The GitLab job uses the build variables CI_PROJECT_URL and CI_COMMIT_REF_NAME to get the details directly from your source control.

The API definitions in the collection show the filepaths they have in the repository:

The report of the run includes a link to each discovered API. You can click on the link to view the detailed audit report of the corresponding API in 42Crunch Platform.

Write summary of the plugin run in a file

If you want, you can set the plugin to write and store a report on the plugin run as a JSON file, so that it is easy, for example, to see and communicate the API UUIDs of the uploaded APIs.

This is not the audit report that provides details on the issues that Security Audit found in your APIs and how to remediate them, but a separate, optional summary providing some basic details on the APIs that the plugin processed. The full audit reports are not included in this summary, but are available in 42Crunch Platform.

1. Go to the project you want, and open the Pipeline Editor.
2. Add the variable X42C_JSON_REPORT to the to the audit stage, and enter the filename or path that you want to use for report file. The report is in JSON format, so ensure that you either specify .json as the file extension (as in test/myReport.json) or omit the extension completely (test/myReport).
   • If you specify only the filename, the plugin creates the file in the root directory that it is pointed to start from.
   • If you specify a path, the path must be under the root directory and all specified directories must exist. The plugin only generates the JSON file, not directories.

   The plugin does not delete or overwrite any summaries from previous runs, so if a file with the same name exists, you see an error. To avoid this, you can create a separate workspace for each pipeline pass, add a task to clear the workspace before starting, or include a variable to the filename (such as the build number or timestamp) to ensure that each filename is unique.

3. Save your changes.

Next time you run the pipeline, the integration writes a summary report as a JSON file in the location you specified. This report shows the details on discovered, such as:

• Filename
• API UUID assigned for the API when it was uploaded to 42Crunch Platform
• Audit score
• Did the pipeline task fail and if yes, why
• Any errors that occurred when processing the API

Fine-tune the plugin configuration

You can further fine-tune how the integration works by adding a configuration file called 42c-conf.yaml to the root directory of your source code repository where the CI/CD pipeline connects to. You can, for example:

• Map OpenAPI files in your repository to API UUIDs of APIs in the platform.
• Specify fail_on conditions to define what the plugin reports as failures.

  The fail-on criteria you set in the CI/CD plugin, such as the minimum score, are independent from the acceptance criteria defined in security quality gates (SQGs). This means that your CI/CD build can fail either because the criteria of the plugin are not met, the criteria of a SQG are not met, or both.

• Control what happens in the discovery phase.

You can specify different configurations for different branches, tags, or even pull requests. For more details, see the configuration examples in our Resources repository in GitHub.

Configure the integration to use HTTP or HTTPS proxy server

REST API Static Security Testing is powered by Security Audit and connects to 42Crunch Platform when it runs. This is not an issue as long as your GitLab is on a machine with direct internet access. However, if you GitLab is behind an HTTP or HTTPS proxy server, additional configuration is needed.

1. Find out and make a note of the address and port of your proxy server (for example, 192.168.0.117 and 8090).
2. In GitLab, open the pipeline you want in Pipeline Editor, and go to the audit job.
3. Depending on which proxy you want to use, add the variable HTTP_PROXY or HTTPS_PROXY, with the address and the port of the proxy server as its value.

    audit:
     stage: test
     image:
       name: /gitlab-api-security-audit:v3
     variables:
       X42C_MIN_SCORE: 51
       X42C_LOG_LEVEL: DEBUG
       X42C_SHARE_EVERYONE: READ_ONLY
       HTTP_PROXY: http://192.168.0.117:8090
     script: /audit/audit.sh

4. Click Commit changes to save your changes to the pipeline.

Your proxy server configuration is saved. Any new jobs will use it when they run, and REST API Static Security Testing will report that it is using the proxy. If there is something wrong with your proxy configuration, the plugin will fail because requests are not going through.

Change the default collection name

By default when running on branch, the plugin uses the naming convention <shortened-source-control-uri> Branch:<branch-name> for the created API collection, for example, 42Crunch/sample Branch:sample. However, you can specify a different syntax for the new collections that the plugin uses by default.

1. Go to the project you want, and open the Pipeline Editor.
2. Add the variable X42C_DEFAULT_COLLECTION_NAME to the audit stage, and enter the syntax for the default collection name you want to use. You can use text (like foo), reference to variables (${repository}), or a combination of the two (foo ${repository}). You can use the following variables that are populated from your repository details:
   • repository: The full repository URL.
   • repo_short_path: A shortened path of the repository URL (the leading / and the trailing .git are omitted).
   • repo_hostname: The hostname from the repository URL.
   • branch_info:The syntax Branch:<branch name>, or an empty string if this information is not available.
   • tag_info: The syntax Tag:<tag name>, or an empty string if this information is not available.
   • pr_info: The syntax PR:<pr id>, or an empty string if this information is not available.
3. When ready, save your changes.

Next time you run the pipeline, the integration plugin uses the syntax you defined and creates new API collections in 42Crunch Platform (if collections with the same names do not yet exist) where it loads the discovered APIs.

You can also define collection names for specific branches, tags, and pull requests using the property collection_name in the configuration file 42c-conf.yaml.

Set the root directory for the plugin

By default, the integration plugin uses the root directory of your repository as its starting point. However, you can also set a specific directory that the plugin will use as its root.

If you have configured 42c-conf.yaml for your plugin, make sure it is located in the root directory that you want the plugin to use. Otherwise, the configuration file is ignored.

1. Go to the project you want, and open the Pipeline Editor.
2. Add the variable X42C_ROOT_DIRECTORY to the to the audit stage, and enter the path of the directory you want to use as the root directory.
3. Save your changes.

Next time you run the pipeline, the integration plugin will start the discovery phase from the directory path you defined and check that directory and any subdirectories under it for OpenAPI files.

Use SQG criteria instead of plugin configuration

By default, the integration plugin configuration defines when the CI/CD task passes or fails, and there are some default values that the plugin uses if nothing else is specified. However, if you are using security quality gates (SQGs) in 42Crunch Platform for quality control, you might prefer SQGs to determine when the CI/CD task passes or fails. In this case, you can set the plugin to skip the locally defined fail-on conditions (such as minimum score, format validity, or forbidden issues) and only use those defined in the SQGs.

1. Go to the project you want, and open the Pipeline Editor.
2. Add the variable X42C_SKIP_LOCAL_CHECKS to the to the audit stage, and set it to true.
3. Save your changes.

Next time you run the pipeline, the plugin will only check the status of all SQGs applied to the APIs it found in the repository when deciding if the build passes or fails, and ignore any fail-on conditions defined in the plugin itself, including the default plugin configuration.

Stop the plugin from failing a pipeline

Sometimes you might want the CI/CD plugin just to report the found issues, not block the pipeline from continuing. For example, your repository might have plenty of APIs in early stages of their lifecycle, or you have just introduced the CI/CD plugin to the pipeline and need time to adjust to the set quality criteria. In this case, you can temporarily switch off all fail-on conditions that the CI/CD plugin would impose on a CI/CD job. The plugin keeps reporting on the discovered APIs but does not block the pipeline from proceeding to subsequent stages.

Switching off the fail-on conditions in the CI/CD plugin means that the plugin will cease to work as a quality control as it will never prevent potential problems in your APIs. We recommend that you use this option only after a careful consideration and only for a limited time. Remember to remove this setting from the plugin as soon as possible.

1. Go to the project you want, and open the Pipeline Editor.
2. Add the variable X42C_IGNORE_FAILURES to the to the action, and set it to true.
3. Save your changes.

Next time you run the pipeline, the integration plugin runs normally, uploading each discovered API definition and its audit report to 42Crunch Platform, but does not fail the CI/CD job because of the audit results or SQG status. However, the plugin still produces logs as per usual, so you can check them to see the status of the discovered APIs.

Ignore network errors

If you are worried that issues in connectivity — such as in the rare case of the CI/CD plugin not being able to communicate to 42Crunch Platform — could unduly hinder an important CI/CD pipeline by causing it to fail, you can set the plugin to ignore connectivity issues.

Setting the CI/CD plugin to ignore network errors lessens the plugin's effectivity as a quality control, because the task does not stop the CI/CD pipeline even though it could not complete successfully due to a connectivity issue. This means that APIs that have quality or security issues in their OpenAPI definitions could slip through upon a successful CI/CD job. We recommend that you use this option only after a careful consideration.

1. Go to the project you want, and open the Pipeline Editor.
2. Add the variable X42C_IGNORE_NETWORK_ERRORS to the to the audit stage, and set it to true.
3. Save your changes.

Next time you run the pipeline, if the integration plugin encounters a connectivity issue, it does not fail the CI/CD job and the pipeline proceed to subsequent steps. Errors that cannot be definitively deemed to be connectivity issues will still cause the plugin to fail the job. You can check the logs that the plugin produces to see if any errors occurred.