Security quality gates

How security quality gates work

Organization administrators can create SQGs and set the baseline for specific quality criteria that APIs in their organization must reach. As part of performing an action like auditing the API definition, APIs must also meet the criteria of the SQG.

Each organization automatically gets a default audit and scan SQGs out of the box, so that you have overall quality indicators at your disposal right from the start. You cannot delete the default SQGs, but you can edit them as needed.

Tags are the mechanism for applying additional SQGs to your APIs. The default SQGs are automatically applied to all APIs in your organization, no tagging required. For other SQGs, you define the category:tag pair that goes with it. Then, to apply the SQG to your APIs, you tag each API with the tag for the SQG you want. For example, you might want to apply different SQGs depending on the execution environment (development, pre-production, production), data sensitivity, or regulatory context (financial, healthcare) of the API.

We recommend adding a dedicated category for SQGs, so that it is easy to find the related tags. We also recommend that you do not allow users to create tags in this category, so that organization administrators stay in control of these.

Combining criteria from multiple SQGs

APIs usually have multiple SQGs applied to them: the default SQG sets the common quality criteria to be met for all APIs in your organization, while tag-based SQGs provide targeted alternatives for specific needs.

This means that there may be multiple layers of criteria that they must meet. In terms of the interplay of these criteria, there are three approaches you can choose the best for your needs:

• Ignore default SQG: SQGs applied from tags always take precedence over the default SQG: for example, if you tag your API to apply a customized SQG for Security Audit, then the default SQG for Security Audit is completely ignored.
  • If you have tagged your API for multiple SQGs, then the criteria from these tag-based SQGs are still combined.
  • This approach allows tailoring SQGs to a high level without interference from the default SQG.
  • However, criteria from a weak SQG do override default SQG so low-quality APIs may slip through.
• Combine criteria: SQGs applied from tags are combined with the default SQG, meaning that all criteria are always considered.
  • This approach ensures that the common criteria are always enforced on all APIs.
  • However, as the baseline from the common criteria is always enforced, it may interfere by blocking APIs that are early on in their life cycle, or you may need to make the common criteria more permissive.
• Targeted override: SQGs applied from tags are combined with the default SQG but you can set particular SQGs to override the common criteria from the default SQG.
  • This approach allows enforcing the common criteria an all APIs while selectively allowing ignoring them on a case-by-case basis.
  • Targeted override requires that combining SQG criteria is allowed on organization level.

Organization administrators can control how default SQGs are handled. For more details, see Manage handling of criteria from default and tag-based SQGs. You can see how SQG criteria combine and which SQG takes precedence for which criteria on the API summary page.

Approval report

When you audit or scan an API that has a SQG applied to it, 42Crunch Platform compares the results to the criteria defined in the SQG. If the criteria are met, the API passes the SQG, and your CI/CD pipeline progress can continue. If the criteria of the SQG are not met, the API fails the gate.

In both cases, you get an approval report that lists the criteria that were not met.

If the API passes the SQG, the report is an assurance on the security of the API. However, when an API fails the SQG, the approval report provides the starting point for aligning the API with the expectations of your organization. Aiming to address the failures from the approval report first could help avoid unnecessary work when fixing things.

The SQGs are grouped to approval report by report type, so you can see the audit SQG in one approval report and the scan SQG in another.

Security quality gates and CI/CD integration

SQGs are most powerful when used in your CI/CD pipeline, acting as gate keepers and stopping APIs that exceed your risk appetite entering your code base if the API does not meet the set criteria.

You must integrate API Security Audit to your CI/CD pipeline before SQGs can affect your builds. CI/CD integration for API Conformance Scan is coming, but currently scan SQGs can only be enforced in 42Crunch Platform, not on your CI/CD.

You can integrate features in 42Crunch Platform with your CI/CD solution through custom plugins that adds a build step to your pipeline, for example, to automate Security Audit from a simple Git push to your project repository. Once you have set up the integration plugin for your CI/CD pipeline, on each build the plugin automatically checks the status of the SQGs applied to the APIs it found in the repository. If any of the SQGs fails, the build automatically fails too.

The fail-on criteria you set in the CI/CD plugin are independent from the acceptance criteria defined in SQGs, and the plugin can fail even if SQGs pass. For example, CI/CD plugins by default fail APIs that do not have valid OpenAPI definitions, but in SQGs you need to specifically switch this on. This means that your CI/CD build can fail either because the criteria of a SQG or the criteria of the CI/CD plugin are not met, or both, and so just looking at SQGs to explain why a build has failed may not be enough.

For more details, see CI/CD integration.

Security quality gates and customization rules

SQGs and customization rules are not one and the same: while customization rules affect what happens within 42Crunch Platform, SQGs take effect in your CI/CD pipeline.

However, sometimes SQGs overlap with customization rules. For example, you could list a particular issue or test ID as forbidden in your SQG, but have a customization rule that excluded that particular ID. In this case, the SQG would be ineffective, because the case that the SQG safeguards against would not be checked for at all and its presence could go undetected.

We recommend that you do not apply both SQGs and customization rules to the same API, or if you do, you carefully check that they do not conflict.

Security quality gates in API Security Audit

SQGs for Security Audit help you ensure that the quality and security of all API definitions that are pushed to your code base are what your want them to be.

You can define several criteria that the OpenAPI definition must reach in the audit, such as:

• Reject OpenAPI definitions that have structural or semantic issues so that they no longer can be considered valid OpenAPI definitions by failing the SQG outright.
• Set minimum scores that OpenAPI definitions much reach.
• Define the allowed severity levels for found issues in each category and subsection. If issues of that severity or higher are found in the audit, the API fails the SQG.
• List issues (by issue IDs) that are never accepted in your APIs. If any if the listed issues is found when auditing an API, the API fails the SQG.

You can also filter your audit report to only show the things you need to fix to pass the SQG.

Security quality gates in API Conformance Scan

SQGs for Conformance Scan help you ensure that all APIs are implemented in a way that meets your requirements and risk appetite.

You can define several criteria that the API implementation must reach in the scan, such as:

• Reject API implementations that could not be scanned fully, for example, because of failing happy path requests or other errors encountered during the scan.
• List which vulnerabilities from OWASP API Security Top 10 are blockers for your API implementations.
• Define the allowed severity levels for problems that the scan finds. If problems of that severity or higher are found in the scan, the API fails the SQG.
• What percentage of the operations must return a HTTP status code that Conformance Scan expected for the test in question.
• List problems (by test IDs) that are never accepted in your API implementations. If any if the listed tests finds problems when scanning an API, the API fails the SQG.