API Security Audit

The starting point for the API security is the API definition itself. If the API definition has gaping security holes, applying security measures on top of that just creates a ticking time bomb. The first step is to properly specify in your API definition the security constraints that an API consumer must conform to so that it can consume the API. This is where auditing the security of your API steps in.

Whenever you import an OpenAPI (formerly known as Swagger) definition into the 42Crunch Platform, API Security Audit automatically performs a static analysis on the API definition. Your API is audited against the OpenAPI Specification (OAS) to check that the definition adheres to the specification and to catch any security issues your API might contain. The results clearly indicate the issues found and their respective severity levels, both when listing the APIs in a collection and in the audit report, so you can prioritize in which order to start fixing things.

What you can audit

Both OpenAPI Specification v2 and v3 are supported. The file size of your API should not exceed 10 MB.

For any given schema in your OpenAPI definition, Security Audit follows and checks the first six consecutive JSON references to other schemas before it calculates the final score for the schema it is currently evaluating. Any further references in that schema are ignored. This does not mean that Security Audit would not evaluate all schemas in your OpenAPI definition: the audit will in turn check each and every schema you have defined in the API, so you do not have to worry that any potential issues would not be discovered. The limit to chained schema references is merely to avoid excessive loops between the schemas when calculating the audit score.

What the audit checks

Security Audit performs over 200 checks on your API contract, ranging from its structure and semantics to its security and input and output data definition. Security Audit reviews your API definition on three levels:

OpenAPI format: Is your API a valid and well-formed OpenAPI file, and does it follow the best practices and the spirit of the OpenAPI Specification? Can it be correctly parsed, reviewed, or protected?
Security: How good are the security definitions in your API? Have you defined authentication and authorization methods, and is your chosen protocol secure enough?
Data validation: What is the data definition quality of your API? How well have you defined what data your API accepts as input or can include in the output it produces, and how strong are the schemas you have defined for your API and its parameters?

Data validation and security definitions are checked both on the global path level (affecting the whole API) as well as on operation level in individual operations.

Inadequate data validation is the most common attack vector in API security. It is very important to properly restrict what gets passed to your API and backend server and what your API can pass back to API consumers. This is reflected in Security Audit: in terms of numbers, checks on data definition quality form the biggest part of the audit. Data validation also contributes more to the audit score.

Audit report

When Security Audit finishes, you get a detailed report of the issues the audit found in your API. The report summary shows:

What kind of issues the audit found in your API
How many times the issues occurred
What is the distribution of the found issues in terms of their severity and category
How many points the issues take away from your audit score (and consequently add most points to your audit score if fixed)

Screenshot of an audit report

You can filter which issues are shown on the report page to home in on particular cases. If you click on an issue, you can view more details on it, such as where it occurred in your API, why it is a problem, and how to fix it. You can also jump directly to Security Editor to fix it in your API and rerun the audit to see the improvement immediately.

Each occurrence of an issue shows a unique fingerprint (identifier). This fingerprint remains the same in subsequent audits until the issue has been fixed, meaning that it can be used to track fixing of the issues.

The screenshot shows a particular audit issue that has occurred in four places in the API. Each occurrence shows the path and operation where it happened as well as the fingerprint of each occurrence underneath it.

The report shows the impact of each issue is, so you can prioritize what to fix first. The list of found issues shows how many points each issue deducted from the audit score of the API. If the issue occurred multiple times, the total impact from all occurrences is shown. Fixing the issues with the biggest impact on the score is the fastest way to a better audit score.

If an issue keeps recurring in multiple places in your API, only the first 30 occurrences of it are shown in detail to avoid cluttering the report up. The rest of the occurrences of the same issue are included in the report on subsequent audits as you fix the ones already reported.

If your API has structural or semantic issues, it is not a valid OpenAPI definition. The API validation fails and you do not get a full audit report until you have fixed these issues. In addition, you cannot proceed to scan or protect your API as long as its structure or semantics does not conform to the OAS.

You can download a copy of the report, for example, to share the results outside the platform.

Audit score

Security Audit calculates an audit score for each API it analyzes, based on the annotations in the OpenAPI definition. Security analysis on the authentication, authorization, and transport of data, and the data definition quality (data validation) in your API definition reveal direct security risks to your API.

Example audit score

Each API definition gets an initial pool of 100 points, split between the two categories of security risks as follows:

Security analysis: max. 30 points
Data validation: max. 70 points

Example screenshot showing the security score and data validation score that make up the global score

During the audit, each security risk that Security Audit finds in the API definition takes away points according to the impact of the found issue, reducing the audit score of the API. In other words, the more points an API definition has, the better and more secure it is.

Security Audit uses a special algorithm to calculate the impact that different security risks from security schemes or requirements or poor data validation have on the audit score. Each risk has a base number of how many points they take away from the audit score. This starting point indicates how bad the issue itself is.

For your convenience, the audit score is shown as a rounded value, but if you hover on the score, you can view the exact value.

But Security Audit also considers the sensitivity level of the whole API or API operation. This reflects the level of risk involved with the API operation or the data it handles, and sets a severity multiplier for how risks are weighed in the algorithm: the higher the sensitivity, the more severe the issue and the more weight it gets. By default, sensitivity is considered to be medium, but you can tweak this directly in your OpenAPI definition if needed (see Customizations for Security Audit).

The audit score of your API definition affects API Protection. API Protection creates an allowlist of the valid operations and input data based on the API contract, and API Firewall enforces this configuration to all transactions, incoming requests as well as outgoing responses. If the audit score is too low, the security in your API definition is not yet good enough for a reliable allowlist. Security Audit should give your API 70 points or more before you can reliably protect it.

To improve the quality and security of your API, and to increase your audit score, you must fix reported issues and re-run Security Audit. For more details on fixing the issues, see Security Editor and extensions for third-party editors.

You can also integrate Security Audit with your CI/CD pipeline so that any changes to APIs in your project are automatically audited for security. For more details, see CI/CD integration.

You can see how the audit score of the API has evolved over time in the Security Audit trends chart on the API summary page:

An example screenshot showing trends in Security Audit for an API.

Multiple security risks in a single security requirement

APIs often define several different security schemes to accommodate for different authentication methods. These can then be combined in different ways in the security requirements that apply the security measures, either to the whole API or to a particular API operation. This in turn means that Security Audit can raise multiple issues in a single security requirement in your API definition.

If you have a single authentication method in a security requirement and Security Audit raises an issue on it, the impact to the audit score is clear and direct. However, if you have defined multiple authentication methods in the security requirement, bit more calculation is needed. In this case, the relative security of different authentication methods changes how the issues impact of the audit score.

In security, the most severe risk is the biggest concern. If the audit finds multiple security risks of varying severity in a single security requirement, it is the least secure authentication methods that pose the biggest risk to your API. Thus, only the methods deemed as the biggest risks affect the security grade and audit score.

The more secure authentication methods provide more protection even if they are not perfect. Issues found in them are raised as per usual, but they do not affect the audit score until the more severe issues found in the same security requirement are fixed. On subsequent audits, the impact of the less severe risks rolls in as the higher level risks get fixed.

When Security Audit finds security risks in a security requirement that applies multiple security schemes, it first checks the sensitivity of the API or the API operation, depending on which level the security requirement has been defined. Based on that, it then checks the severity of each issue found, and rates the different authentication methods (like OAuth 2.0, the various kinds of HTTP authentication, API keys) based on how secure they are on a scale of 1—5, from not severe and most secure to critical and least secure. Once Security Audit knows which issues are the most immediate concern, it can the calculate the impact on the audit score.

Example

Imagine your API definition only allows HTTPS connections and has two operations that both have multiple security requirements defined. You are using a secure OAuth 2.0 flow in your OAuth2 scheme. The operation /Cat is sensitive, while the operation /Dog is not sensitive at all. You have used the x-42c-sensitivity extension to tell Security Audit that these operations should not be considered equal:

{
    "servers": [
            {
                "url": "https://my.api.server.com/",
                "description": "API server"
            }
        ],
        // ...
    "paths": {
        "/Cat": {
            "x-42c-sensitivity" : 4,
            // ...
            "security" : [
                { "BasicAuth": [] },
                { "Oauth2": ["readOnly"], "BasicAuth" },
                { "apiKey1": [],"apiKey2": []  }
            ],
            // ...
        }
        "/Dog": {
            "x-42c-sensitivity" : 1,
            // ...
                "security" : [
                    { "BasicAuth": [] },
                    { "Oauth2": ["readOnly"], "BasicAuth" },
                    { "apiKey1": [],"apiKey2": []  }
                ]
                // ...
        }
    }
}

Security Audit checks the security requirements, what issues should be raised for them, and what is their severity. Only HTTPS is allowed, so no issues on cleartext are raised. The operation /Dog is marked as not sensitive, so Security Audit raises no issues on it. For the operation /Cat, Security Audit notes the added sensitivity from x-42c-sensitivity and gives the following rating:

BasicAuth: issue severity is 3
OAuth2: issue severity is 1
apiKey1 and apiKey2: issue severity is 3

{
    "/Cat": {
        "x-42c-sensitivity" : 4,
        // ...
        "security" : [
            { "BasicAuth": [] }, -->3
            { "Oauth2": ["readOnly"], "BasicAuth" }, --> 1,3
            { "apiKey1": [],"apiKey2": []  } --> 3, 3
        ],
        // ...
    } 
    "/Dog": {
        "x-42c-sensitivity" : 1,
        // ...
            "security" : [
                { "BasicAuth": [] }, --> 1
                { "Oauth2": ["readOnly"], "BasicAuth" }, --> 1,1
                { "apiKey1": [],"apiKey2": []  } --> 1,1
            ]
            // ...
    }
}

Security Audit ignores BasicAuth that is applied together with OAuth2, because the more secure OAuth2 provides protection for it:

{
    "/Cat": {
        "x-42c-sensitivity" : 4,
        // ...
        "security" : [
            { "BasicAuth": [] }, --> 3
            { "Oauth2": ["readOnly"], "BasicAuth" }, --> 1, 3
            { "apiKey1": [],"apiKey2": []  } --> 3, 3
        ]
    }
}

OAuth 2.0 over HTTPS uses a good flow and is secure, so Security Audit does not raise any issues for it:

{
    "/Cat": {
        "x-42c-sensitivity" : 4,
        // ...
        "security" : [
            { "BasicAuth": [] }, --> 3
            { "Oauth2": ["readOnly"], "BasicAuth" }, --> 1
            { "apiKey1": [],"apiKey2": []  } --> 3, 3
        ]
    }
}

This means that only the three security requirements in the operation /Cat rated at severity 3 will affect the audit score. Now, Security Audit is ready to calculate the score impact.

The maximum points for security is 30 points. Security risks in more sensitive operations pose a bigger threat, so they get a bigger share of the total pot, but the maximum grade is divided between all operations in the API. So even though the operation /Dog did not raise any issues and does not affect the audit score at all, it does have a role to play in the calculations.

In this example, the potential points are allocated to the two operations as follows:

/Cat counts 24 points towards the maximum of 30
/Dog counts 6 points

Now that Security Audit knows how many points the operation /Cat could have gotten, it can count how many points the found issues take away from the audit score. In the end, the three issues found reduce the score for security and the audit score of your API by 18 points to 12/30.

You decide to remove the lone BasicAuth because you realize that you do not actually need it:

{
    "/Cat": {
        "x-42c-sensitivity" : 4,
        // ...
        "security" : [
            { "BasicAuth": [] }, --> 3
            { "Oauth2": ["readOnly"], "BasicAuth" }, --> 1
            { "apiKey1": [],"apiKey2": []  } --> 3, 3
        ]
    }
}

However, this is not yet enough to increase your audit score, because you still have the issues from apiKey1 and apiKey2 left. The found severity 3 issues still take away 18 points as before, but they are now reallocated between the two remaining issues. Only after you have fixed the issues with apiKey1 and apiKey2 does the severity of the found issues raise to 1 from the OAuth2 and your audit score increases:

{
    "/Cat": {
        "x-42c-sensitivity" : 4,
        // ...
        "security" : [
            { "BasicAuth": [] }, --> 3
            { "Oauth2": ["readOnly"], "BasicAuth" }, --> 1
            { "apiKey1": [],"apiKey2": []  } --> 3, 3
        ]
    }
}

Customizations for Security Audit

Not all APIs and API operations are equal, so one size does not fit all. It might be an overkill to require the strictest security from an API that does not handle sensitive data. This also applies on operation-level, an operation listing ATM locations does not require same level of security as, say, payment operations.

Customizations let you enrich your OpenAPI definitions with additional information on how they should be handled during audit, for example:

Declare that your API uses mutual TLS (mTLS) authentication
Allow empty security requirements (security: []), to indicate where authentication is not needed
Switch off authentication checks
Define the sensitivity of an operation

Because the customizations for audit use x-42c vendor extensions to the OpenAPI Specification (OAS), they can be done in two ways:

Define customizations as audit rules in 42Crunch Platform. The extensions are wrapped into a directives object that is then applied to your API definitions in 42Crunch Platform on the global level when they are tagged with the associated category:tag pair. When the audit rule takes effect, subsequent audits on the tagged APIs follows the specified directives. This is the recommended way.
Define customizations locally by manually adding the x-42c extensions directly to the specific objects in your API definition. The extensions take effect the next time the API definition is audited. This may be a good way to try different extensions, but is much more difficult to manage and maintain.

What is...

Security Editor and extensions for third-party editors

APIs and API collections

API Conformance Scan

Customizations

Security quality gates