API Conformance Scan settings

When you create a scan configuration for running Conformance Scan, there are various settings that you can choose to configure. Configuring these settings is entirely optional and not needed in most cases, the default settings that Conformance Scan uses are usually enough. However, for more advanced use, the settings let you tweak some aspects of the scan, such as memory limits.

We have introduced a new version of Conformance Scan, referred to as Scan v2. For backward compatibility and to avoid the adoption of the new version disrupting your day-to-day work, we have retained the previous version, Scan v1, and you can choose to continue to use it until June 2024. Both versions of Conformance Scan share the same core features and operation, but the new Scan v2 offers additional features and more flexibility. Where applicable, the difference between the versions has been clearly indicated in this documentation.

You can run Scan v1 in 42Crunch Platform or on premises as a Docker image. Scan v2 is currently available only as Docker image for on-premises scan. Scan v2 does not yet support customization rules.

Here you can find the full list of settings that you can configure for Conformance Scan and how they affect the scan operation. The settings have been grouped according to the scan version (do they apply to Scan v1 or Scan v2) or where the scan is run (on premises).

The property names given in parentheses after the setting name indicate how the setting is called in the Scan v2 scan configuration file.

Settings for all scan types

The following settings are applicable to both Scan v1 and Scan v2, both when run in 42Crunch Platform or on premises.

Setting	Description	Default value
Flow rate (`requestFlowrate`)	How long the scan waits (in milliseconds) before it sends the next request to the API. This setting defines the constant flow rate that the scan uses.	`100`
Follow redirects (`responseFollowRedirection`)	Define if the scan follows redirects (`HTTP 3XX`) it receives as responses to the tests requests: If switched off (`false`, recommended), the scan does not follow redirects to the final response, but analyzes the received redirect instead for response conformity. If switched on (`true`, not recommended), the scan follows redirects and analyzes the final response it receives, which could lead to the scan not completing successfully but ending with an error if the final response is in a unsuitable format. Maximum of 10 successive redirections are allowed before the scan stops.	`false`
Log level (`logLevel`)	The lowest severity level of log messages included in the scan logs. The possible values are, from most verbose to the least, `debug`, `info`, `error`, and `critical`. This limit helps to control how much detail you have in the logs. Only `error` and `critical` level scan logs are uploaded to 42Crunch Platform. Other log levels are written as standard output (STDOUT) to console, and how you can retrieve them depends on the environment (like Docker or Kubernetes) where you are running Conformance Scan on-premises. For more details, see the documentation of your environment. You can also mount a volume when you run the on-premises scan and write the logs to file. Normally, the log level is set in the scan configuration, but when running the scan on premises, you can set it in your Docker command, for example, to simply check how different levels affect the logs. What you need to add to your Docker command depends on which scan version you are using: Scan v1: `-logger='<log level>'`, for example: docker run -v=<replace with your local filepath>:/opt/scand/log -e SCAN_TOKEN=<your scan token> -e PLATFORM_SERVICE=services.<your hostname>:8001 /scand-agent:latest -logger='info' Scan v2: `-e SCAN42C_LOG_LEVEL='<log level>'`, for example: docker run -e SCAN_TOKEN=<your scan token> -e PLATFORM_SERVICE=services.42crunch.com:8001 -e SCAN42C_SECURITY_OAUTH2='<the token value you want to use>' 42crunch/scand-agent:v2.0.0 -e SCAN42C_LOG_LEVEL='info' If you set the log level to `debug` in when running Scan v2, all the values of all variables in your scan configuration are printed in the clear in the scan logs. Depending on your API, this can include credentials.	`error`
Maximum scan duration (`maxScanDuration`)	How long the scan runs in total (in seconds) before it stops, even if the whole API was not yet scanned. This limit ensures that the scan process will not run indefinitely even if there was an unexpected error causing a loop in the process.	`3600`

Settings for all on-premises scans

The following settings are applicable to both Scan v1 and Scan v2 when run on premises.

Setting	Description	Default value
Max consumed memory (`memoryLimit`)	How much memory (in bytes) you allow the scan process to consume. The scan stops when it reaches this value. This limit ensures that the scan process does not consume more memory than what you have available. To switch this limit off, set the value to `0`.	`2147483648`
Memory check interval (`memoryTimeSpan`)	How often (in seconds) the current memory consumption of the scan process is assessed.	`30` for Scan v1, `10` for Scan v2

Settings for Scan v2

The following settings are only applicable to the scan configuration for Scan v2.

Setting	Description	Default value
Generate cURL command (`reportGenerateCurlCommand`)	This setting controls if Conformance Scan includes the corresponding cURL request for each conformance request in the scan report. If you want to restrict the file size of your scan report, you can switch this to `false` to leave the cURL requests out of the scan report.	`true`
Happy path only (`happyPathOnly`)	This setting controls which requests are included in the scan. By default, Conformance Scan first sends happy path requests to establish the baseline for the normal API behavior, then the actual test requests (conformance requests) to detect any problems in how the API implementation conforms to the API definition. If set to `true`, Conformance Scan only runs the happy path requests during the scan before stopping, instead of a full scan.	`false`
Include only issues in the report (`reportIssuesOnly`)	This setting determines what Conformance Scan includes in the scan report: If set to `true`, only test requests that uncovered problems in your API implementation are included in the scan report, to avoid overly large report size. If set to `false`, both requests that uncovered problems and the ones that did not (where all was good) are included in the scan report, significantly increasing the report size. In Scan v1, you can control this with a flag in your Docker command, see Change the scan report scope.	`false`
Include request body in the report (`reportIncludeRequestBody`)	This setting controls if Conformance Scan includes the request bodies of the tests that it sends into the scan report: If set to `true`, all request bodies are included in the report. Be careful if you decide to include sent request bodies in the scan reports: depending on the request, they could include sensitive information, such as credentials. If set to `false`, all request bodies are excluded from the report, and you will not see the raw HTTP or the cURL command when viewing the issue details. This allows you to efficiently restrict the report size if you cannot scale up the required resources for the scan. This setting is not included in the default configuration, but you can add it manually. The setting is only supported in the Scan v2 Docker image `v2.0.3` or higher.	`true`
Include response body in the report (`reportIncludeResponseBody`)	This setting controls if Conformance Scan includes the response bodies that it receives from the API into the scan report: If set to `true`, all response bodies are included in the report. Be careful if you decide to include received response bodies in the scan reports: depending on your API, they could include sensitive or personally identifiable information, such as user details. If set to `false`, all response bodies are excluded from the report, and you will not see the raw HTTP when viewing the issue details. This allows you to efficiently restrict the report size if you cannot scale up the required resources for the scan. This setting is not included in the default configuration, but you can add it manually. The setting is only supported in the Scan v2 Docker image `v2.0.3` or higher.	`true`
Log destination (`logDestination`)	The destination where Conformance Scan pushes all logs that it produces during a scan. The possible values are `stdout`, `files`, and `platform`. You can select multiple outputs by adding the character `+` between the values.	`stdout+platform`
Max log file size (`logMaxFileSize`)	The maximum size (in bytes) allowed for the log file that Conformance Scan produces when the log destination includes `files`.	`2147483648`
Max reported issues (`reportMaxIssues`)	The maximum number of problems that Conformance Scan can uncover and include in the scan report before it must stop. This setting helps to control the report size: you can iterate on scanning your API, and as you fix the problems found in previous scans, the scan keeps including subsequently discovered problems.	`1000`
Max report size (`reportMaxSize`)	The maximum scan report size (in bytes). The scan stops when the size limit is reached. In Scan v1, you can control this with a flag in your Docker command, see Restrict the scan report size.	`20971520`
Max size for a response body (`responseMaxBodySizeScan`)	The maximum body size (in bytes) that Conformance Scan consumes from an API response, to avoid massive memory consumption.	`10485760`
Max stored happy path request size (`reportMaxRequestSizeHappyPath`)	The maximum size (in bytes) of the HTTP requests that the scan sends for a happy path request that is included in the scan report. This setting is not included in the default configuration, but you can add it manually. The setting is only supported in the Scan v2 Docker image `v2.0.3` or higher.	`8092`
Max stored happy path response size (`reportMaxHttpResponseSizeHappyPath`)	The maximum size (in bytes) of the HTTP responses received for a happy path request that is included in the scan report.	`8092`
Max stored response body size for happy path requests (`reportMaxBodySizeHappyPath`)	The maximum response body size (in bytes) that the scan includes in the scan report for a happy path request.	`8092`
Max stored response body size for test requests (`reportMaxBodySizeTest`)	The maximum response body size (in bytes) that the scan includes in the scan report for a test request.	`8092`
Max stored request size for test requests (`reportMaxRequestSizeTest`)	The maximum size (in bytes) of the HTTP requests that the scan sends for a test request that is included in the scan report. This setting is not included in the default configuration, but you can add it manually. The setting is only supported in the Scan v2 Docker image `v2.0.3` or higher.	`8092`
Max stored response size for test requests (`reportMaxHttpResponseSizeTest`)	The maximum size (in bytes) of the HTTP responses received for a test request that is included in the scan report.	`8092`
Scan testing mode (`laxTestingModeEnabled`)	Scan testing mode changes how Conformance Scan deals with failed happy path requests: If set to `false`, Conformance Scan runs in normal testing mode, meaning that conformance tests are not run on API operations where the happy path request fails. This is the recommended option. If set to `true`, Conformance Scan runs in lax testing mode, meaning that all tests are run on all API operations even if the happy path request had failed. This allows running more tests in the scan, but the scan results may not be as reliable as in the normal, where all tests can be compared against an established baseline. Lax scan testing mode is only supported in the Scan v2 Docker image `v2.0.6` or higher.	`false`
Scenario ID header (`requestHeaderNameScenarioId`)	The name of the header that contains the scenario ID of the request that Conformance Scan sends.	`x-scan-scenario-id`
Skip TLS verification (`requestTlsInsecureSkipVerify`)	This setting controls how Conformance Scan handles certificates in TLS verification: If set to `true`, Conformance Scan accepts any certificate that the server presents and any host name in that certificate. If set to `false`, Conformance Scan only accepts the certificate that the server presents if the certificate and its host name can be verified through the chain of Trusted CA. This can pose problems, for example, if an API in your internal environment uses a certificate from your own internal CA. In this case, the scan fails because Conformance Scan cannot trust the presented certificate as it cannot validate the chain of trust, and the TLS authentication fails.	`true`
Transaction ID header (`requestHeaderNameRequestId`)	The name of the header that contains the request ID of the request that Conformance Scan sends.	`x-scan-request-id`
Timeout (`requestTimeout`)	The maximum time (in seconds) that the scan waits for a response from the API. We do not recommend setting the timeout value to `0`, because this switches the timeout off completely. This means that if something goes wrong, the scan might run indefinitely. In Scan v1, you can control this with a flag in your Docker command, see Specify longer timeout for the on-premises scan.	`30`

Settings for Scan v1

The following settings are only applicable to the scan configuration for Scan v1, both when run in 42Crunch Platform or on premises.

Setting Description Default value

Setting	Description	Default value
Connectivity `GET` call	By default, Scan v1 sends a request `GET /` to check the connectivity to the API before it sends happy path requests. This avoids trying to run the scan on an API that it cannot connect to. However, sometimes this is not supported by the API implementation and the scan fails to start. In this case, you can use this setting to switch off the connectivity check completely to avoid the false negative on the connectivity because the API implementation did not respond as Conformance Scan expected.	`On`
JSON complexity check	Is the complexity of the JSON checked or not. This check safeguards against overly complex API definition that could lead the scan taking up too long and consuming too much system resources.	`Off`

Connectivity GET call

By default, Scan v1 sends a request GET / to check the connectivity to the API before it sends happy path requests. This avoids trying to run the scan on an API that it cannot connect to.

However, sometimes this is not supported by the API implementation and the scan fails to start. In this case, you can use this setting to switch off the connectivity check completely to avoid the false negative on the connectivity because the API implementation did not respond as Conformance Scan expected.

On

JSON complexity check

Is the complexity of the JSON checked or not. This check safeguards against overly complex API definition that could lead the scan taking up too long and consuming too much system resources.

Off

What is...

API Conformance Scan

API Security Audit

Customizations

Scan configuration details

How to...

Scan API conformance

Use Conformance Scan v1

Use Conformance Scan v2

Audit API security

Fix APIs