API Conformance Scan settings

The lowest severity level of log messages included in the scan logs. The possible values are, from most verbose to the least, debug, info, error, and critical. This limit helps to control how much detail you have in the logs.

Only error and critical level scan logs are uploaded to 42Crunch Platform. Other log levels are written as standard output (STDOUT) to console, and how you can retrieve them depends on the environment (like Docker or Kubernetes) where you are running Conformance Scan on-premises. For more details, see the documentation of your environment. You can also mount a volume when you run the on-premises scan and write the logs to file.

Normally, the log level is set in the scan configuration, but when running the scan on premises, you can set it in your Docker command, for example, to simply check how different levels affect the logs. What you need to add to your Docker command depends on which scan version you are using:

• Scan v1: -logger='<log level>', for example:

  docker run -v=<replace with your local filepath>:/opt/scand/log -e SCAN_TOKEN=<your scan token> -e PLATFORM_SERVICE=services.<your hostname>:8001 /scand-agent:latest -logger='info'

• Scan v2: -e SCAN42C_LOG_LEVEL='<log level>', for example:

  docker run -e SCAN_TOKEN=<your scan token> -e PLATFORM_SERVICE=services.42crunch.com:8001 -e SCAN42C_SECURITY_OAUTH2='<the token value you want to use>' 42crunch/scand-agent:v2.0.0 -e SCAN42C_LOG_LEVEL='info'

If you set the log level to debug when running Scan v2, all the values of all variables in your scan configuration are printed in the clear in the scan logs. Depending on your API, this can include credentials.

Define if the scan follows redirects (HTTP 3XX) it receives as responses to the tests requests:

• If switched off (false, recommended), the scan does not follow redirects to the final response, but analyzes the received redirect instead for response conformity.
• If switched on (true, not recommended), the scan follows redirects and analyzes the final response it receives, which could lead to the scan not completing successfully but ending with an error if the final response is in a unsuitable format. Maximum of 10 successive redirections are allowed before the scan stops.

This setting determines what Conformance Scan includes in the scan report:

• If set to true, only test requests that uncovered problems in your API implementation are included in the scan report, to avoid overly large report size.
• If set to false, both requests that uncovered problems and the ones that did not (where all was good) are included in the scan report, significantly increasing the report size.

In Scan v1, you can control this with a flag in your Docker command, see Change the scan report scope.

The maximum scan report size (in bytes). The scan stops when the size limit is reached.

In Scan v1, you can control this with a flag in your Docker command, see Change the scan report size limit.

If you run Conformance Scan in 42Crunch Platform, you cannot change maximum scan report size. However, when running the scan on premises, you can increase the maximum report size up to 128 MB.

Large scan reports can take a long time to load in your browser. Depending on your browser memory limits and your device, if your scan report size exceeds 20 MB, you may not be able to view the scan report correctly on the platform UI. However, larger reports can be viewed in IDE. For more details on running Conformance Scan in IDE, see IDE integration.

We do not recommend setting the timeout value to 0, because this switches the timeout off completely. This means that if something goes wrong, the scan might run indefinitely.

In Scan v1, you can control this with a flag in your Docker command, see Specify longer timeout for the on-premises scan.

The setting controls that Conformance Scan checks connectivity to API when running the scan. If connectivity is lost, the scan stops. If you set log level to debug, you can see the end state in scan logs.

This setting is only supported in the Scan v2 Docker image v2.0.8 or higher.

This setting controls which requests are included in the scan.

By default, Conformance Scan first sends happy path requests to establish the baseline for the normal API behavior, then the actual test requests (conformance requests) to detect any problems in how the API implementation conforms to the API definition.

If set to true, Conformance Scan only runs the happy path requests during the scan before stopping, instead of a full scan.

Scan testing mode changes how Conformance Scan deals with failed happy path requests:

• If set to false, Conformance Scan runs in normal testing mode, meaning that conformance tests are not run on API operations where the happy path request fails. This is the recommended option.
• If set to true, Conformance Scan runs in lax testing mode, meaning that all tests are run on all API operations even if the happy path request had failed. This allows running more tests in the scan, but the scan results may not be as reliable as in the normal, where all tests can be compared against an established baseline.

Lax scan testing mode is only supported in the Scan v2 Docker image v2.0.6 or higher.

This setting controls how many times Conformance Scan retries sending a request that times out. If the request continues to time out when the limit is reached, the scan stops.

This setting is only supported in the Scan v2 Docker image v2.0.8 or higher.

This setting controls if Conformance Scan includes the request bodies of the tests that it sends into the scan report:

• If set to true, all request bodies are included in the report.

  Be careful if you decide to include sent request bodies in the scan reports: depending on the request, they could include sensitive information, such as credentials.

• If set to false, all request bodies are excluded from the report, and you will not see the raw HTTP or the cURL command when viewing the issue details.

This allows you to efficiently restrict the report size if you cannot scale up the required resources for the scan.

This setting is only supported in the Scan v2 Docker image v2.0.3 or higher.

This setting controls if Conformance Scan includes the response bodies that it receives from the API into the scan report:

• If set to true, all response bodies are included in the report.

  Be careful if you decide to include received response bodies in the scan reports: depending on your API, they could include sensitive or personally identifiable information, such as user details.

• If set to false, all response bodies are excluded from the report, and you will not see the raw HTTP when viewing the issue details.

This allows you to efficiently restrict the report size if you cannot scale up the required resources for the scan.

This setting is only supported in the Scan v2 Docker image v2.0.3 or higher.

The name of the header that states the type (happy path, unhappy path, conformance, authentication...) of the request that Conformance Scan sends.

This setting is only supported in the Scan v2 Docker image v2.0.8 or higher.

This setting controls how Conformance Scan handles certificates in TLS verification:

• If set to true, Conformance Scan accepts any certificate that the server presents and any host name in that certificate.
• If set to false, Conformance Scan only accepts the certificate that the server presents if the certificate and its host name can be verified through the chain of Trusted CA. This can pose problems, for example, if an API in your internal environment uses a certificate from your own internal CA. In this case, the scan fails because Conformance Scan cannot trust the presented certificate as it cannot validate the chain of trust, and the TLS authentication fails.

By default, Scan v1 sends a request GET / to check the connectivity to the API before it sends happy path requests. This avoids trying to run the scan on an API that it cannot connect to.

However, sometimes this is not supported by the API implementation and the scan fails to start. In this case, you can use this setting to switch off the connectivity check completely to avoid the false negative on the connectivity because the API implementation did not respond as Conformance Scan expected.

Is the complexity of the JSON checked or not. This check safeguards against overly complex API definition that could lead the scan taking up too long and consuming too much system resources.