Use Conformance Scan v1

You can configure and run Conformance Scan on your API directly in 42Crunch Platform.

Run Conformance Scan

Conformance Scan runs the scan against the selected API endpoint. When the scan is finished, you can see the number of issues where your API implementation does not conform to the API contract. The badge above the results indicates that the scan was run in 42Crunch Platform (as opposed to on-premises) and the version of the scan engine used. If Conformance Scan could not scan one or more API operations, for example, because a happy path request failed, the scan is marked as incomplete to reflect that more work is still needed.

For the full list of found issues, click Read report.

View scan reports

You can check how many and what kind of issues Conformance Scan found in the implementation of your API.

1. Go to the API you want to scan, and click Conformance Scan .

   

2. Click on the time stamp of the report from Scan v1 run in 42Crunch Platform. You can see the list of tests the scan run and the found issues, and click on them to see more details.

   

3. Use the "Critical to Success" filter bar above the result list to filter what is being shown in the result list. You can also further refine the results with the additional dropdown filters.

   

4. Use the filters on the sidebar on the left to show issues only for a particular path or method.

   

5. To view the details of an issue and the test that was run, click the issue on the list.

   

You can also see how the number of issues that Conformance Scan found in your API has changed over time in the Conformance Scan trends chart on the API summary page.

Export scan reports

You can export a scan report from 42Crunch Platform to share with others. If you do not want the full report with all the details, you can also export just the found issues as a CSV list.

We recommend you treat the exported report as company-confidential information and store it safely because it can reveal information on security risks in your APIs to outsiders.

1. Click Conformance Scan, and go to the scan report you want to export.
2. Click Export, and choose what you want to export:
   • To export the full report, click Save report as JSON.
   • To export a report overview, click Save report as PDF.

     The overview PDF shows the current report page that you are on, so you can use the filters or export a different page to change what is included in the PDF.

   • To export just the list of all found issues as CSV, click Save issue list as CSV.

     CSV export is only supported for reports from Scan v1 v1.22.14 or higher.

Change scan settings

If you find that the default settings in the scan configuration do not work for your API, you can adjust them to better customize the scan. You cannot directly edit the existing scan configuration, but if you create a new Scan v1 configuration it will replace the existing one.

1. In 42Crunch Platform, click Scan API.
2. Configure the API endpoint, authentication, and the optional mutual TLS as usual.
3. Click Configure settings, and switch Use custom scan settings on.
4. Change the settings you want. For the full list of settings and which scan type they apply to, see API Conformance Scan settings.
5. Click Review setup, and verify the configuration, then click Start the scan.

The updated scan configuration replaces your existing Scan v1 configuration for running Conformance Scan in 42Crunch Platform, and Conformance Scan reruns the scan using the new configuration.

You can deploy and run Conformance Scan locally as a Docker image.

Create scan configuration for on-premises scan

You can have two separate scan configurations for a single API: one for running Conformance Scan on premises and another one for running it in 42Crunch Platform.

1. Find the API that you want to scan. The quickest way to find API definitions in 42Crunch Platform is to click Find API in the main menu.
2. Click Conformance Scan > Add configuration > Version 1.
3. Select the API endpoint you want to use from the list or enter the URL, and click Configure general information. For more details, see API endpoints.

   

4. For Scan v1 scan configurations, you cannot change the name of the configuration or what it is based on. Click Configure authentication.
5. Fill in the details for the security schemes you want to use in the scan. The authentication details are stored encrypted as part of the scan configuration in the platform. The authentication details are not retrievable and credentials are hidden. If your API uses mutual TLS authentication, you can also switch Use mutual TLS for client authentication on, and fill in the details for it. The client certificate must be in pkcs12 format. When ready, click Configure settings.

   

   To configure OAuth2 authentication, you must first manually obtain an access token that Conformance Scan can use to authenticate to your API. In the scan configuration wizard, authentication with OAuth token is configured like bearer token. For more details on OAuth2, see RFC 6749.

6. Review the default settings for the scan configuration. In most cases, you do not have to change these values. For the full list of settings and which scan type they apply to, see API Conformance Scan settings..
7. When you are ready, click Create scan configuration.

You have now created a scan configuration for the API that you selected, as well as a scan token that tells on-premises scan which API to scan and with what settings.

You cannot directly edit the existing scan configuration, but if you create a new Scan v1 configuration it will replace the existing one. You must reconfigure the scan before you can see the effect from some updates to Conformance Scan.

Run on-premises scan

Before you can run Conformance Scan on premises, you must have created a scan configuration for the API in 42Crunch Platform and have Docker installed. For more details on Docker, see the Docker documentation.

When the on-premises scan starts, it establishes a two-way, HTTP/2 gRPC connection to 42Crunch Platform at the address services.<your hostname> and the port 8001. Make sure that your network configuration (like your network firewall) authorizes these connections. The on-premises scan uses this connection to verify the scan token and to download the scan configuration you have created. During runtime, on-premises scan uses the connection to send the scan report and logs to the platform.

If you are an enterprise customer not accessing 42Crunch Platform at https://platform.42crunch.com, your hostname is the same one as in your platform URL.

1. Go to the API you want to scan, and click Conformance Scan.
2. Click on the Scan v1 on-premise scan configuration, and copy and run the sample Docker command shown on the General information.

   

   Scan configurations and tokens are specific to a scan version: you cannot run Scan v2 using a Scan v1 scan token, and vice versa. When running a scan, make sure you specify the right scan token for the scan version you are using, otherwise Conformance Scan cannot use the associated scan configuration and fails to run.

The on-premise scan starts and checks the scan configuration. If the scan configuration is valid, Conformance Scan runs and produces a scan report. If the scan configuration is not valid, you need to fix it before the on-premises scan can run successfully.

You only need to pull the image when you first run an on-premises scan, or if your want to update the scan image version. Because you must not run Conformance Scan in production, using latest for the version tag of the Docker image is fine.

If you are on the API summary page or in the list of scan reports on the UI when the scan finishes, you must refresh the page before the on-premises scan report is available on the UI.

Run on-premises scan behind HTTP or HTTPS proxy

In addition to connecting to the API to be scanned, on-premises scan also connects to 42Crunch Platform when it runs to fetch the scan configuration and to upload logs. This is not an issue as long as the machine it runs on has direct internet access.

However, if the internet access is behind an HTTP or HTTPS proxy, some additional configuration is needed. Conformance Scan uses two environment variables that you must configure depending on which connections must go through the proxy.

1. Configure the proxy behavior for your Docker daemon. For more details, see Docker documentation.
2. Update your Docker command for the on-premises scan depending on which connections must go through the proxy:
   • Scan must connect to 42Crunch Platform through proxy:

     docker run -e SCAN_TOKEN=<your token value>  -e PLATFORM_SERVICE=services.<your hostname>:8001 -e HTTP_PROXY=<replace with your proxy server address> -e HTTPS_PROXY=<replace with your proxy server address> 42crunch/scand-agent:latest

   • Scan must connect to the API through proxy:

     docker run -e SCAN_TOKEN=<your token value>  -e PLATFORM_SERVICE=services.<your hostname>:8001 -e HTTP_PROXY_API=<replace with your proxy server address> -e HTTPS_PROXY_API=<replace with your proxy server address> 42crunch/scand-agent:latest

Replace the placeholders for the variables with the actual values you want to use. If you do not need both HTTP and HTTPS proxy, remove the unnecessary one. If you are a community user, you do not need the flag PLATFORM_SERVICE.

View the scan report from the on-premises scan

When the on-premises scan runs, it creates and uploads a scan report and scan logs to 42Crunch Platform. You can view the report from your latest on-premises scan in the platform.

1. Go to the API you want, and click Conformance Scan.
2. Click on the time stamp of the report from Scan v1 run on premises. You can see the list of tests the scan run and the found issues, and click on them to see more details.

   

3. Use the "Critical to Success" filter bar above the result list to filter what is being shown in the result list. You can also further refine the results with the additional dropdown filters.

   

4. Use the filters on the sidebar on the left to show issues only for a particular path or method.

   

5. To view the details of an issue and the test that was run, click the issue on the list.

   

6. To view the logs that the scan produced and check how the scan progressed, go to Logs.

Export scan reports

You can export a scan report from 42Crunch Platform to share with others. If you do not want the full report with all the details, you can also export just the found issues as a CSV list.

We recommend you treat the exported report as company-confidential information and store it safely because it can reveal information on security risks in your APIs to outsiders.

1. Click Conformance Scan, and go to the scan report you want to export.
2. Click Export, and choose what you want to export:
   • To export the full report, click Save report as JSON.
   • To export a report overview, click Save report as PDF.

     The overview PDF shows the current report page that you are on, so you can use the filters or export a different page to change what is included in the PDF.

   • To export just the list of all found issues as CSV, click Save issue list as CSV.

     CSV export is only supported for reports from Scan v1 v1.22.14 or higher.

Change the scan report scope

By default, Conformance Scan includes the details of all test results from a scan. However, to avoid the report size getting massive — especially with large request bodies — you can choose to exclude test results where no problems were found.

1. Prepare your Docker command for the on-premises scan as before (see Run on-premises scan).
2. Add REPORT_FULL=false to your Docker command, for example:

   docker run -e SCAN_TOKEN=<your token value>  -e PLATFORM_SERVICE=services.<your hostname>:8001 -e REPORT_FULL=false 42crunch/scand-agent:latest

   Replace the placeholders with the actual values you want to use.

3. Run the on-premises scan again.

The on-premises scan now excludes the details of fully successful tests from the scan report it produces. This means that the filter bar now shows the result of test requests where both the response code and the contract conformance were expected as zero. These results are also excluded when calculating the percentages shown on the filter bar.

Change the scan report size limit

By default, the maximum size for a scan report is 32 MB. Depending on your API, this limit can be reached quickly, especially if you do not exclude tests where no issues were found from the report (REPORT_FULL=false) but want to include everything in the report.

If you run Conformance Scan in 42Crunch Platform, you cannot change maximum scan report size. However, when running the scan on premises, you can increase the maximum report size up to 128 MB.

Large scan reports can take a long time to load in your browser. Depending on your browser memory limits and your device, if your scan report size exceeds 20 MB, you may not be able to view the scan report correctly on the platform UI. However, larger reports can be viewed in IDE. For more details on running Conformance Scan in IDE, see IDE integration.

1. Prepare your Docker command for the on-premises scan as before (see Run on-premises scan).
2. Add the flag max-report-size to your Docker command, with the maximum size limit defined in bytes, for example:

   docker run -e SCAN_TOKEN=<your token value>  -e PLATFORM_SERVICE=services.<your hostname>:8001 -e max-report-size=67108864 42crunch/scand-agent:latest

   Replace the placeholders with the actual values you want to use.

3. Run the on-premises scan again.

The on-premises scan now restricts the report size to 64 MB. When the report reaches this limit, the scan stops and uploads the report of the completed test requests to 42Crunch Platform.

Change the settings for on-premises scan

If you find that the default settings in the scan configuration do not work for your API, you can adjust them to better customize the scan. You cannot directly edit the existing scan configuration, but if you create a new Scan v1 configuration it will replace the existing one.

Each API can have only one scan configuration. Before you can create a new configuration, you must delete the existing configuration.

1. Go to the API you want to scan, and click Conformance Scan > Add configuration > Version 1.
2. Configure the API endpoint, authentication, and the optional mutual TLS as usual (see Create scan configuration for on-premises scan).
3. Click Configure settings, and switch Use custom scan settings on.
4. Change the settings you want. For the full list of settings and which scan type they apply to, see API Conformance Scan settings.
5. Click Review setup, and verify that the configuration is as you want it. If you want to change something, you can click on the tabs in the scan configuration wizard to jump to the correct section.
6. When you are ready, click Create scan configuration.
7. Prepare your Docker command for the on-premises scan as before (see Run on-premises scan), and run the on-premises scan again.

Specify longer timeout for the on-premises scan

We do not recommend setting the timeout value to 0, because this switches the timeout off completely. This means that if something goes wrong, the scan might run indefinitely.

1. Prepare your Docker command for the on-premises scan as before (see Run on-premises scan).
2. Run the cURL against your API to see how long it takes for it to respond.
3. Add SCAN_HTTP_TIMEOUT with a timeout value (in seconds) larger than your API's reponse time to your cURL, for example:

   docker run -e SCAN_TOKEN=<your token value>  -e PLATFORM_SERVICE=services.<your hostname>:8001 -e SCAN_HTTP_TIMEOUT=<timeout in seconds> 42crunch/scand-agent:latest

   Replace the placeholders with the actual values you want to use.

4. Run the on-premises scan again.

The on-prem scan now waits for the API to respond until the timeout value you specified, so your API should be able to respond in time. If the first value does not solve the timeout error, try running the cURL against your API more times to see if there is variation to the response time.

Use environment variables in scan configuration

For Scan v1, each API can have one scan configuration for running Conformance Scan on 42Crunch Platform and another for running it on premises. However, for authentication details you can use environment variables and supply the values you want to use in your run command.

Each API can have only one scan configuration. Before you can create a new configuration, you must delete the existing configuration.

1. Go to the API you want to scan, and click Conformance Scan > Add configuration > Version 1, and configure the API endpoint as per usual.
2. When configuring authentication, enter an environment variable to any field you do not want to hard-code. The environment variable:
   • Must be inside curly brackets ({})
   • Must start with $
   • Cannot contain other special characters than -, _, and .

   

3. Finish creating the scan configuration, and copy the scan token value.

   Environment variables are currently not supported for mutual TLS password.

4. Prepare your Docker command for the on-premises scan as before (see Run on-premises scan).
5. Provide the values for your environment variables in the command. You must add the prefix SECURITY_ to each variable, for example:

   docker run -e SCAN_TOKEN=<your token value>  -e PLATFORM_SERVICE=services.<your hostname>:8001 -e SECURITY_ACCESS_TOKEN='<the access token value you want to use>' 42crunch/scand-agent:latest

   Replace the placeholders with the actual values you want to use.

6. Prepare your Docker command for the on-premises scan as before (see Run on-premises scan), and run the on-premises scan again.

Set on-premises scan to write logs to a file

By default, Conformance Scan run on premises writes all log levels as standard output (STDOUT) to console and your terminal, but only uploads error and critical level scan logs to 42Crunch Platform. You can also direct the STDOUT logs to be consumed downstream services (see the documentation for your environment for how this is done), or write the logs in a volume that you mount as part of your run command. The scan writes logs to the file <your local filepath>/opt/scand/log/<task ID>-<epoch timestamp>-scand.log.

1. Prepare your Docker command for the on-premises scan as before (see Run on-premises scan).
2. Add the volume where the log file is stored to your Docker command, for example:

   docker run -v=<replace with your local filepath>:/opt/scand/log -e SCAN_TOKEN=<your token value> -e PLATFORM_SERVICE=services.<your hostname>:8001 42crunch/scand-agent:latest -logger-file-path=/opt/scand/log

   Replace the placeholders with the actual values you want to use.

3. Run the on-premises scan again.

On-premises scan runs and writes all logs both as STDOUT in the console and into the specified logger file. In addition, it still uploads error and critical level logs to 42Crunch Platform.

The level of logs written depends on the logger level you have set (defaults to error). Normally, you set this as part of the scan configuration (see Change the settings for on-premises scan), but you can set it in your Docker command, for example, to simply check how different levels affect the logs by adding the flag -logger='<log level>' (for example, -logger='info') to your command. The flag in your Docker command overrides the logger level set in the scan configuration.

For more details, see Scan logs.