Python Agent Effects on Application Performance (Assess)

Question

After configuring your application to use the Contrast agent in Assess mode, what is the anticipated performance impact on the application?

Answer

When running in Assess mode, Contrast's analysis will make your application run a little slower. But, the time difference is usually minimal, and the results are definitely worth it.

It's probably more important to think about how Contrast affects the round-trip time. In typical applications, Contrast adds most time to a request that contains a lot of business logic. Round trip times for static resources typically don't get measurably worse. In requests where the total round-trip time is dominated by database or Web Service calls, Contrast's effect will be less noticeable.

Performance tuning

The following steps can be taken to tune the agent's performance in Assess mode:

• Ensure that the server meets the recommended system requirements and the server has enough free memory before the Python agent is installed.
• Run Contrast during nightly integration tests.
• Run Contrast in an alternate environment (QA system or DEV environment).
• Run Contrast on a single node in a load-balanced environment.
• Check that the agent's logging level is set to WARN or ERROR.

Turn off stack traces creation

In both modes, the agent builds stack traces to tell the user where the attack happened or the vulnerable data flowed.

This may be considered useful information, but it’s very costly for the agent to compute. For that, we have a common config option to turn this feature fully or partially off. We suggest the following configuration:

assess:
  stacktraces: NONE # or SOME

This should improve performance for vulnerable requests only. That is, any request that is not vulnerable and hence did not create a stack trace will not be affected.

Use event limit configuration

Starting with version 4.11.0, the Python Agent comes with a default behavior that will limit

1. sources created within one request to 100

2. propagations happening within a request to 1000

3. the number of vulnerabilities of one type reported per 5 minutes of agent run time to 100

While the defaults should be sufficient, you can decrease the sources and propagations value to boost performance via the Contrast configuration file like so:

assess:
  max_context_source_events: .. #default is 100
  max_propagation_events: ... #default is 1000

assess:
   sampling:
     enable: true # default is always false so must enable it here or in Teamserver UI
     baseline: 5 # default is 5
     request_frequency: 10 # default is 10
     window_ms: 180000 # default is 180000

Further analysis

If performance is still outside of the expected degradation after following the steps above, we can investigate further by recording a full profile of your application whilst running with Contrast by following these steps:

Protect & Assess: enable cProfile

Starting in v.5.1.0, the Python Agent comes with a configuration option to dump cProfile data for each request. This can be enabled by adding the following to your Contrast configuration (.yaml) file:

agent:
  python:
    enable_profiler: true

or the corresponding env variable (CONTRAST__AGENT__PYTHON__ENABLE_PROFILER=true)

Once configured, run your application and make requests to the endpoints that are experiencing performance issues. You will then see output files named cprofile-*-*.out, one for each request. These files can be opened with various tools, such as Python’s pstat or snakeviz.

Send these .out files to our team for further analysis.

Debug agent logs covering the performance issue may also be helpful - How to get logs from the Python agent. However, it is expected that debug logging will further impact application performance and as such may not always be feasible to collect.  Providing the log file at even WARN or ERROR level may contain some useful information.

Please submit a ticket attaching the results of this profiling and any agent/application logs.