Python Agent Troubleshooting

  • Updated

Question

What are some common troubleshooting steps for the Python agent?

 

Prerequisites

Before installing the Python agent, check the system requirements outlined in https://docs.contrastsecurity.com/en/python-system-requirements.html. In particular, ensure that gcc, make, automake and autoconf are installed on the target system. The package names may be different on different platforms but installing your platform's version of build-essential should cover most of the prerequisites. If running an agent on Alpine OS, libtool is also  required.

At this time, the Python agent only supports Unix-like operating system but does not work on Windows.

Agents prior to 5.24.0 are not designed to work on M1/arm architecture. For experimental M1/arm support, try reinstalling the agent with CONTRAST_NO_FUNCHOOK=1 set in your environment. For Docker, You can pass the --platform linux/amd64  flag to Docker to enable x86 emulation.

 

In case of problems, ensure the system is running the latest version of the agent. For a list of available versions, run  pip index versions contrast-agent or pip install contrast-agent==. To get the currently installed version, run pip show contrast-agent. Older versions may have different requirements from the latest agent. In particular, agent prior to v5.19.0 employ the Contrast-service in order to communicate with the Contrast UI. This requires setting service properties in the contrast_security.yaml, and may require additional memory. For details, see https://docs.contrastsecurity.com/en/contrast-service.html

Python agents prior to v5.24.0 require the agent to be added manually as middleware in the project under test. For details, see https://docs.contrastsecurity.com/en/python-middleware.html. Newer agents should use the contrast-python-run command though the middleware system should still be supported. 

 

Possible Problems

[1] The app does not start with the agent or the agent fails to install/run:

When installing the agent, and an error like Building wheel for contrast-agent (pyproject.toml) did not run successfully is shown, this may indicate a missing dependency. The output should have details, such as ./autogen.sh: line 2: autoheader: command not found, indicating autoheader is not installed. The exact package would depend on the platform/environment, but for Linux, it may help to install automake, build-essentails or linux-headers. The agent needs these tools to build C extensions.

 

If importing middleware, ensure the the Contrast middleware is the first to be called. Also, note the change in middleware imports with agent version 5.0.0 (see https://docs.contrastsecurity.com/en/python-middleware.htm for details). 

Note: Middlewares are called in the reverse order they are added when using add_middleware.

 

Some servers require certain configuration options in order to work properly with Contrast. If an application fails to get to a ready state with the agent attached, there are some properties that can be set for servers such as Gunicorn, Uvicorn and uWSGI. For details, see https://docs.contrastsecurity.com/en/python-webserver.html

 

If application startup fails with a  TypeError, it may be possible to bypass the error by disabling the rewriter via the setting

agent:
  python:
  rewrite: false

 

If Assess or Protect are not enabled at startup, try enabling the feature via the YAML, such as 

assess:
enable: true

 

If possible, try increasing memory/CPU resources available to the application to see if that addressed any problems.

 

[2] The agent does not report the the Contrast UI:

Ensure the contrast_security.yaml (or the corresponding environment values) used with the agent does provide the correct API credentials for the agent, which can be obtained from User Menu->Organization Settings->Agent.

 

The Contrast UI may reject the application because an archived app with the same name exists. In this case, the log should read something like “Contrast received instructions to disable itself”. To address this, ensure there is no archied app with the same name or - if there is - delete or unarchive the archived application.

 

The Contrast UI may reject the application because required metadata has not been provided. In this case, the log should read something like DISABLING Contrast agent due to metadata violation”. To address this, provide the required metadata or modify the metadata settings for the organization via User Menu->Organization Settings->Applications (requires Admin access to the organization).


If a proxy is required to communicate with the Contrast UI, settings for

api:
  proxy:
  url: scheme://host:port

may be required

 

In case of a certificate problem, the log may show an error like “certificate verify failed: self signed certificate in certificate chain”. To test this, you can disable certificate validation via

api:
certificate:
  ignore_cert_errors: true

 

[3] Performance Issues:

 

As a rule of thumb, do not run Assess and Protect on the same system, and do not run Assess in production as it has a significantly higher overhead than Protect. If the application performance still shows a significant impact, it may be possible to disable specific rules to improve startup performance in the Contrast UI.

For Protect, known safe endpoints where analysis does take a considerable amount of time can be excluded via a url_exclusion_pattern, such as

 

agent:
  python:
    url_exclusion_patterns:
      - /static/.*\.js
      - .*very_bad_url\.pdf.*
      - .*/cmdi.*
      - /.*\.ico

 

There are several configuration options to help reduce the impact of the agent when running in Assess mode, such as Sampling, which limits how frequently the agent will analyze the same request, the maximum events the agent is analyzing for each request, and limiting the stacktraces being collected:

 

assess:
sampling:
enable: true # default is always false so must enable it here or in Teamserver UI
baseline: NUMBER # default is 5
request_frequency: NUMBER # default is 10
window_ms: NUMBER # default is 180000
max_context_source_events: NUMBER # default is 100
max_propagation_events: NUMBER # default is 250
stacktraces: NONE or SOME

Note: Stacktraces contain information developers may need when fixing vulnerabilities that NONE will suppress.

 

While DEBUG or TRACE logging are generally required for troubleshooting, excessive logging will affect performance. Using the default (ERROR) is recommended for normal operation.

 

For Flask/uWSGI, you can try to disable automatic application reloading by setting py-autoreload=0 in uwsgi.ini.

 

Additional Troubleshooting

If issues persist after trying these steps, please submit a ticket to our online support portal and include any errors displayed, an agent TRACE or DEBUG log (see https://support.contrastsecurity.com/hc/en-us/articles/360038113411-How-to-get-logs-from-the-Python-agent?source=search), the requirements.txt and the output of pip list.


For performance issues, it may also help to get profiler logs by enabling

agent:
  python:
    enable_profiler: true

and collecting the *-profiler-stats.out logs for each request.

Was this article helpful?

0 out of 0 found this helpful

Have more questions? Submit a request