Node.js - Startup Troubleshooting

Issue

Some behaviors noticed when starting up your application with the Node agent:

• Your application starts but it is not reporting to the Contrast Server

• The following error(s) are being seen in the Contrast Node agent service logs when trying to instrument our application:

ERROR TSClient: Could not send preflight check {"progname": "Contrast Service", "error": "Put \"https://app.contrastsecurity.com/Contrast/api/ng/preflight\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"}

ERROR TSClient: Error in preflight endpoint {"progname": "Contrast Service", "error": "Put \"https://app.contrastsecurity.com/Contrast/api/ng/preflight\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"}

ERROR TSClient: Could not send server activity {"progname": "Contrast Service", "error": "context deadline exceeded"}

ERROR TsReporter: got error from tsclient attempting to send server activity {"progname": "Contrast Service", "error": "context deadline exceeded"}

• Your application/container crashes without any indicators or error messages

• You notice a large memory/CPU spike during application start-up

Cause

As far as start-up issues are concerned, they fall into two buckets:

1. Misconfigurations
2. Exceeding available resources.

Running the Contrast Security agent along with a Node.js application (and any other technology actually) will always provide a little bit of an overhead resulting in increased Memory and CPU usage and request latency. As we know this is inevitable because we are running additional code - the agent’s code along with the code of the given application.

We can divide the application life into two phases - application start-up and handling requests.
At the start of each new application, the agent is performing a rewriting of some of the code provided so it can be instrumented. And during the handling requests phase, the agent is tracking each request made. There is a possibility of performance issues in both phases. Here, we will talk only about performance issues in the first phase - during application start-up. If your performance issues are experienced during the handling requests phase, please follow the runtime performance troubleshooting guide: Node.js - Runtime Performance Troubleshooting

Resolution

Misconfiguration

Your application doesn’t start after being configured to use the agent, or it starts up fine, but you do not see your server or application in the Contrast UI.

What to check:

• You’re not blocked by a proxy or firewall rule

• Ensure you have the latest LTS version of Node.js
• Ensure your application is supported by Contrast - Node.js Supported Technologies

  • You’re not bundling your server-side Node.js code with a tool like webpack, parcel, or esbuild

• Your agent configurations are being picked up by the agent

  • Your YAML/environment variables are in the correct location - Order of Precedence

• Your application is written with CommonJS (CJS) modules

  • We have limited support for ECMAScript Modules (ESM) - Use the Node.js agent with ESM

• The command used to run your application preloads our module at start-up: node -r @contrast/agent <app-main>.js

  • If you’re already using a --require command when running this application to set environment variables or secrets then make sure to include that --require when composing the new run command with the Contrast agent. Always try to have the Contrast agent --require as the first or left-most command.

  • eg. You normally start your application with node -r dotenv/config index.js. You’ll want to change that to node -r @contrast/agent -r dotenv/config index.js.

Exceeding available resources

When you start your instrumented application, Contrast applies source transformations to both your application and the dependency code your application loads. The rewriting process can take a long time and can cause container health checks to fail and cause containers to get stuck in a bad re-start cycle.

Starting with version 4.X, the Node.js agent includes a command line utility you can use to pre-compile applications before starting them. The CLI-rewriter was built to deal with start-up memory, CPU spikes, and health-check timeouts. When started with Contrast, the pre-compiled application loads the rewritten files from disk and significantly improves startup time.

Use the rewriter

1. In the Node.js agent's configuration file (contrast_security.yaml):

   • Enable rewrite caching and specify the path of the cache location.

   • Specify the logging level.

     Note: You can override the YAML configuration settings by invoking the rewriter with environment variables. For example, to prevent any logging to stdout or to disk you could use these overrides:

     export CONTRAST__AGENT__LOGGER__STDOUT=false
     export CONTRAST__AGENT__LOGGER__PATH=/dev/null

   • You must also explicitly enable Protect or Assess.

   For example:

   agent:
     logger:
       level: info
       path: ./node-contrast
     node:
       rewrite_cache: 
         enable: true 
         path: ./rewrite_cache
   assess: 
     enable: true
   protect:
     enable: false

2. Invoke the executable and provide it with your application's entry point (for example, index.js). For example:

   npx -p @contrast/agent -no contrast-transpile index.js;
   ...
   trace: 2021-07-12T18:31:15.128Z 2934603 contrast:rewrite - successfully rewrote code for /path/to/app/index.js
   trace: 2021-07-12T18:31:15.186Z 2934603 contrast:rewrite - successfully rewrote code for /path/to/app/node_modules/koa/lib/application.js
   trace: 2021-07-12T18:31:15.251Z 2934603 contrast:rewrite - successfully rewrote code for /path/to/app/node_modules/koa-router/lib/router.js
   trace: 2021-07-12T18:31:15.314Z 2934603 contrast:rewrite - successfully rewrote code for /path/to/app/node_modules/@koa/router/lib/router.js
   ...
   info: 2021-07-12T18:31:41.608Z 2946030 contrast:cli-rewriter - rewriting complete [26.625s]

   Once contrast-transpile is complete, you should see the rewritten code in the path specified in your configurations (for example, ./rewrite_cache as set in contrast_security.yaml).

    The same rewrite_cache config settings must be used when running the Contrast agent.

3. Once rewriting completes, start your application with Contrast as you usually would. (for example, node -r @contrast/agent index.js).

If your application is containerized, you could perform the rewriting as part of the Docker image creation. Here is an example Dockerfile without the rewriter and with the rewriter:

# Dockerfile
ARG node_version=16                                      
FROM node:$node_version                                  
                                                         
WORKDIR /app                                             
COPY . .                                                 
RUN npm ci                                               
ENV DEBUG=contrast:*                                     
ENTRYPOINT ["node", "-r", "@contrast/agent", "server.js"]

# Dockerfile.precompile
ARG node_version=16                                            
FROM node:$node_version                                        
                                                               
WORKDIR /app                                                   
ENV DEBUG=contrast:*                                           
COPY . .                                                       
RUN npm ci                                                     
# Add the following lines
ENV CONTRAST__AGENT__LOGGER__STDOUT=false
ENV CONTRAST__AGENT__LOGGER__PATH=/dev/null                                     
RUN npx -p @contrast/agent -no contrast-transpile server.js
# ---                                                          
ENTRYPOINT ["node", "-r", "@contrast/agent", "server.js"]

 If issues persist after making this change, please submit a ticket to our online support portal.