|License Type||SaaS & On-Premise|
|Agent Mode||Assess & Protect|
|Main Product Category||.NET Core Agent|
In rare scenarios, bad instrumentation causes a web server process to crash or a specific page to error out. If you ever encounter a crash or error caused by Contrast, please report the error via the Submit a Request link at the bottom of this page. If possible, follow the steps below to gather agent logs; this additional information is vital to reproducing and fixing these types of bugs.
Agent Logs Directory
You can ask Contrast to log verbose information by setting the logging level to TRACE with one of the following methods.
Note! The default logging path cannot be changed via YAML settings. Default paths are:
- Contrast_security.yaml file: (more information can be found here in our docs)
- Environment Variables: (more information can be found here in our docs)
These options will put Contrast into TRACE mode and log all output to the log file specified or in the default location. It will also prevent the log from rolling over into multiple files.
It is also very helpful to provide detailed information on the .NET environment. The following command produces most of what support will need.
Types of Bugs
There are a few primary types of agent bugs for which Contrast needs to gather logs and other information:
- Agent Fails to Load
- Process Crash
- Unhandled Managed Exception/Page Error/500
For Linux Based Operating Systems:
Agent is failing to load
If the agent is failing to load then there will be no logs created. This command can help us diagnose why the agent is not loading into their process.
This runs `dotnet SampleApp.dll` and outputs all kernel calls to the strace_output.txt file.
strace -o strace_output.txt dotnet SampleApp.dll
If the process is crashing while we are attached/trying to attach these environment variables can be set to create a dump on crash.
The dumps created by the command will be located at
Page Crash/500 Error
If the process is running fine but a page is throwing an error or a page is failing to load dumps can be taken. The location of the `createdump` command may differ based on which version of .NET Core they have installed but usually lives in
ps -ax | grep [process name]
createdump -h [pid]
The dump will be written to /tmp/coredump.[pid]
bob@ubuntudesktop:~$ ps -ax | grep dotnet
9186 pts/0 SLl+ 0:00 dotnet run
9241 pts/0 SLl+ 0:01 /usr/share/dotnet/dotnet /home/bob/.nuget/packages/microsoft.aspnetcore.razor.design/2.1.1/tools/rzc.dll server -p razor-issues-2406
9258 pts/0 SLl+ 0:06 dotnet exec /home/bob/Desktop/SampleApp/bin/Debug/netcoreapp2.2/SampleApp.dll
In this case we want the `dotnet exec` process because it is the real process running the application
bob@ubuntudesktop:~$ /usr/share/dotnet/shared/Microsoft.NETCore.App/2.2.6/createdump -h 9258
Writing full dump to file /tmp/coredump.9258
Written 12285857792 bytes (2999477 pages) to core file
For Windows Based Operating Systems:
Agent is failing to load
You will first want to identify how the dotnet core app is running.
- IIS In-Process.
- IIS Out-of-process
- Standalone process.
If the app is hosted in IIS, check the web.config. The “hostingModel” element of the “aspNetCore” tag will tell if it’s in-process or out of process.
Then use Window's procdump.exe. (documented in detail here)
If app is hosted on IIS and running in InProcess mode, you will need w3wp.exe process.
procdump.exe -ma <process id>
Otherwise dump the dotnet.exe process which is running the application. Identifying this process is best done with SysInternals Process Explorer. Look for the dotnet.exe or w3wp.exe processes, then check their environment variables. Look for the CORECLR variables relating to the Contrast agent.
procdump.exe -ma w3wp.exe
procdump.exe -ma dotnet
Page Crash/500 Error
Procdump can be used to wait for a process and write a dump on a process. (documented in detail here)
Run Procdump first
Procdump.exe -e -w dotnet
Then start the .net core app
Verify an unhandled exception
The above process also helps the .Net Core engineering team resolve issues such as application errors caused by the .Net Core agent. Use the following indicators to determine if the .Net Core agent is causing an application error.
- You've observed the application working normally without the agent.
- You've observed a page of the application "crashing" (returning a 500 error) under the agent.
- There are no errors for "Application Error" in the Windows Event Log.
- There may be warnings for "ASP.NET" in the Windows Event Log. The warning should look similar to the following:
Source: ASP.NET 4.0.30319.0 Date: 10/9/2017 9:22:46 AM Event ID: 1309 Task Category: Web Event Level: Warning Keywords: Classic User: N/A Computer: FOO.COMPUTER.COM Description: Event code: 3005 Event message: An unhandled exception has occurred. Event time: 09/10/2017 9:22:46 AM Event time (UTC): 09/10/2017 2:22:46 PM Event ID: f706787c1f1247e6a87b777a90413c3d Event sequence: 9 Event occurrence: 1 Event detail code: 0 Application information: Application domain: /LM/W3SVC/1/ROOT/FOO-1-131520325424796488 Trust level: Full Application Virtual Path: /Foo Application Path: E:\MCMSFiles\inetpub\wwwroot\Foo\ Machine name: FOO Process information: Process ID: 176840 Process name: w3wp.exe Account name: System Exception information: Exception type: ArgumentOutOfRangeException Exception message: Index was out of range. Must be non-negative and less than the size of the collection. Parameter name: index at System.Collections.ArrayList.get_Item(Int32 index) at System.Web.UI.WebControls.DataListItemCollection.get_Item(Int32 index) at Fabrikam.SetTabCount(Int32 index, NullableInt32 summaryCount) in C:\Foo\Fabrikam.aspx.cs:line 1686 at Fabrikam.GetSummaryCounts() in C:\Foo\Fabrikam.aspx.cs:line 1468 at Fabrikam.OnPreRender(EventArgs e) in C:\Foo\Fabrikam.aspx.cs:line 549 at System.Web.UI.Control.PreRenderRecursiveInternal() at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) Request information: Request URL: https://www.foo-staging.com/Foo/Fabrikam.aspx Request path: /Foo/Fabrikam.aspx User host address: 18.104.22.168 User: msteeber Is authenticated: True Authentication Type: Thread account name: System Thread information: Thread ID: 19 Thread account name: System Is impersonating: False Stack trace: at System.Collections.ArrayList.get_Item(Int32 index) at System.Web.UI.WebControls.DataListItemCollection.get_Item(Int32 index) at Fabrikam.SetTabCount(Int32 index, NullableInt32 summaryCount) in C:\Foo\Fabrikam.aspx.cs:line 1686 at Fabrikam.GetSummaryCounts() in C:\Foo\Fabrikam.aspx.cs:line 1468 at Fabrikam.OnPreRender(EventArgs e) in C:\Foo\Fabrikam.aspx.cs:line 549 at System.Web.UI.Control.PreRenderRecursiveInternal() at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
As the process hasn't crashed, ProcDump won't capture process dumps. Instead, you must gather the process dump manually by completing the following steps.
Find the Process ID of the worker process that you need.
IIS Manager > Worker Processes: Find the "Application Pool Name" you need, and take note of the value in the "Process Id".
From an administrator command prompt, replace
NNNNNwith the process ID from the previous step.
C:\>procdump -ma NNNNN
Follow a similar process to gather agent logs, windows events and process dumps to include with your bug report.
Gather information on the process crash
Complete the following steps to gather information to send to Contrast.
- Set up the ProcDump utility to capture crash dump.
- Download current version of ProcDump from the Microsoft documentation site to the Windows server with the agent.
- From an administrator command prompt:
md c:\dumps procdump.exe -ma -i c:\dumps
- Install the latest .net Core agent.
- Stop the application running with Contrast
- Enable additional logging.
- Start -> Notepad -> Right click -> Run as Administrator
- File -> Open ->%ProgramData%\Contrast\dotnet-core\contrast_security.yaml
- Add the following configuration to the yaml file to enable verbose logging and logging of every method JIT-compiled by the CLR:
- Start the application w/Contrast
- Exercise the application to reproduce the crash.
Once you've reproduced the issue, gather the following items and include them in your bug report:
- Agent Logs: All files in the agent log directory,
%ProgramData%\Contrast\dotnet-core\logs; right click on the LOGS folder > Send To > Compressed (zip) folder.
- Windows Event Log: Event Viewer > Windows Logs > Application > Save All Events As > "MyEvents.evtx"
- Crash Dumps: Create a zip file of each process dump file in C:\dumps (e.g., w3wp.exe_171002_151601.dmp). Dump files can be quite large.
You can then uninstall ProcDump with
These verbose logs will allow our support team to diagnose any issues you may encounter. To contact the Contrast Support Team, please submit a ticket to our online support portal.