AEM_Different ways to take JVM thread dump

AEM OutOfMemoryError: Java heap space or OutOfMemoryError: gc overhead limit exceeded.

Heap histogram

• The heap histogram is a simple measurement of the number of live objects and memory used per Java class. 
• To create a heap histogram, you first need the process id of the Java process. To get it, run ps or (if available), run:

              jps -l

This Java tool gets the process ids of all running Java processes. Example:

          327 
          3332 sun.tools.jps.Jps
          3313 crx-quickstart-....jar

Now run the following command:

        jmap -histo 3313

• The list is sorted by total memory required (shallow: excluding referenced objects). 
• The first 20 lines of the output are the most interesting. Example output:

JVM version is 1.5.0_20-141
Iterating over heap. This may take a while...
Warning: skipping invalid TLAB for thread t@62211
Warning: skipping invalid TLAB for thread t@62467
...
Size Count Class description
-------------------------------------------------------
10592904 12916 byte[]
10285840 75255 * ConstMethodKlass
6283176 58388 char[]
6042304 14928 int[]
4995752 116201 * SymbolKlass
4220896 75255 * MethodKlass
4196512 6969 * ConstantPoolKlass
2928560 6969 * InstanceKlassKlass
2631008 6066 * ConstantPoolCacheKlass
2395872 149742 org.apache.jackrabbit.core.query.lucene.DocId$PlainDocId
1476008 7003 java.util.HashMap$Entry[]
1396128 58172 java.lang.String
1070232 44593 java.util.HashMap$Entry
753984 10036 short[]
735464 54 org.apache.jackrabbit.core.query.lucene.DocId[]
720192 7502 java.lang.Class
640704 13348 com.day.crx.persistence.tar.index.IndexEntry
...

Additional information

To help analyzing the problem, we also need to know the following information:

• CRX or CQ version, including a listing of all installed hot fixes version number.
• Operating system, JVM vendor, and version.

Reference URL :

https://helpx.adobe.com/experience-manager/kb/AnalyzeMemoryProblems.html

AEM Performance analysis using built in profiler

Issue :

• Some process are really slow, but not blocking.  
• The server OS shows high CPU utilization from the AEM java process.

Cause :

There are many possible causes for high CPU utilization, here are a few:

• Inefficient application code
• Traversal of large content structures
• High application memory utilization causing a lot of time spent in JVM Garbage Collection

Analysis & Solution :

Using the built-in CPU profiler

A simple CPU profiling tool is included in AEM 6.x.

1. Open this URL: http://aem-host:port/system/console/profiler
2. Expand "Options"
3. Set the "Sample Interval (ms)" (or use the default)
4. Set the "Stack trace depth" (recommended value : at least 50)
5. Click "Start Collecting"
6. Wait 3-10 minutes to collect data while CPU utilization is high
7. Click "Stop" to collect the data (output is shown on the same page)
   

The profiler collects stack traces of running threads (threads actively taking CPU time).  

While collecting the stacks, it finds duplicate traces and sorts them from most common to least common.