AEM Sites Performance Issues

Symptoms of a performance issue

1. Slow loading of pages  
2. Slow creation or editing of pages
3. AEM response times are slow
4. AEM is not responding for some requests
5. The request.log on AEM shows slow response times

What causes performance issues

1. Thread contention - long running requests such as slow searches, write-heavy background jobs, moving of whole branches of site content, etc.
2. High CPU utilization
3. Expensive requests such as expensive searches or inefficient application code, components, etc.
4. Lack of proper maintenance
5. Insufficient dispatcher caching
6. Lack of CDN
7. Lack of browser caching
8. Too many scripts loaded on page and loaded at top of page
9. CSS loaded throughout page instead of in the HTML head
10. Insufficient server sizing or incorrect architecture
11. Memory issues (see below)

How to analyze the performance issue

1. Capture a series of thread dumps and analyze them

2. Check at the OS level if the AEM java process is causing high CPU utilization

    Linux: use the top command to check CPU utilization.

    Window: use the Windows Task Manager

    If AEM is causing high CPU utilization then run the out-of-the-box profiling tool for a few minutes and analyze the result.

3. Analyze the request.log file for any slow requests

4. Review your system maintenance procedures and ensure that you are doing proper maintenance on AEM including the following:

• Revision Clean Up (MongoMK and Database DocumentNodeStore's only) - daily or more frequent
• Offline Tar Compaction (TarMK only) - bi-weekly
• Data Store Garbage Collection (Systems with FileDataStore or S3 DataStore only) - weekly
• Workflow Purge - weekly
• Version Purge - weekly
• AuditLog Purge - weekly

See this article for more details on AEM maintenance.

5. Review caching strategies implemented at the AEM dispatcher level.  The best place to start is to gain an understanding of when and how the dispatcher caches files and invalidates cached files.

6. Review your site's caching.

7. Use client-side site analysis tools such as the Audits feature in Google Chrome browser Developer Tools panel.  These tools will give you recommendations on client-side performance improvements.

Solutions to common performance issues

• Refer to this article for detailed steps on ways to optimize performance.
• Review the performance tuning tips

AEM Assets Performance Issues & How to Fix

Symptoms of an Assets performance issue

• Slow file uploads to /assets.html or /damadmin UI
• Thumbnails are taking too long to be generated
• Assets operations such as move, delete, edit, and metadata update taking too long

What causes issues with Assets performance

• Lack of proper maintenance
• Latest fix packs not applied
• Optimizations not applied
• Inadequate server sizing for the user load

How to analyze the Assets performance issue

• Review steps 1-4 in sites performance analysis above.

Solutions to common Assets performance issues

• Review the AEM Assets Performance Tuning Guide.
• Solutions to some issue scenarios and their solutions can be found here.
• Tune asset processing performance, see this article.

AEM Memory Issues and How to Fix

Symptoms of a memory issue

• AEM crashes randomly and in the logs OutOfMemoryError is observed
• AEM gets slower over time and eventually crashes
• AEM is unresponsive

Diagnosing a memory issue

• Search the log files for OutOfMemoryError, if you find any matches then you have a memory issue
• Review the http://aem-host:port/system/console/memoryusage screen
  If the "Old Generation" (JDK 7 and earlier) or "Tenured Generation" (JDK8 or later) usage is high then this could be a sign of a heap memory utilization issue.  Click "Run Garbage Collector" to request the JVM to run a full heap garbage collection.  If the high heap utilization stays high after requesting GC then there is likely an issue.  On an AEM instance with Oak Tar storage, if the tenured usage is higher than 3GB then there might be a problem.  High heap utilization on a system with Mongo storage could be due to the in-memory cache configuration.
• Take thread dumps and top output and perform thread analysis.  Check if the threads causing high CPU utilization are native JVM Garbage Collection threads.  If the thread using the most CPU time are the "VM Thread" or any garbage collection threads then there is likely a memory issue.

What causes memory issues

• Java application memory leak
• Java Finalizer pile up due to incorrect use of finalize in custom code
• Insufficient max heap configuration

How to analyze the cause of your memory issue

See this article for details on how to capture a heap dump.

The best way to identify the cause of a memory issue is to analyze a heap dump.  

Once you've captured a Heap Dump file then open it in Eclipse MAT or IBM Memory Analyzer tool.  In Eclipse MAT, run the Leak Suspects report and open "Thread Details" view to see potential causes for the memory issue.

Solutions to common memory issues

• Optimize your application code to utilize less memory if you notice long garbage collection pauses.  Most Garbage Collection issues can best be solved by optimizing the application versus tuning the JVM.
• If you have already optimized your application and still experience long GC pauses then focus on tuning the JVM.