AEM_How to find requests with long duration times

Solution:

§  AEM includes the rlog.jar file in the below path:  /crx-quickstart/opt/helpers

§  It is used to find requests with long duration times

The following command shows the possible arguments of rlog.jar file

$java -jar rlog.jar

Request Log Analyzer Version 21584 Copyright 2005 Day Management AG

Usage:

  java -jar rlog.jar [options]

Options:

  -h               Prints this usage.

  -n   Limits output to lines.

  -m Limits input to requests.

  -xdev            Exclude POST request to CRXDE.

For example, you can run it specifying request.log file as a parameter and show the 10 first requests that have the longest duration:

$ java -jar ../opt/helpers/rlog.jar -n 10 request.log

Trace results :

*Info * Parsed 464 requests.

*Info * Time for parsing: 22ms

*Info * Time for sorting: 2ms

*Info * Total Memory: 1mb

*Info * Free Memory: 1mb

*Info * Used Memory: 0mb

------------------------------------------------------

18051ms 31/Mar/2009:11:15:34 +0200 200 GET /content/geometrixx/en/company.html text/ html

2198ms 31/Mar/2009:11:15:20 +0200 200 GET /libs/cq/widgets.js application/x-javascript

1981ms 31/Mar/2009:11:15:11 +0200 200 GET /libs/wcm/content/welcome.html text/html

1973ms 31/Mar/2009:11:15:52 +0200 200 GET /content/campaigns/geometrixx.teasers..html text/htm

1883ms 31/Mar/2009:11:15:20 +0200 200 GET /libs/security/cq-security.js application/x-javascript

1876ms 31/Mar/2009:11:15:20 +0200 200 GET /libs/tagging/widgets.js application/x-javascript

1869ms 31/Mar/2009:11:15:20 +0200 200 GET /libs/tagging/widgets/themes/default.js application/x-javascript

1729ms 30/Mar/2009:16:45:56 +0200 200 GET /libs/wcm/content/welcome.html text/html; charset=utf-8

1510ms 31/Mar/2009:11:15:34 +0200 200 GET /bin/wcm/contentfinder/asset/view.json/ content/dam?_dc=1238490934657&query=&mimeType=image&_charset_=utf-8 application/json

1462ms 30/Mar/2009:17:23:08 +0200 200 GET /libs/wcm/content/welcome.html text

Apache_bench marking tool_to identify the HTTP server performance along with AEM server Laod testing

Solution :

• ab is a tool for benchmarking your Apache Hypertext Transfer Protocol (HTTP) server. 
• It is designed to give you an impression of how your current Apache installation performs. 
• This especially shows you how many requests per second your Apache installation is capable of serving.
• To minimize the impact of  garbage collection, etc, it is recommended to use a tool such as apachebench 
• To help identify memory leaks and selectively analyze response time.

Apache Bench can be used in the following way:

$ ab -c 5 -k -n 1000 "http://localhost:4503/content/geometrixx/en/company.html"

this is ApacheBench, Version 2.3 <$Revision: 655654 $>

Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/

Licensed to The Apache Software Foundation, http://www.apache.org/T

Benchmarking localhost (be patient)

Completed 100 requests

Completed 200 requests

Completed 300 requests

Completed 400 requests

Completed 500 requests

Completed 600 requests

Completed 700 requests

Completed 800 requests

Completed 900 requests

Completed 1000 requests

Finished 1000 requests

Server Software: Day-Servlet-Engine/4.1.52

Server Hostname: localhost

Server Port: 4503

Document Path: /content/geometrixx/en/company.html

Document Length: 24127 bytes

Concurrency Level: 5

Time taken for tests: 69.766 seconds

Complete requests: 1000

Failed requests: 998

(Connect: 0, Receive: 0, Length: 998, Exceptions: 0)

Write errors: 0

Keep-Alive requests: 0

Total transferred: 24160923 bytes

HTML transferred: 24010923 bytes

Requests per second: 14.33 /sec (mean)

Time per request: 348.828 [ms] (mean)

Time per request: 69.766 [ms] (mean, across all concurrent requests)

Transfer rate: 338.20 [Kbytes/sec] received

Connection Times (ms)

min mean[+/-sd] median max

Connect: 0 1 3.9 0 58

Processing: 138 347 568.5 282 8106

Waiting: 137 344 568.1 281 8106

Total: 139 348 568.4 283 8106

Percentage of the requests served within a certain time (ms)

50% 283

66% 323

75% 356

80% 374

90% 439

95% 512

98% 1047

99% 1132

100% 8106 (longest request)

• apachebench also displays the time per request as the mean, across all concurrent requests;

see Time per request: 54.595 [ms] (mean, across all concurrent requests). 

• You can change the value of the concurrency parameter -c(number of multiple requests to perform at a time) to see any effects.

Command details :

-c concurrency

Number of multiple requests to perform at a time. 
Default is one request at a time.

     -k

Enable the HTTP KeepAlive feature, i.e., perform multiple requests within one HTTP session. Default is no KeepAlive.

     -n requests

Number of requests to perform for the benchmarking session. The default is to just perform a single request which usually leads to non-representative benchmarking results.

Reference : http://httpd.apache.org/docs/2.2/programs/ab.html

AEM_Tools for Monitoring and Analyzing Performance

The following gives a short overview of some of the tools available for monitoring and analyzing performance.

Some of these will be dependent on your operating system.

Tool	Used to analyze...	Usage / More information...
request.log	Response times and concurrency.	Inter request.log.preting the
truss/strace	Page Loads	Unix/Linux commands to trace system calls and signals. Increase the log level to INFO. Analyze the number of page loads per request, which pages, etc.
Thread dumps	Observe JVM threads. Identify contentions, locks and long-runners.	Dependent on the operating system: - Unix/Linux: kill -QUIT <pid> - Windows (console mode): Ctrl-Break Analysis tools are also available, such as TDA.
Heap Dumps	Out of Memory issues that cause slow performance.	Add the:     -XX:+HeapDumpOnOutOfMemoryError option to the java call to CQ. See the Troubleshooting Guide for Java SE 6 with HotSpot VM.
System calls	Identify timing issues.	Calls to System.currentTimeMillis() or com.day.util.Timing are used to generate timestamps from your code, or via HTML-comments. Note: These should be implemented so that they can be activated / deactivated as required; when a system is running smoothly the overhead of collecting statistics will not be needed.
Apache Bench	Identify memory leaks, selectively analyze response time.	basic usage is: ab -k -n <requests> -c <concurrency> <url> See Apache Bench and the ab man page for full details.
Search Analysis		Execute search queries offline, identify response time of query, test and confirm result set.
JMeter	Load and functional tests.	http://jakarta.apache.org/jmeter/
JProfiler	In-depth CPU and memory profiling.	http://www.ej-technologies.com/
JConsole	Observe JVM metrics and threads.	Usage: jconsole See jconsole and Monitoring Performance using JConsole. Note: With JDK 1.6, JConsole is extensible with plug-ins; for example, Top or TDA (Thread Dump Analyzer).
Java VisualVM	Observe JVM metrics, threads, memory and profiling.	Usage: jvisualvm or visualvm See jvisualvm, visualvm and Monitoring Performance using (J)VisualVM. Note: With JDK 1.6, VisualVM is extensible with plug-ins.
truss/strace, lsof	In depth kernel call and process analysis (Unix).	Unix/Linux commands.
Timing Statistics	See timing statistics for page rendering.	To see timing statistics for page rendering you can use Ctrl-Shift-U together with ?debugClientLibs=true set in the URL.
CPU and memory profiling tool	Used when analyzing slow requests during development.	For example, YourKit.
Information Collection	The ongoing state of your installation.	Knowing as much as possible about your installation can also help you track down what might have caused a change in performance, and whether these changes are justified. These metrics need to be collected at regular intervals so you can easily see significant changes.