JMeter Performance Testing in Cognos Analytics

Chris Chapman, March 17, 2017

Determining the optimal tuning settings for a Cognos Analytics environment is often challenging and confusing. Even when you understand how a specific setting behaves, there is no guarantee until it is implemented that it will have the desired effect within a specific environment.  Additionally, Cognos environments evolve over time as user bases shift and reporting requirements change.   This leaves many Cognos administrators guessing and checking as they make settings changes to their Cognos environments.  This methodology, unsurprisingly, is problematic.  A way to resolve this problematic practice is to run a performance test. Conducting a performance test is a great way to validate an environment after changes have been made. 

And here lies the challenge.  Setting up a performance test can be intimidating.  What tool should you use?  What is the cost?  Where do you start with the script?  Out-of-the-box solutions can be very costly and still require scripting and setup.

Fortunately, there is JMeter for Cognos Analytics

Table of Contents

What exactly is JMeter?

JMeter is a pure Java application that is part of the Apache project.  It is designed to perform load tests and performance tests against Web applications and resources.  The flexibility of JMeter provides users the ability to test against a growing list of different systems.  I have included a link for further reading at the end of this article.

IBM released their own JMeter test script for Cognos 10.2.x. The link to the Proven Practices article is provided below.  The script and article are informative for those working with JMeter or testing for the first time.  However, for extended use, or use in a continually changing environment, I have found this script to be inflexible in its design. JMeter is full of features like the ability to read in variables from CSV files, or to properly handle SSL connections.  These features greatly improve the usability of a JMeter script to provide a more robust and thorough test.  I highly recommend you become familiar with the JMeter documentation to learn about these additional features and incorporate them into a script.

Let’s take a look a couple of practical examples of ways to leverage JMeter for Cognos.

Regression Test

Large scale changes to an environment, such as a model change, can be difficult and time consuming to test.  It often leads to the tedious process of launching individual reports to validate them one-by-one.  With JMeter, you can execute a test that will run all reports in your environment to ensure they are properly working.  Or, you can scale the test to only run reports that are impacted by a recent change (e.g., only selecting reports impacted by the model change).  This test will cycle through the list of reports and run them one-by-one.  You can then view the results in JMeter to see if any reports returned errors.  An example of the results output is in the below screenshot.

You can see JMeter highlighting the report that returned the error.  This assertion (a test element that detects responses) records that an error occurred.  We do not have any insight into what specific error was returned.  However, using XPath, you can create an assertion that can grab the specific error Cognos returned.  The outputs shown can also be saved to a CSV file and opened in Excel if additional reporting or sorting is needed.

This simple script can save many valuable man hours individually validating reports after major environment changes.

Load Test

JMeter lets you utilize the same script used for the regression test for more intensive load test against the system.  Instead of running reports one-by-one, multiple concurrent users can be simulated running multiple reports.  This load can then be used to test for bottlenecks or instabilities in the environment. 

The Thread Group element in JMeter contains the information for defining the specific load profile you are seeking.  The key components are:

• Number of Threads (users): Each thread will execute the test plan in its entirety.  Multiple threads are used to simulate multiple users.
• Ramp-Up Period (in seconds): Tells JMeter the amount of time to ramp-up to the total number of threads/users defined.
• Loop Count: Number of times a thread will loop through each element of a test.
• Scheduler: This can be used to provide the test with a start and stop time.

A sample configuration could look similar to the screenshot below.  This configuration will simulate 5 users, over a 20 minute test, and will ramp up to the max number of users over a 5 minute period.

Most likely you will not want to run all reports in your environment for the load test.  Instead, it may make more sense to choose a profile of reports that represent typical usage in your environment.  Or, select high priority or high cost reports that are more susceptible to failure under heavier workloads. 

Like the regression test, JMeter will provide test results to review.  In this instance, we are looking at report performance over time.  Is report performance or runtime being impacted as the test ramps up to full load?  The Response Time Graph listener will provide a graphical view that shows report runtimes over the length of the test.  Other listeners (a test element that records results) can provide more granular detail, such as the Aggregate Report.

Conclusion

Taking the time to leverage JMeter for Cognos and create a script for your environment can help you save time and identify problems before they hit the end users.  You can check out the aforementioned link here:

IBM Cognos BI 10.2.1.x JMeter Stability Package: https://www.ibm.com/developerworks/library/ba-pp-performance-load_testing-page692/index.html

Subscribe to our e-newsletter for more technical articles and updates delivered directly to your inbox.

Next Steps

If you have any questions or would like PMsquare to provide guidance and support for your analytics solution, contact us today.