Cognos Server Maintenance Guide
Routine Cognos server maintenance is critical in order to ensure your environment performs optimally and remains reliable. Failure to attend to this could likely cause unnecessary downtime and impact critical deadlines.
Similar to maintaining a vintage car – all it requires is some shrewd TLC.
Below I review some key checks to assist in maintaining your Cognos Analytics servers.
Orphaned BIBusTKServerMain.exe processes hanging
The Cognos community has all occasionally experienced some reports in their environment ceasing to run effectively and further witnessed that the BIBusTKServerMain processes become unresponsive and ultimately end up hanging.
The result of this might be that the system ends up returning the following error messages:
DPR-ERR-2074 Failed to receive a timely response from an external process with a PID <id>
DPR-ERR-2077 Failed to forward the request because the associated external process with PID is unavailable
DPR-ERR-2056 The Report Server is not responding
BIBusTKServerMain.exe processes are spawned by your Cognos BI java.exe process when running Compatible Query Mode (CQM) reports. There can be several different reasons for many of these processes.
Let’s provide some reasons why this may occur:
A user ran a huge report that might not be well designed and the BIBus process ran out of memory.
The dispatcher is, for some reason, unable to communicate with the BIBus process and thus the process doesn’t kill itself after the timeout period.
As the administrator, a good suggestion would be to refer to the Current Activities in your Administration console under the Status tab and note down if the same report with the same parameters was re-submitted a number of times during the past 24-48hrs. Users may very well re-submit reports when they are not familiar with the expected run time of a report or if a report runs longer than they anticipated. This problem often occurs during month-end periods where users are not making use of scheduling of reports during-off peak hours and there is a grab for resources during peak times of the day, for example, 6am – 10am.
Resolving the Problem
Log onto your dispatcher machines and open the taskbar and look under the details tab. The details tab will show the list of processes that are currently executing on the associated dispatcher. Sort the processes by clicking on the Name column and this will list the BIBus processes together. These are the BIBus processes related to Cognos application’s report/job executions. Look for any processes which have been running for > 24hrs or have a memory K size larger than 1.2MB.
To End any specific BIBus process, right-click and select the end task option. This will terminate the BIBusTKServerMain.exe process. Refer to the Administration console prior to terminating any processes to make sure you aren’t canceling/impacting any critical reports that are currently being executed. If you need to terminate them due to congestion in the environment, make sure that the failed reports are re-executed after you’re done cleaning up the processes.
The temporary directory may contain large files at certain times, depending on server activity. If the load on the server is such that many users are running reports, these directories require sufficient space to accommodate files from all the users.
Please see https://www.ibm.com/docs/en/SSEP7J_11.0.0/com.ibm.swg.ba.cognos.ug_cra.doc/ug_cra.pdf for full details on logs, errors, and temp files.
Cleaning up Dump Files
Core dump files are created for serious problems, such as an unhandled exception or an abnormal termination of an IBM Cognos process.
Since core dump files are big and a new one is created each time the problem recurs, this could cause your dispatcher machine to run low on disk space. This is due to BiBus processes’ crashing and generating .dmp files.
You would typically see the error: Report Server not responding.
Resolving The Problem
Check <cognos_install_directory>\bin and <cognos_install_directory>\bin64 directory for *.dmp files.
In a Microsoft Windows environment, core dump files have a .dmp extension and the file name processID.dmp, such as BIBusTKServerMain_seh_3524_3208.dmp.
Remove all dump (*.dmp) files, make sure this does not end up in your Recycle bin.
Consider disabling the creation of core dump Files. These can be used for troubleshooting and can be turned on again as and when required.
Steps to Turn Off Core File Creation for IBM Cognos
On the server where IBM Cognos BI is installed, open the cclWin32SEHConfig.xml file from the <cognos_install_directory>\configuration directory.
In the configuration element, change the value of the environment variable setting to 0 (zero) so that it reads
<env_var name="CCL_HWE_ABORT" value="0"/>Save the file.
Restart Cognos
Cleaning up UDA Files
UDA temp files are created when a report is executed in Compatible Query Mode (CQM) where some or all of the processing has been set to take place locally (vs. on the database). In the case of large files, it would appear that one (or more of the reports) are:
a) returning very large amounts of data.
b) have the processing set to local.
Another possible reason for the files being created is the user canceling the report before it finishes executing.
Errors can appear in the Details section of the error screen or in the Cognos Server logs.
UDA-TBL-0004 There was a Write error while processing a temporary file
UDA-SOR-0001 Unable to allocate memory.
UDA temp files are supposed to recycle themselves, at the latest, at a restart of the Cognos service. However, if the report process ended unexpectedly, the UDA temp files need to be cleared up manually after the Cognos service has been stopped.
Resolving the Problem
If you have the ability to recycle your Cognos service, stop your service. Delete all temp files under <cognos installation> / temp and then restart your service.
Navigate to the <cognos_install_directory> on the dispatcher that you’d like to clean the files on <cognos_install_directory>\temp and <cognos_install_directory>\temp\XQE. However, there are times when the application has a delay in cleaning up these files.
In this case, end the BI Bus Processes on the dispatcher which will then release the hold that the application has on these files. You can then delete these files. Below are the locations where you could find these files.
<Cognos Installed directory> \temp, <Cognos Installed directory> \temp\XQE
DO NOT Delete any other temp files besides UDA, DMP. This needs to be done only when the Administration team reaches out with Hard Disk space issue.
Another suggestion would be to change the temporary file’s location in Cognos Configuration to point to a location outside the Cognos installation for example an entire mapped drive just for Temp files.
Should the issue re-occur and you are able to identify if this is down to a specific report, the administrator may have to contact the developer to see if he is able to optimize the design of the report and its source. This may include adding additional filters as a selection option to the report, which correlates to indexed fields in the database. Another suggestion to consider is to evaluate and determine with the report owner if it is feasible to reduce the number of rows returned for rendering. This may require some form of pre-aggregation on the source tables and setting your reports to instead execute off same.
Cleaning up CM Directory
The Content Manager (CM) is the Cognos Business Intelligence service that manages the storage of customer application data, including security, configuration data, models, metrics, report specifications, and report output.
Before using this functionality, set the Save report outputs to a file system property in IBM Cognos
Configuration to true.
Each output file also has an output descriptor of the same name, with an XML extension.
You must specify a location in Content Manager where copies of the report output files will be saved. The location applies to saved output originating from the selected Content Manager service. This location is represented by the CM.OutPutLocation parameter.
This parameter is mandatory if you want to save report output files in IBM Cognos software and specifies a location in IBM Cognos software where copies of report output files are saved.
Procedure to follow:
1) Start IBM Cognos Configuration
2) Click Data Access and then Content Manager
3) Select True for "Save report output to a file system"
4) Save and exit from Cognos Configuration
5) Restart Cognos Configuration
6) On Cognos Connection, select Launch---> Administration.
7) Select System and from the All Server drop-down menu, select Services and then Content Manager
8) From ContentManagerService drop-down menu, select "Set Properties"
9) Select Settings tab and then click Edit on "Advanced Settings" row.
10) Put the parameter name: "CM.OUTPUTLOCATION" in the Parameter column and a fully qualified path in the Value column.
This is the directory where the reports will be saved.
11) Click OK and run your reports.
Report outputs will always be written to the directory configured for each Delivery Service instance. In the context of this article, it is important to not forget that old report versions are not deleted from this location when new versions are saved. This location must be properly managed so that only selected report versions are kept.
Conclusion
There are many different aspects to server upkeep, for example, performing regular service checks or even daily health checks. One great tool we love to leverage at PMsquare that acts as a health check that’s always on is Thrive. Check it out here if you want to learn more about this awesome tool.
Overall, this can feel like a lot for a team to incorporate into their busy schedules. If you feel like server upkeep is a task that’s always put on the back burner (even though it shouldn’t be!), we recommend checking out our cost-efficient managed services offering. It’ll take care of software updates, maintenance, and environment administration, paired with our proactive services to ensure your systems running without the headache.
We hope you found this article to be informative. Be sure to subscribe to our newsletter for more PMsquare articles, updates, and insights delivered directly to your inbox.