Live Commander Observation Queue Monitoring

Title

In certain situations, you may find it necessary to capture the state of the system as the observation queue approaches an overflow state. This can occur when you have an issue with the observation queue overflowing, but it occurs irregularly with an undetermined cause that renders observation recording ineffective. Live monitoring of the observation queue provides the following benefits:

Approximate capture of when the queue trips beyond a pre-determined threshold: By periodically querying the /rest/v3/system-health endpoint, we can pick up on when a critical event occurs for further analysis of manual or scheduled processes that could contribute to an overflow state.
Capturing a snapshot of the system logs: The monitoring script will also package the Commander logs to a ZIP file, allowing for easy analysis of the state of the system prior to (but in the process of) overflow, allowing for the identification of issues.

Preparing the Script Files

Download and extract the attached LiveObservationMonitoring.zip to a convenient folder on the Commander application server. You will have two files extracted:

cmdrObservationMonitoring.ps1
ObservationMonitoringTask.xml

Edit the following parameters in cmdrObservationMonitoring.ps1:

Parameter	Description
BaseURL	The base URL for accessing Commander.
user	A username for a credential that shares the same username and password for Commander and SMTP. These credentials will be used for querying Commander, and authenticating with the SMTP server for sending emails.
pass	The password for a credential that shares the same username and password for Commander and SMTP. These credentials will be used for querying Commander, and authenticating with the SMTP server for sending emails.
Logfile	The name of the log file that will contain records of the observation queue size at the time of execution.
CmdrLogPath	The absolute path where Commander’s logs are kept (i.e. `C:\program files\embotics\commander\tomcat\logs`).
DiagnosticBundle	The name of the compressed ZIP file containing the Commander logs at the time the observation queue exceeded 1000 items.
logPath	The absolute path where the observation queue log and ZIP file are kept (i.e. `C:\Support`).
BypassCert	Used for ignoring unsigned certificates. Default to “Yes” for compatibility unless you’re running into issues.
SMTPServer	The IP address of the mail server used for sending messages.
SMTPPort	The port associated with the IP address of the mail server used for sending messages.
From	The sender email address.
MailSubject	The subject of the overflow warning email.
To	The recipient e-mail address.

Configuring the Scheduled Task

Next, a scheduled task is configured to run the scripts prepared in the section above.

On the Commander application server, launch the Windows Task Scheduler.
Right-click the Task Scheduler Library and click Import Task.

Browse to and select the ObservationMonitoringTask.xml file extracted onto the Commander server in the previous section.
Update the task by:
1. Designating a local user account.

Switch to the Triggers tab and update the existing daily trigger to specify a start time 15 minutes from the current time. Ensure Repeat task every: 15 minutes is set as shown below.

Under the Actions tab, edit the existing task if necessary to specify the file path to the cmdrObservationMonitoring.ps1 script.
Click OK to save your changes and entering the credentials for the account specified in the script.

When the task starts its first execution, you should see an observationMonitoring log file indicating that the monitoring script is working as expected:

A ZIP file will also appear when the observation queue first exceeds 1000 items.

You should also see a Last Run Result in Task Scheduler indicating that the task was run successfully:

Responding to a Detected Issue

When the observation queue backlog exceeds 1000 items (or the number configured in the script), an email will be sent off. As an additional precaution, the /rest/v3/system-health endpoint as well as the logPath for where the ZIP file will be placed should be monitored in case the email fails to send due to an invalid configuration.

Once an overflow is detected, please perform the following steps as soon as possible:

Generate a diagnostic bundle in Commander. It’s possible that the overflow hasn’t happened yet, and this could possibly provide even more information through an additional snapshot in time.
Collect the diagnostic bundle, the observationMonitoring log, and the vcmdrLogs ZIP file, and supply all three to Engineering for further analysis.

Was this helpful?

Support Hub

Product

Snow Commander

Deployment type

On-premises

Visibility

Guest

URL Name

Live-Commander-Observation-Qeueu-Monitoring

Are you sure you want to cancel the case creation?

Case closed successfully

Action performed successfully

Upload Attachment

Maximum file size allowed is 3 MB.

File type not supported.

Supported file types:

Documents (.txt, .doc, .docx, .pdf), Images (.jpg, .png), Comma Separated Files (.csv) Speadsheets (.xlsx, .xls)

Are you sure you want to close this case

Get Hotline support

Products	Region	Phone Numbers
FlexNet Operations FlexNet Embedded FlexNet Publisher FlexNet Connect FlexNet Code Insight InstallAnywhere InstallShield	North America *	+1 630-332-2513 (toll) +1 877-279-2853 (toll-free in North America)
	Europe *	+44 1925 944367 (toll) +44 800 047 8642 (toll-free in Europe)
	Japan *	+81 3-4540-5335 (select option 2)
	Australia *	+61 3 9895 2177 +61 1800 560 603 (toll-free in Australia)
Usage Intelligence (formerly Revulytics) Compliance Intelligence	Please use the Case Portal to submit your support ticket or reach out to your Revenera contact.

Preparing the Script Files

Configuring the Scheduled Task

Responding to a Detected Issue

Related Articles

Please wait...

Case id: 00001065