IBM Tivoli Netcool Integration

Tivoli Netcool / OMNIbus is a carrier-class service and business assurance system that collects and consolidates enterprise-wide event and alarm information in real-time from many different network data sources and presents a simplified view of this information to operators and administrators.

Tivoli Netcool / OMNIbus has automation functions that can perform intelligent processing on managed alerts. Netcool triggers form the basis of the ObjectServer automation subsystem. Triggers automatically fire (execute a trigger action) when the ObjectServer detects an incident associated with a trigger. In a trigger, you can execute SQL commands and call procedures in response to the change. Netcool triggers have also the ability to call external procedures, which opens the way to use triggers as a notification system.

What does Opsgenie offer IBM Tivoli Netcool users?

Opsgenie's Netcool plugin provides a bi-directional integration with Netcool which enables users to receive notifications for Netcool events via email, SMS, phone call, and iPhone/Android push notifications, and interact with events (acknowledge, write to journal, etc.) directly from mobile devices using Opsgenie apps.

Scenario 1: Email Integration

Netcool events can simply be forwarded to Opsgenie via email to create alerts in Opsgenie. This integration does not require any client components and relies on Netcool's email capabilities. Netcool's default installation comes with a temporal trigger called "mail_on_critical" which periodically checks critical events that are not unacknowledged after 30 minutes and send mails via "send_mail" procedure. This trigger is an example of how to create Opsgenie alerts by sending emails. To send Netcool incidents to Opsgenie via email, have a correctly configured email sending tool (mailx, sendmail, etc.) and the process agent. The configuration of the process agent, SMTP server, etc. is out of scope for this document.

The steps below describe how to create Opsgenie alerts for critical events which are unacknowledged for 30 minutes:

• Please create an Opsgenie account if you haven't done so already.
• Go to Email Integration page and add one. You can find more information on Opsgenie email integration support here.

📘

An alternative for this step is to add the integration from the Team Dashboard of the team which will own the integration. To add an integration directly to a team, navigate to the Team Dashboard and open Integrations tab. Click Add Integration and select the integration that you would like to add.

🚧

For Free and Essentials plans, you can only add the integrations from the Team Dashboards, please use the alternative instructions given above to add this integration.

• Modify the action section of "mail_on_critical" trigger, using the email address from the Opsgenie Email Integration (shown below) and enable it. Note that we use the "Summary" field instead of "Netcool Email" text for the email subject, which will be transformed to Opsgenie alert message.

That's all. If set up correctly, OpsGenie creates alerts for every critical Netcool event which is unacknowledged for 30 minutes.

If a problem arises, check that the email utility is working correctly. Change the contact email to a mailbox to test your Netcool procedure is working correctly. Please don't hesitate to contact us, if a problem persists.

Scenario 2: Integration via Lamp & Marid

Opsgenie also provides the Netcool Integration plugin that provides integration via the web API (HTTPS) using Opsgenie-provided tools: Lamp and Marid. When installed and appropriately configured, the plugin creates Opsgenie alerts via Lamp command line utility and enables recipients to execute actions like acknowledge, write to journal, etc. on Netcool events directly from their mobile devices or Opsgenie web console.

Installation

The steps below describe how to create alerts and execute actions on alerts using Opsgenie Netcool integration package.

• Download Opsgenie Netcool Integration files, which includes:
• Lamp - Opsgenie command line utility to create alerts
• Marid - Opsgenie Integration Server
• Netcool Integration shell scripts
• Unzip Netcool Integration package to a directory on the Netcool server.
• Lamp installation:
• Netcool Integration Files contains a customized version of lamp, you need to install this version.
• Install lamp by referring to Lamp Documentation - Installation steps- Do not download lamp again
• Make sure your Netcool shell scripts have correct LAMP_HOME, your lamp installation directory
• Create an API Integration and obtain apiKey, then put into the opsgenie-integration/conf/opsgenie-integration.conf file
• Verify Lamp is working: Create a test alert, using the lamp utility from the command line, make sure the alert is visible on the Opsgenie Alert dashboard. Please refer to Lamp Documentation for more information
• The integration package provides a UNIX based shell script which will be executed by ObjectServer procedures. It's a very simple script which makes little modifications to the command line arguments passes them to Lamp. Other operating system users can prepare very similar versions of the script in their preferred scripting language. Prepare netcoolShellScripts/opsgenie shell script like the following:
• Copy the shell script to a directory in the system where your Process Agent is installed. This path will be pointed when Netcool procedures are configured
• Configure LAMP_HOME to point your lamp installation directory
• Make sure the file is executable by your Process Agent
• Create a Netcool procedure with name "create_opsgenie_alert". Use the "opsgenie" shell script, and configure the "host" property appropriately

create or replace procedure create_opsgenie_alert
(in serial integer, in node character(255), in severity integer, in summary character(255), in count integer)
executable '/usr/bin/opsgenie'
host 'localhost'
user 0
group 0
arguments serial, '\'' + node + '\'', severity, '\'' + summary + '\'', count;

• Create a Netcool temporal trigger similar to "mail_on_critical" with name "opsgenie_alert_on_critical" like the following:

create or replace trigger opsgenie_alert_on_critical
group default_triggers
priority 1
comment 'Send opsgenie alerts about critical alerts that are unacknowledged after 30 minutes'
every 10 seconds
evaluate
   select Serial, Node, Severity, Summary, Identifier, Count from alerts.status where
   Severity = 5 and Grade < 2 and Acknowledged = 0 and LastOccurrence <= (getdate -(60*30))
bind as criticals
begin
   for each row critical in criticals
   begin
      execute create_opsgenie_alert(critical.Serial, cricritical.Node, critical.Severity,    critical.Summary, critical.Count);
      update alerts.status via (critical.Identifier) set Grade=2;
   end;
end;

The steps described so far cover the alert creation part of the integration. If everything goes well Opsgenie alerts are created for every critical Netcool event which is unacknowledged for 30 minutes. Check Lamp logs if a problem is encountered.

To execute actions on Netcool events, install and appropriately configure Marid. The integration package provides a Groovy - a powerful Java based scripting language - script called "netcoolActionExecutor.groovy" which accesses the ObjectServer at the database level and executes the specified event action. Install Marid into a system which can access ObjectServer's database. Follow the steps below to complete the installation:

• Make sure that you have at least one alerts created on Opsgenie from Netcool.
• Verify that alerts created on Opsgenie with Netcool Integration have actions "acknowledge", "deacknowledge" , "add to task list" and "remove from task list" on Opsgenie alert details.
• Netcool Integration Files contains a customized version of Marid, you need to install this version.
• Install Marid by referring to Marid Documentation - Installation steps. Do not download marid again.
• Create an Opsgenie Marid Integration, make sure your Marid integration Alert Filter processes only Netcool alerts.
• Put Marid Integration apiKey into the opsgenie-integration/conf/opsgenie-integration.conf
• Configure database access parameters in opsgenie-integration/conf/opsgenie-integration.conf file

netcool.db.driver=com.sybase.jdbc2.jdbc.SybDriver
netcool.db.host=192.168.1.178
netcool.db.port=4100
netcool.db.username=root
netcool.db.password=root

• Start Marid.

Testing actions from Opsgenie to Netcool

• Go to any Opsgenie app Iphone, Android, Opsgenie Dashboard . Find an alert created with Netcool Integration.
• Execute acknowledge action on Opsgenie app
• Verify that the action is visible on Netcool event journals. The result looks like:

Executing actions from Opsgenie app :

Action Result on Netcool Event List :