Most IT organizations are divided into operational units, to manage, design, and increase the efficiency of their business operations to meet customer requirements. There are many different organizational considerations depending on the company itself. Some of them can be organized by technical specialization, by activity, by services, by geography, or any combination of these. Naturally, these operational units are responsible for the problems occurred in their environment.
At OpsGenie we provide “Teams” as a way for the customers to represent their organizational structure, so that these operational units can deal with the problems using OpsGenie's alert/incident management and notification capabilities. Customers can define any number of teams.
To add members to a team, first the user should have been invited to the account. A user can be a member of multiple teams. Also you can give a "team admin" role to your members, to handle administrative tasks of the team.
An on-call schedule is used to determine who is on-call at a given time.
With OpsGenie an on-call schedule is automatically created when the team is created. OpsGenie supports creating and managing on-call schedules in just a few short steps. Admins (and team admins) can set up schedules with daily, weekly, and custom rotations. Also, you can specify the schedule to determine who should be notified according to the on-call schedule.
To create a schedule, go to the “On-Call Schedules” page, and click on the “Add Schedule” button.
Name: Give the schedule a name using only letters, numbers, underscores, and periods. You will use this name when configuring the on-call schedule as the recipient of an alert.
Timezone: Select the appropriate time zone and click save.
A rotation is group of people who rotate through the same shift. You can provide multiple rotations for the schedule; and rotations can be restricted to specified time periods. When an alert is escalated through the schedule, all on-call users in all the different rotations are notified (cumulative).
OpsGenie supports multiple rotations with different participants, rotations and time intervals to be defined for a schedule. When a schedule is being used to determine who should be notified, all on-call users from different rotations are notified
To define rotations first click on the plus sign button next to the “Rotations” section to begin populating your schedule.
As rotations are defined, OpsGenie automatically calculates and displays who will be on-call and when, according to the parameters below:
Starting date and time indicates when the rotation becomes active. Scheduled rotations are calculated from this date and time. The time is often particularly important as it becomes the handover time for on-call responsibilities (e.g. start of business day vs. start of actual day at midnight). Default rotation start time is 08:00.
End date and time indicates when the rotation ends and becomes inactive. No on-call rotation can be used after the specified end date of the rule. End date is optional; if it's left unspecified the rotation will continue indefinitely.
Participants can be users, groups, teams, and escalations. A user or group can be included multiple times for more frequent participation in the rotation. Order of the participants can be changed by dragging and dropping the participants in the list. Each participant is assigned a color and is represented with that color in the schedule preview.
Rotation can be daily, weekly, or custom. Select daily rotation if participants will rotate every 24 hours starting from the specified starting date and time. Select weekly rotation to rotate participants every week starting from the specified date and time. The custom rotation feature is useful to specify a different length of rotation (e.g. 8 hours, 3 days, 3 weeks, etc.) or to specify multiple start or end times and days of the week.
Rotations can be restricted to certain days and times of the week. Limiting the time frame of the rotations allows defining different on-call schedules at different time periods, such as working hours, off hours, and weekends. Rotations are cumulative. If the time frames for rotations overlap, all the rotations apply, hence there may be multiple participants assigned to be on-call at the same time.
Users can view and export schedules assigned to them by using “Your on-call schedule” button on the “My Profile” page.
OpsGenie allows schedules to be exported in iCalendar format which is supported by Google Calendar, Microsoft Outlook, and others. It includes the next 3 months of the schedule.
To export a schedule click on the Export icon on the right of the schedule:
A user can also export schedules assigned to him on the “Your On-Call Schedule page:
When multiple users or a group (with multiple users as members) are specified as the recipients of an alert, OpsGenie notifies each user at the same time according to their notification preferences. However, notifying users in a desired order is required in most cases, instead of notifying all of them at once.
Escalations are used to notify the recipients of each rule according to the given order and timeline to meet this requirement. After an escalation is added as a recipient to an alert, the escalation rules will notify their recipients when the specified time is over and the state of the alert meets the specified condition.
Each escalation rule can have one of the following as its recipient:
- A single user
- A group
- All members of a team
- On-call members of a schedule
- Next on-call member of a schedule
- Previous on-call member of a schedule
While adding rules to an escalation, a condition for the rule to notify its recipient should be specified. Each rule can have one of the following as its notify condition:
- Alert is not acknowledged
- Alert is not closed
If an alert is either acknowledged or closed when the time comes for an escalation rule with the condition “alert is not acknowledged,” the recipient of this rule will not be notified. On the other hand, if an alert is closed when the time comes for an escalation rule with the condition “alert is not closed,” the recipient of this rule will not be notified.
The following is an example escalation setup:
Escalations also have an option of repeating. When the time for the last escalation rule passes and there is still at least one rule with a matching condition of alert state, the escalation policy will be executed from scratch after the time for repeating after (if one is specified) passes. Please note that if the alert is acknowledged and all escalation rules have the condition of "if alert is not acknowledged" or if the alert is closed, the escalation will not be repeated.
When the recipients field of an alert is set to an escalation policy, OpsGenie uses that escalation policy to determine who should be notified when (in what order). In addition, a default team escalation policy is created automatically for each team. When a team is added to an alert, the escalation policy of the team is used to determine who should be notified, and when. The following is an example escalation of a team:
The default team escalation policy is a good example to understand escalations. Using this escalation policy, OpsGenie will first send notifications to the on-call user(s) based on the on-call schedule. On the example above, on-call users can view and acknowledge/close the alert. If/when the alert is acknowledged or closed, escalation policy stops. If the alert is NOT acknowledged or closed after 5 minutes, then OpsGenie will execute the second escalation step and notify the next user on rotation. Finally, if the alert is still not acknowledged or closed after 10 minutes, the third escalation step is executed and all members of the team are notified.
There are times when your on-call users go on vacation, need to swap shifts or cannot respond to alerts in general.
For these situations, OpsGenie provides ad-hoc schedule exceptions, allowing schedule modifications without altering your schedule (rotations) settings. Overrides can be defined by admins or team members
A team member can take the on-call responsibility for the time interval he/she specifies. Team admins can define overrides for other members of the team as well. A schedule override replaces all on-call users from different rotations of the schedule.
Schedule overrides can be created by users themselves, but admins can create them on behalf of the users as well.
OpsGenie also provides "Forwarding Rules" which enable users to forward all notifications to another user.
It is possible to define an override to take on-call from one or multiple rotations of the schedule. To manage this, specify rotation names while creating the override.
If an override is defined to take on-call from specific rotations, rotation names will be shown on the override tooltip. Also in the final schedule, overrides that are defined for specific rotations will be shown just on these rotations.
When an alert is created and assigned to a team, the team's escalation policy is run to determine who should be notified for that alert. Team assignment to an alert can be made through web UI, rest API, or integrations. An alert can be assigned multiple teams. Until the alert is acknowledged or closed, users are notified according to their notification preferences.
OpsGenie also provides an optional alert field called "recipients" to specify who should be notified for that alert. If the recipients field is used when an alert is getting created, recipients field supersedes the team's assignment. The team members would still have access to the alerts (can see the alerts and execute actions), but the recipients field is used to determine who should be notified. The recipients field can be used to specify different users, groups, schedules, or escalations instead of the team escalation policy. For existing alerts, add recipients and add teams actions can be used to notify other users that are may not be members of the team.
The notification tab will allow you to add your preferred method of notification (email, SMS, phone, mobile app), add rules, mute notifications, enable quiet hours, and stay up to date with your company’s news and announcements.
Yes, under the notifications tab you can set up “Notification Rules” which is the most important setting for you as a user. In OpsGenie, users have full control on how they get notified for each alert. This page empowers you to differentiate between alerts based on content, action, or event time. Rules follow a top- down flow- the first matched rule will create the alert, and OpsGenie will stop matching it to the rules below that.
A simple example would be the following: Say you would like to be notified differently for alerts that, contain “critical” in the message. In this case, you would set up a “New Alert” rule, on the top of the list, filtering for that condition, and giving custom ways and orders to notify yourself, if that happens. If the alert does not contain “critical” in the message, OpsGenie would jump to your second “New Alert” rule, which has a “match all” condition, and you would get notified based on the steps captured in that rule.
We at OpsGenie recommend that you do not use a voice call first, and to use either SMS, push, or email first, then any of the other notification methods minutes later.
OpsGenie can send notifications by email, SMS, phone calls, and push notifications where a user can view the notification. Users can also edit these contact methods under the notification tab.
- Users can enable/ disable contact methods at anytime
- If users have used any mobile apps, they can view the app info
As a user, you can configure when and how you want to receive notifications- to do this, you need to create a rule for each of your contact methods by specifying when to receive it. You can delay receiving notifications up to 1 day, you may have multiple rules, and you can also enable/ disable rules anytime.
OpsGenie will try each method, one by one, in the order you specify until you respond to the
- Click on the “+” on the left
- Then, add in the name of the alert and action (Please note: you may restrict to a specific time range)
- The under the “if” category select between either: “Match All Alerts,” Match One or More conditions below,” or “Match all conditions below.”
- The complete the action necessary to complete the rule
- Once you’ve completed the “If” category, select one or more notification methods such as email, sms, mobile, and/or voice.
Try it out for the following notification categories:
- New alert
- Acknowledge alert
- Closed alert
- Re-notified alert
- Assigned alert
- Add note
Users can mute notifications for a specified amount of time.
You can create alerts using the OpsGenie website. On the Alerts page, click on the “New Alert” button- The Message and Recipients are the only mandatory fields. The question mark buttons provide tips on what to enter on each field.
Message field is limited to 130 characters. Recipients can be groups or individual users. Recipients field auto completes; recipient names can be entered as username (email@example.com) or as full name (John Smoth).
Click on the Advanced button to see the optional alert fields.
Alias field is a user defined identifier for the alert and there can be only one alert with open status with the same alias, allows alert deduplication. Limited to 512 characters.
Entity field can be used to specify the entity alert is related to. For example, name of the server, application, etc.
Tags field can be used to attach labels to alerts for easier identification and categorization of alers. Use commas in between for multiple tags. You can overwrite Quiet Hours settings for urgent alerts by adding OverwriteQuietHours tag.
Actions field can be used to specify the list of actions a recipient can execute to respond to an alert. For multiple actions separate with commas.
Description field can be used to provide a detailed description of the alert, anything that may not have fit in the Message field which is limited to 130 characters.
Extra properties field can be used to store additional alerts properties. Enter the name of the property in the first field, the value in the second, and click Add.
- Message: Server apollo average CPU utilization has been over 60% for 15 minutes
- Recipients: You - Start typing your name or username , select your name when appears on list.
- Click on Advanced button.
- Tags: urgent, ec2
- Actions: acknowledge , stop , restart
- Description: Detailed description of the problem
- Extra Properties: Enter "Region" to left column - name , "AWS East Region 1c" to right column - value , Click on Add
- Extra Properties: Enter "Service" to left column - name , "R101" to right column - value , Click on Add
*You might add some other Extra Properties
This will create an alert and specify you as the only recipient.
Depending on the notification methods and rules you have specified on your Profile, OpsGenie will send notifications till you view the alert.
OpsGenie can integrate with any software or service that is able to send emails. OpsGenie can process incoming emails and create, acknowledge, and close alerts from incoming emails by applying user defined rules. Note that OpsGenie also provides product specific Integration Plugins for many popular monitoring tools as well as a Web API and command line utility (lamp) for integration.
Email based integrations configuration can be done on Integrations page.
Basic configuration to create alerts:
1) Please create an OpsGenie account if you haven't done already
2) Go to OpsGenie Email Integration page
3) Each email integration is assigned an email address that you can customize
4) Recipients field is used to specify "who" should be notified for the alerts created by this email. Recipients can be users, groups, schedules, or escalations. This field has auto-complete support hence available options are displayed as you type. You can also use "all" as the recipient to notify all the users.
5) Optionally, you can change the name of the integration.
6) Click on "Save Integration"
You're done! OpsGenie would create an alert for each email that is sent to the specified email address. By default, email integration uses email subject as the alert message, and email body is put into the alert description field. If the incoming email comes in HTML format, OpsGenie also attached the original email to the alert.
- If an email integration is disabled, emails sent to that email address are ignored by OpsGenie.
- If Suppress Notifications is check, OpsGenie would create alerts, but users would not receive any notifications for the alerts.
OpsGenie supports responses with text messages to acknowledge and close OpsGenie alerts by responding to the text messages sent by OpsGenie.
SMS notifications sent by OpsGenie include a short URL that opens up the web browser and displays the alert details. Users can view alert details, and execute any standard or custom action using the mobile OpsGenie app. As an alternative, users can reply to the text message without having to run the mobile app to acknowledge or close the alert
The short URL that is included as part of the SMS notification has the data necessary to identify the alert the user is responding to. For example, if the SMS has the following short URL - http://opsg.in/i/4421 - user should use 4421 as the identifier. To acknowledge the alert user can reply via SMS with the following text:
To close the alert, user can reply with the following text:
When OpsGenie receives the SMS from the user, appropriate action is executed and the outcome is sent to the user as a text message as well.
OpsGenie alerts page is the list page for all alerts that are visible or assigned to a user. You can apply filters or execute bulk actions (close multiple alerts, ack all, etc.) on the alerts page.
You can use the "My alerts" filter to show only the alerts you're a recipient of. This means both alerts you're directly the recipient of and alerts assigned to your schedule, team, group, or escalation.
Note that the "My alerts" filter is only usable by owner or admin users.
Clicking on an alert will open the details window for that alert.
You can select and execute many actions on an alert on the Alerts page. Just select the action you want to take from the dropdown.
On Alerts page, you can select one or more alerts and execute actions on them. This includes all actions you can do from the details page of an alert, aside from executing custom alert actions. The advantage of the alerts page is the ability to execute these actions on multiple alerts. You can, for example, select multiple alerts and acknowledge them all with a single button.
"Ack All" button allows you to acknowledge all alerts with a single action. Upon clicking and confirming the prompt, OpsGenie starts an asynchronous process that assigns ownership of every open alert to you.
"Close All" button allows you to close all alerts with a single action. Upon clicking and confirming the prompt, OpsGenie starts an asynchronous process that closes every open alert.
OpsGenie push notifications enable you to quickly review your alerts and respond to them with same speed. Helping you to take your actions as quickly as possible.
When you first open & login to the app you will see a message saying "Device registered to receive notifications”.
Once you receive the “Device registered” message, navigate to the Right Menu: “Notification Rules” you should see your device listed to receive "New Alert" notifications. Also you can scroll down to Contact Methods: “Mobile App” and see your device is also listed there. If you are not seeing your device on this section you will not be able to receive push notifications. You can close & open Opsgenie App again, for your device to register for OpsGenie notifications.
In order to receive other notifications Acknowledgement, Close Alert, etc. please modify your settings from OpsGenie Web > My Profile > Notification Rules.
When there is an alert you will be notified by your phone.
Tapping on notification will navigate you to details of that alert. You can execute actions by tapping on actions button.
Push notifications play an essential role for OpsGenie iOS v2. When you login with a valid OpsGenie account for the first time, your mobile phone will be registered automatically to receive push notifications from OpsGenie. You can keep track of recently created alerts, executed actions on them, and start/end of schedule periods easily by the help of push notifications. You can always change your settings for push notifications from your Profile->Notification page.
When a notifications comes, the sound you have selected as notification sound will play if your device is not in silent mode.
You may execute actions on alerts via notifications without any need of opening the application. Because this functionality has come with iOS8, executing actions on OpsGenie notifications is not available on iOS7.
You may swipe the notification left on Notifications Center (while lock screen is active or Notifications Center is opened) to see quick actions (Ack and Close) and tap one of them to execute.
If the device is not on locked but application is not active (i.e. application is closed or in background), you may swipe down the notification to see quick actions (Ack and Close) and tap one of them to execute.
If the OpsGenie application is active (i.e. on foreground), you may swipe the notification left to see quick actions (Ack, Close, and Mute) and tap one of them to execute.
Furthermore, you can change notification style of OpsGenie notifications to “Alert” style from Settings->OpsGenie->Notifications on your device. When you select “Alert” style for OpsGenie notifications, notifications will appear as iOS alerts. You can tap “Options” to display available actions for received notification.