Integrate Your Monitoring System With PagerDuty Using Events API V2
PagerDuty's Events API V2 lets you push events from your monitoring systems to PagerDuty. You can push such events when there is a triggered, updated, or resolved incident.
The lifecycle of an incident will typically go through these states
State | Triggered By | Source |
---|---|---|
Triggered | Automatic | Monitoring system |
Acknowledged | On-call Engineer | PagerDuty app/Phone call |
Updated | Automatic | Monitoring system |
Resolved | On-call Engineer | PagerDuty app/Phone call |
You can either use any of the PagerDuty client SDKs to send events, or roll out your own.
Self-hosted and SaaS monitoring tools have inbuilt PagerDuty integration where you need to provider the API key.
A typical event push will look look like this (example in NodeJS):
import { event } from "@pagerduty/pdjs";
.....
event({
"data": {
"routing_key": "Your-Routing-Key-Here",
"event_action": "trigger",
"dedup_key": DEDUP_KEY,
"payload": {
"summary": "Event processor in us-east-1",
"source": "rnmd-2398.xyzcloud.io",
"severity": "critical",
"timestamp": "2024-07-17T08:42:58.315+0000",
},
"links": [
{
"href": "https://incidenthub.cloud/dashboard",
"text": "Go to dashboard",
},
],
},
.....
When your monitoring system sends this event to trigger an incident, it's important to have a unique DEDUP_KEY. This field determines whether subsequent events for this incident will be grouped together in PagerDuty. When your system sends an update, or a resolved event, the DEDUP_KEY must match the one sent during the trigger call. In other words, the DEDUP_KEY must be unique per incident.
IncidentHub integrates with PagerDuty and uses the incident's public URL as the DEDUP_KEY as that is unique globally, and also remains the same for an incident. Each incident update event has the same DEDUP_KEY.
Let us look at a Google Cloud example. An incident affecting Anthos Service Mesh in Nov 2023 went through 4 updates including trigger and resolve. The URL remained the same for the incident as it went through the lifecycle.
References