7 posts tagged with "product"

Introduction​

For the last two months, we have focused on improving the onboarding experience for users so that they can get started with monitoring with minimal effort. We have also added several improvements in the backend to make the service more robust and reliable. Some of the usability improvements are driven by user feedback. Others incorporate what we would personally like to see in such a monitoring service. We have also improved the dashboard user experience.

• Introduction
• Easier onboarding
  • Service Component Autodetection
  • Commonly Used Services
  • Test Channels
• Dashboard Improvements
  • Services With Incidents
• Reliability Improvements
  • Alert Emails for Webhook Failures
  • Bug Fixes and Robustness Improvements

Introduction​

Staying on top of your third-party Cloud and SaaS service outages is crucial to maintain the reliability of your own applications. Like many modern teams, Slack might be your communication tool of choice. You can keep up with such incidents by pushing these events to a Slack channel.

There are different ways of pushing incident events to Slack. In this article we will explore how to integrate IncidentHub incident lifecycle events using an incoming webhook. An incoming webhook can be used to send incident trigger, update, and resolve events to a specific Slack channel.

Note that IncidentHub also has an option to integrate with custom webhooks, which is different from Slack's webhooks. If you are using Slack, choose the Slack option. For a custom webhook server, choose the Webhook option. The format of the Slack webhook payload is different from that of the Slack webhook.

Slack Incoming Webhook Configuration​

You must have the correct permissions on your Slack workspace to be able to do this.

Follow these steps to configure an incoming webhook in your Slack workspace.

Introduction​

Staying on top of your third-party Cloud and SaaS service outages is crucial to maintain the reliability of your own applications. If Discord is your communication tool of choice, you can keep up with such incidents by pushing these events to a Discord channel.

Discord webhooks allow external applications to send messages to specific channels within a Discord server. This article describes how to integrate Discord as a channel in your IncidentHub account using webhooks.

Note that IncidentHub also has an option to integrate with custom webhooks, which is different from Discord's webhooks. If you are using Discord, choose the Discord option. For a custom webhook server, choose the Webhook option.

Discord Server Webhook Configuration​

You must have the correct permissions on your Discord server to be able to do this.

A few weeks ago we released a feature on IncidentHub which gives you a historical view of your monitored services' availability.

Why Was This Needed?​

On the dashboard where you can add services and channels, there is an overview panel that shows total incidents in the last 24 hours. You can get into a more detailed view by clicking on the button next to it. This opens up a popup where you can see active and resolved incidents - in the last 24 hours - and filter them by service.

This panel is good enough for a quick view on what's affecting your dependent services. However, sometimes there is a need to look back further. This is what the Availability page gives you - an overview of service health over the last 30 days.

Let's look at a few examples:

Chances are, most of your third-party cloud and SaaS dependencies are globally distributed and have many regions of operation. Chances are, your applications use a subset of a cloud or SaaS service. If you are monitoring such a service, why should you receive alerts for all regions or every single component in the service?

E.g. if you use Digital Ocean, you might be using Kubernetes in their US locations (NYC and SFO). You would want to know only when there is an outage in one of these locations. Digital Ocean's status page gives you the option to subscribe to outages across the board - it’s all or nothing. This is the case with most services with a few exceptions.

Choosing Specific Components to Monitor​

You can now choose which components/regions you wish to monitor in IncidentHub. Let us continue with our Digital Ocean example.

You can choose to monitor all components:

Why should you monitor your third-party Cloud and SaaS vendors if you are in SRE/Ops?

As part of an SRE team, your primary responsibility is ensuring the reliability of your applications. What makes you responsible for monitoring services that you don't even manage? Third-party services are just like yours - with SLAs. And outages happen, affecting you as well as many others who depend on them.

It's a no-brainer that you should know when such outages happen to be on top of things if/when it affects your running applications.

Most of your third party dependencies will have a public status page or a Twitter account where they publish updates on their outages. Here are some seemingly easy ways to monitor these pages

• Subscribe to the RSS feed of these pages
• Follow the Twitter account
• Sign up for Slack, Email, SMS notifications on the status page itself if the page supports these

If you have a software-based business, you are using at least a few cloud based tools. It does not matter if you are a solo developer, or part of a 50-member team in a large organization. Take this random list and chances are you are using at least half of them:

• Zoom
• Google Workspace
• Slack
• Public cloud/PaaS - GCP/[AWS]/Azure/Render/Heroku/Railway/DigitalOcean/Hetzner
• PagerDuty/Opsgenie
• Cdnjs
• DockerHub
• GitLab/GitHub
• TravisCI/CircleCI/Semaphore
• Let’s Encrypt

Your entire business - irrespective of org or market size - including your development tools, collaboration/communication tools, infrastructure and hosting, monitoring, even email - is dependent on services that you don’t control. They are provided by other vendors.

Of course, you pay for some of them and they all have SLAs. Having an SLA does not translate to 100% uptime. Companies will try their best to meet SLAs - which promise a percentage of uptime (usually 99.xx). There are going to be incidents in your providers at some point, and the effect will cascade to the service that you provide to your customers. This means that your own product's SLA can be breached due to causes outside your control.