14 posts tagged with "monitoring"

Introduction​

There are different ways you can use to deploy the Prometheus monitoring tool in your environment. One of the fastest ways to get started is to deploy it as a Docker container. This guide shows you how to quickly set up a minimal Prometheus on your laptop. You can then extend that setup to add a monitoring dashboard, alerting, and authentication.

• Introduction
• Deploying Prometheus in a Docker Container
  • Basic Setup
    • Useful Tip
  • Separating the Configuration
  • Making the Data Storage Persistent
  • Further Configuration
• Conclusion
• References

Deploying Prometheus in a Docker Container​

Basic Setup​

Running Prometheus in Docker is very simple using this command

docker run -p 9000:9090 prometheus prom/prometheus:v3.0.0

This will pull and run the latest version (as of this writing) of Prometheus. You can access the Prometheus UI at localhost:9000. Note that the container port 9090 is forwarded to the localhost port 9000.

Introduction​

The Prometheus monitoring tool can store its metrics either locally or remotely. You can configure a remote data store using the remote_write configuration. This article describes the various data store options available as well as how to set up a remote store.

Overview of Remote Storage​

By default, Prometheus stores data locally wherever it is installed. The data directory can be configured by using the --storage.tsdb.path command line option when starting Prometheus. In practice you can use a separate disk for higher performance attached to the machine where Prometheus is running.

However, this may not be possible or optimal in all situations as you might want a data store that is more suited for time series data, and has larger storage capabilities for higher data retention. Prometheus would usually run in a standalone VM or a Kubernetes pod or a Docker container, and it would not have access to such data stores by default.

A remote store can add these capabilities to Prometheus. The remote storage option can be set by using the remote_write key in the Prometheus configuration YAML file.

Introduction​

Service discovery (SD) is a mechanism by which the Prometheus monitoring tool can discover monitorable targets automatically. Instead of listing down each and every target to be scraped in the Prometheus configuration, service discovery acts as a source of targets that Prometheus can query at runtime.

Service discovery becomes crucial when there are dynamically changing hosts, especially in microservices architectures and environments like Kubernetes. In Prometheus parlance, service discovery is a way of discovering "scrape targets".

For example, pods are created dynamically in Kubernetes as a result of new services being deployed and undeployed, autoscaling events, and errors causing pods to crash and go away. If you are using Prometheus for scraping pods in such an environment, Prometheus has to know which pods are running and scrapable at any given point in time. The Kubernetes service discovery pluging enables this. Similarly, there are SD plugins for other common environments.

You can use service discovery in Prometheus with the predefined plugins, or write your own custom ones using file or HTTP, depending on the situation.

Introduction​

Maintaining transparent communication about service availability is crucial for businesses of all sizes. Status pages are an important part of your communication strategy during times of outages and maintenance events.

You can choose to go with a fully managed status page provider, or host an open-source one yourself.

Open source status page providers offer a cost-effective and customizable solution. However, then can come with their own drawbacks. This guide explores open source status page providers in 2024 to help you choose the right tool for your needs.

• Introduction
• List of Open Source Status Page Providers
  • 1. Cachet
  • 2. Statping-ng
  • 3. Cstate
  • 4. Upptime
  • 5. Staytus
  • 6. Vigil
  • 7. Gatus
  • 8. Statuspal
  • 9. Uptime Kuma
  • 10. OneUptime
  • 11. Kener
  • 12. OpenStatus
• Choosing the Right Open Source Status Page Provider
• Conclusion

List of Open Source Status Page Providers​

1. Cachet​

Cachet is a popular open source status page system built with PHP and Laravel. It offers a clean, minimalist design and a robust feature set.

Introduction​

Downtime is inevitable but what sets successful businesses apart is how they handle it. A key part of incident management is incident communication with both internal and external stakeholders. A status page is a crucial tool for maintaining clear communication with users during outages or service interruptions. There are numerous status page providers available with different features. This article will guide you through best practices for selecting a provider that suits your needs.

The Importance of a Status Page​

An internal status page provider your colleagues and stakeholders in your organization to get a snapshot of of the current status. It can help reduce unnecessary back and forth between teams, and help people to prioritize their work better. It also creates internal transparency and trust between teams.

An external status page is crucial if you say you are committed to open communication with your end users or customers. Whether you are B2B or B2C, a public status page would be the first thing people would check if they face issues. Being open about incidents and your efforts to mitigate them build user trust. They can also decrease support ticket volume during incidents.

You can choose an open source status page provider, or one that is managed. This guide focuses on the factors to look at while evaluating managed providers.

Key Factors to Consider When Choosing a Status Page Provider​

1. Reliability​

Your status page needs to be accessible especially when your main services are down. Your provider should be able to guarantee a reasonable amount of

• Uptime SLA
• Globally distributed infrastructure for high availabilty
• Redundant systems to ensure failover and availability
• Scalability to handle increased traffic during major incidents

Introduction​

Incident updates on the public status pages of your cloud providers are often the first indication that they might have an outage. Providers also post updates about upcoming and ongoing maintenance on their status pages. Thus, monitoring your cloud status pages becomes crucial to your business operations. This article will guide you through the process of effectively monitoring such status pages.

• Introduction
• Identify Your Cloud Providers
• Locate Their Public Status Pages
• Understand the Status Page Structure
• Configure Notifications
  • Notification Challenges
• Best Practices
  • Filtering Your Monitors
  • Do Periodic Reviews
  • Have a Single View Across All Providers
• Include in Your Incident Response Plan
• Use a Monitoring Tool
  • The Benefits of Using a Monitoring Tool
• Conclusion
• FAQ

Identify Your Cloud Providers​

Work with your Dev/Ops/SRE and IT teams to come up with a comprehensive list of your cloud providers. Any service that is not managed by your teams is by definition a cloud service. Although we focus on Cloud providers - i.e. providers that let you deploy your services on their infrastructure - this article is equally applicable to any of your external SaaS vendors.

Introduction​

Modern businesses depend heavily on Software as a Service (SaaS). SaaS is not limited to being used by software development teams.
Almost all aspects of business operations - accounting, HR, payroll, marketing, IT, sales, support - depend on one or more SaaS applications. Given this dependency on SaaS applications, their uptime becomes tightly tied to a business's uptime. Any SaaS downtime can affect both a business's daily operations as well as the user experience.

How to check if a SaaS is experiencing downtime? Follow the steps below:

• Introduction
• Visit the SaaS Provider's Status Page
  • Locate the SaaS provider's Status Page
  • Understanding Status Pages
• Use External Monitoring Services
  • Setting Up a Monitoring Tool
• Check Social Media
• Run Manual Tests
• Incident Communication
• Conclusion
• FAQ
• Popular SaaS Service Statuses

Visit the SaaS Provider's Status Page​

The SaaS provider's status page will have first-hand information about ongoing issues.

Introduction​

A recent question in an SRE forum triggered this train of thought.

How do I deal with alerts that are triggered by internal patching/release activities but don't actually cause a downtime? If we react to these alerts we might not have time to react to actual alerts that are affecting customers.

I've paraphrased the question to reflect its essence. There is plenty to unravel here.

My first reaction to this question was that the SRE who posted this is in a difficult place with systemic issues.

Systemic Issues​

Without knowing more about the org and their alerting policies, let's look at what we can dig out based on this question alone

A few weeks ago we released a feature on IncidentHub which gives you a historical view of your monitored services' availability.

Why Was This Needed?​

On the dashboard where you can add services and channels, there is an overview panel that shows total incidents in the last 24 hours. You can get into a more detailed view by clicking on the button next to it. This opens up a popup where you can see active and resolved incidents - in the last 24 hours - and filter them by service.

This panel is good enough for a quick view on what's affecting your dependent services. However, sometimes there is a need to look back further. This is what the Availability page gives you - an overview of service health over the last 30 days.

Let's look at a few examples:

Chances are, most of your third-party cloud and SaaS dependencies are globally distributed and have many regions of operation. Chances are, your applications use a subset of a cloud or SaaS service. If you are monitoring such a service, why should you receive alerts for all regions or every single component in the service?

E.g. if you use Digital Ocean, you might be using Kubernetes in their US locations (NYC and SFO). You would want to know only when there is an outage in one of these locations. Digital Ocean's status page gives you the option to subscribe to outages across the board - it’s all or nothing. This is the case with most services with a few exceptions.

Choosing Specific Components to Monitor​

You can now choose which components/regions you wish to monitor in IncidentHub. Let us continue with our Digital Ocean example.

You can choose to monitor all components: