A little bit of everything this week, with an emphasis on logging, Kubernetes, and time-series databases. Enjoy! 🍩☕📈

This issue is sponsored by:

Chronosphere logo

What does the future of OpenTelemetry hold? Chronosphere's Co-founder and CEO, Martin Mao, breaks down his predictions for open standards, open protocols and more specifically, OpenTelemetry in our newest video series, Observability for What’s Next. Watch the clip here.



Articles & News on monitoring.love

Observability & Monitoring Community Slack

Come hang out with all your fellow Monitoring Weekly readers. I mean, I’m also there, but I’m sure everyone else is way cooler.

From The Community

TEMPLE: Six Pillars of Observability

Because everyone knows that having six pillars is twice as good as only having three. I bet I could come up with another few and venture capitalists would be banging my door down.

P.S. All kidding aside, this is a great review of the “pillars of observability”, adding more context and broadening our definition of the data we use to understand our systems better.

Relational Databases vs Time Series Databases

We read about time-series databases all the time, with an implicit acknowledgement that they’re different from traditional relational databases. Here’s a helpful post that compares their raisons d’être.

Understanding Kubernetes Evicted Pods

An excellent article on Kubernetes Pods eviction, along with a pointer on how to monitor them in Prometheus.

Creating a Human and Machine freindly logging format

Some tips and best practices to consider if you’re able to log in JSON (aka events).

People-Centric Software Operations Culture

Good culture should always strive for a sustainable environment for employees, but it’s easy to take traditional responsibilities (e.g. on-call) for granted. This article is a good reminder of the behaviors we want to model and incentivize among our engineering teams.

Root Cause Analysis: How can an idea that’s wrong be so useful?

There are some really insightful bits in here, particularly about how we reason about complex systems (with a nod to Dr. Richard Cook), before veering off the rails into a predictable “existing tools suck, buy mine” spiel (sigh).

How to easily configure Grafana Loki and Promtail to receive logs from Heroku

As an ex-Herokai and long-time Heroku fan, I’m excited to see support added to Grafana Loki to accept log drains from the popular PaaS offering.

A Practical Guide to Observability in DevOps

If you’re looking to run a pure open source monitoring stack, this guide should help you get started. No big surprises here, but it covers basically all the tools you’d expect.

Grafana security releases: New versions with moderate severity security fixes for CVE-2022-35957 and CVE-2022-36062

Moderate severity fixes affecting Grafana Auth Proxy and RBAC. There appears to be a manual workaround to mitigate the RBAC privilege escalation attack vector.

Monitoring Amazon EMR Step Failures

If you work with any teams using Apache Spark on Amazon EMR, here’s a helpful pattern for monitoring those Spark jobs.

See you next week!

– Jason (@obfuscurity) Monitoring Weekly Editor