Local timeline
Pretty cool to see the #cloudflare outage in these new charts. The `push` queue on #mastodon is showing heightened errors since 12:48. The `retry` queue is rapidly growing.
Built a Mastodon observability stack over the weekend for toot.community:
Grafana dashboards (web/sidekiq/streaming/edge), VictoriaMetrics rules, SLOs & alerts. The ultimate goal was to get meaningful symptom (user-facing) alerts and to learn about Jsonnet and Grafonnet.
Will publish the repository under https://github.com/toot-community after I find some time to polish things up later this week.
#devops #victoriametrics #mastoadmin #kubernetes #tootcommunity
Still one of my favorite photos of Donna, who unfortunately passed away last year. Perfect representation of “If I fits, I sits.”
RE: https://mastodon.social/@MonaApp/115555724255071879
The old "oh no, your lifetime purchase was for our OLD app.."
One thing I’ve learned is that your quality of life as an adult is very much linked to your ability to have difficult conversations with your peers.
Doing the outside cooking thing again today. A slow simmer bolognese sauce. Already looking forward to tonight.
Might as well bite the bullet and migrate directly to Gateway API. I suspect Kubernetes will eventually deprecate the Ingress resource too, since Gateway API offers that and more.
To the maintainers: Huge thanks for all your time maintaining this project all this time.
https://www.kubernetes.dev/blog/2025/11/12/ingress-nginx-retirement/
Should anyone want to try this, I used these two connectors:
1. https://github.com/matin/garth-mcp-server
2. https://github.com/eddmann/strava-mcp
I've been experimenting with MCP servers in Claude, which essentially allow you to connect data sources like Garmin and Strava to AI.
This week, I noticed that my heart rate was higher than usual at the same pace, so I asked Claude to analyze the data. To my surprise, it found a cause and supported its findings with ample data.
I'm not thrilled about needing to run more Z2 to recover from fatigue, but technologically, I find this _really_ cool.
Here's your occasional reminder to download a complete backup of your Mastodon data.
Go to `/settings/export` on your instance and hit request.
The Gitlab Helm chart requires a whole new degree. I get why it's complex, but I don't like that it's complex.
"The Philips Hue Pro bridge can detect movement in a room, just by measuring interference between multiple lights in the same room. With this, it can turn on lights when people enter the room."
..What? 🤯
New data suggests it's a crawler by Tencent, and there is no relation to the latest 4.5 release. Will research a bit more later.
Here's a view from seven days earlier. The emojis don't even rank in the top ten most requested URLs.
Another interesting adventure from the life of a #mastoadmin: "Where do all these requests come from?"
These spikes come and go on a fairly steady rhythm (each ~60 minutes). I found this to be interesting, so I investigated which URLs are being requested the most during these spikes.
Turns out: It's the emojis. I don't mind as they're being returned early by Varnish, but why so much? 2.5 million times per 24 hours seems like _a lot_.
Two red lines mark the Mastodon 4.5 upgrade.
Accumulated some tiredness in my body over the last few days, so this morning's tempo run turned into a threshold run.
Double down on the protein and rest for the next few days.