Tag: Kubernetes
A central repository for container orchestration patterns, cluster architecture, and cloud-native tooling.
🏗 Cluster Engineering Link to heading
- Control Plane: Etcd performance, API server tuning, and scheduler logic.
- Distributions: Specific notes on Talos Linux and standard upstream K8s.
- Networking (CNI): Cilium eBPF implementation, load balancing, and ingress strategies.
🚀 Workload Management Link to heading
- GitOps: ArgoCD application sets and synchronization patterns.
- Storage (CSI): Persistent volume management via TrueNAS (iSCSI/NFS) and local path provisioners.
- Observability: Prometheus, Grafana, and Loki stack configurations.
📚 External Resources Link to heading
- Kubernetes Documentation - The official source of truth.
- CNCF Landscape - Cloud Native Computing Foundation ecosystem.
- 💀 How I Accidentally DDoSed My Own Kubernetes Cluster
Introduction For the past few weeks, my production Kubernetes cluster has been fighting a ghost. Every few days (or sometimes hours), the entire Control Plane would commit suicide. Critical components like the scheduler …
- 🔒Locking Down Mastodon: Cilium Network Policies & Audit Mode
The Goal: Defense in Depth Running Mastodon on my bare metal cluster exposes it to the wider internet for federation. If the Mastodon container were ever compromised (RCE, supply chain attack, etc.), I need to ensure the …
- 🚨Alerting on Kyverno Policy Validation Failures with Loki
Context We needed to create an alert for Kyverno policy violations across our clusters. The goal was to catch when a resource fails validation (e.g., policy="require-requests-limits") and surface relevant metadata like …
- Cilium Network Policies
apiVersion: cilium.io/v2 kind: CiliumNetworkPolicy metadata: name: mastodon-egress-federation namespace: mastodon spec: egress: - toCIDRSet: - cidr: 0.0.0.0/0 except: - 192.168.0.0/16 - 100.64.0.0/10 - 10.244.0.0/16 - …
- Cloudflare Tunnel - Troubleshooting - Cannot determine default origin certificate path
Cloudflare Tunnel - Troubleshooting - Cannot determine default origin certificate path Problem I installed a Cloudflare Tunnel on my Kubernetes cluster using the Helm Chart and I used the token given by the Cloudflare …
- Cloudflare Tunnel for Kubernetes
Cloudflare Tunnel for Kubernetes Problem Hosting services on Kubernetes is amazing but running Kubernetes inside a home network can be quite risky as it's incredibly difficult to ensure the environment is secure, …
- Disaster Recovery: Talos etcd Corruption & Kyverno CNI Deadlock
Incident Summary: Talos Control Plane & CNI Deadlock Overview During a physical disk migration for the TrueNAS VM (running on the Proxmox bare-metal host), severe I/O latency spikes caused a cascading failure across the …
- enable_audit.sh
#!/usr/bin/env bash set -euo pipefail NAMESPACE="mastodon" SELECTOR="app.kubernetes.io/name=mastodon" echo "🔍 Finding Mastodon pods and enabling Cilium Policy Audit Mode..." # 1. Loop through pod names directly for …
- Kubernetes - List Abnormally Terminated Pods Across All Namespaces
Problem I want to get a list of all pods that have terminated abnormally along with their restart reason and count. Solution Run the following command: kubectl get pods --all-namespaces -o …
- Post-Mortem: Recovering a Corrupted Talos etcd Cluster
The Incident During a recent physical disk migration for a TrueNAS VM running on Proxmox, the resulting I/O latency spikes and storage disruption caused a massive ungraceful state change. This led to a complete loss of …
- Post-Mortem: The Admission Webhook CNI Deadlock
The Incident Following a successful bare-metal restore of a Talos etcd cluster, the control plane was healthy, but the entire pod network remained offline. Cilium (running in strict kube-proxy replacement mode) was …
- 🧠 Analysis: Etcd Consensus in a Hybrid Optane/SATA Cluster
Architecture: Node 1 (CP1): Intel Optane (Ultra-Low Latency ~10µs) Node 2 (CP2): Intel Optane (Ultra-Low Latency ~10µs) Node 3 (CP3): SATA SSD rpool (Variable Latency 10ms - 500ms+) 1. The Physics of Consensus (Speed) …