brainstate

deedee-ops/home-ops · source

Snapshot 0 · 2026-11-04T12:47:27Z · YAML · WTFPL

Content fingerprint757323f6d42018d555560c9d2e54f0d3d2ea6f24e5f32a229e076cd931f8b1ef

GitHub description at capture

GF-approved geeked homelab k8s cluster deployed with Talos Linux; automated via FluxCD, Renovate and GitHub Actions 🤖

README at capture

<!-- markdownlint-disable MD013 MD033 MD041 -->
<div align="center">
  <img src="https://raw.githubusercontent.com/deedee-ops/home-ops/refs/heads/assets/k8shappy.png" alt="kubepepe" />
  <br>
  <sup><sup>
    Art by <a href="https://twitter.com/SkeletalGadget">@SkeletalGadget</a>
  </sup></sup>

### My Home Operations Repository ☸

_... managed with [Flux](https://fluxcd.io/), [Arcane](https://getarcane.app/), [OpenTofu](https://opentofu.org/), [Renovate](https://github.com/renovatebot/renovate) and [Forgejo Actions](https://forgejo.org/docs/latest/user/actions/) 🤖

</div>

<div align="center">

[![Discord](https://img.shields.io/discord/673534664354430999?label&logo=discord&logoColor=white&color=blue)](https://discord.gg/home-operations)&nbsp;&nbsp;
[![Talos](https://kromgo.ajgon.casa/badges/talos_version)](https://talos.dev)&nbsp;&nbsp;
[![Kubernetes](https://kromgo.ajgon.casa/badges/kubernetes_version)](https://kubernetes.io)&nbsp;&nbsp;
[![Flux](https://kromgo.ajgon.casa/badges/flux_version)](https://fluxcd.io)&nbsp;&nbsp;

</div>

<div align="center">

[![Age](https://kromgo.ajgon.casa/badges/cluster_birth_age)](https://github.com/home-operations/kromgo)&nbsp;&nbsp;
[![Uptime](https://kromgo.ajgon.casa/badges/cluster_uptime_age)](https://github.com/home-operations/kromgo)&nbsp;&nbsp;
[![Nodes](https://kromgo.ajgon.casa/badges/cluster_node_count)](https://github.com/home-operations/kromgo)&nbsp;&nbsp;
[![Pods](https://kromgo.ajgon.casa/badges/cluster_pod_count)](https://github.com/home-operations/kromgo)&nbsp;&nbsp;
[![CPU](https://kromgo.ajgon.casa/badges/cluster_cpu_usage)](https://github.com/home-operations/kromgo)&nbsp;&nbsp;
[![Memory](https://kromgo.ajgon.casa/badges/cluster_memory_usage)](https://github.com/home-operations/kromgo)&nbsp;&nbsp;
[![Power](https://kromgo.ajgon.casa/badges/cluster_power_usage)](https://github.com/home-operations/kromgo)&nbsp;&nbsp;
[![Alerts](https://kromgo.ajgon.casa/badges/cluster_alert_count)](https://github.com/home-operations/kromgo)

</div>
<!-- markdownlint-enable MD013 MD033 -->

---

## <img src="https://fonts.gstatic.com/s/e/notoemoji/latest/1f4a1/512.gif" alt="💡" width="20" height="20" /> Overview

This is a mono repository for my home infrastructure and Kubernetes cluster. I try to adhere to Infrastructure as Code
(IaC) and GitOps practices using tools like [Arcane](https://getarcane.app/), [OpenTofu](https://www.opentofu.org/),
[Kubernetes](https://kubernetes.io/), [Flux](https://github.com/fluxcd/flux2), [Renovate](https://github.com/renovatebot/renovate),
and [Forgejo Actions](https://forgejo.org/docs/latest/user/actions/).

> [!NOTE]
> Old [ArgoCD](https://argo-cd.readthedocs.io/en/stable/) is not maintained anymore,
> but it is [available here](https://github.com/deedee-ops/home-ops/tree/argocd-legacy) for reference.

---

## <img src="https://fonts.gstatic.com/s/e/notoemoji/latest/1f331/512.gif" alt="🌱" width="20" height="20" /> Kubernetes

My Kubernetes cluster is deployed with [Talos](https://www.talos.dev). This is a semi-hyper-converged cluster, workloads
and block storage are sharing the same available resources on my nodes while I have a separate server with ZFS
for NFS/SMB shares, bulk file storage and backups.

There is a [great template made by onedr0p](https://github.com/onedr0p/cluster-template) if you want to try and follow
along with some of the practices I use here.

### Core Components

- **Networking & Service Mesh**: [cilium](https://github.com/cilium/cilium) provides eBPF-based networking,
  [tailscale](https://tailscale.com/) carries public ingress from a VPS edge back home (and everything else remote),
  and [external-dns](https://github.com/kubernetes-sigs/external-dns) keeps DNS records in sync automatically.
  All egress traffic is carefuly filtered using network policies.
- **Security & Secrets**: [cert-manager](https://github.com/cert-manager/cert-manager) automates SSL/TLS certificate
  management. For secrets, I use [external-secrets](https://github.com/external-secrets/external-secrets) with
  self-hosted [OpenBao](https://openbao.org/) to inject secrets into Kubernetes.
- **Storage & Data Protection**: [rook](https://github.com/rook/rook) provides distributed storage for persistent
  volumes, with [kopiur](https://github.com/home-operations/kopiur) handling backups and restores.
  [spegel](https://github.com/spegel-org/spegel) improves reliability by running a stateless, cluster-local OCI image mirror.
- **Automation & CI/CD**: [forgejo-runner](https://code.forgejo.org/forgejo/runner) runs
  self-hosted GitHub Actions runners directly in the cluster for continuous integration workflows.

### GitOps

[Flux](https://github.com/fluxcd/flux2) watches the clusters in my [kubernetes](./kubernetes/) folder
(see Directories below) and makes the changes to my clusters based on the state of my Git repository.

The way Flux works for me here is it will recursively search the `kubernetes/clusters/<cluster name>` folder
until it finds the most top level `kustomization.yaml` per directory and then apply all the resources listed in it.
That aforementioned `kustomization.yaml` will generally only have a namespace resource and per-cluster Flux kustomization
for subset of apps used in said cluster. Under the control of those Flux kustomizations there will be a `HelmRelease`
or other resources related to the application which will be applied.

[Renovate](https://github.com/renovatebot/renovate) watches my **entire** repository looking for dependency updates,
when they are found a PR is automatically created. When some PRs are merged Flux applies the changes to my cluster.

### Docker

Machines which are not feesible to be maintained by kubernetes (like NAS), are managed by [Arcane](https://getarcane.app/)
and [Docker Compose](https://compose-spec.io/) files. Directories are organized per application, and instance specific options
are configured directly in Arcane.

### Directories

This Git repository contains the following directories.

```sh
📁 bootstrap      # initial set of files necessary to kickstart the cluster
📁 docker         # application compose files with configurations
📁 kubernetes
├── 📁 apps       # application templates with base rules shared among clusters
├── 📁 clusters   # per-cluster configurations of said apps
└── 📁 components # re-useable kustomize components
📁 opentofu       # opentofu plans for external services like cloudflare
📁 talos          # per-cluster talos configurations
```

---

<!-- markdownlint-disable MD013 -->
## <img src="https://fonts.gstatic.com/s/e/notoemoji/latest/1f636_200d_1f32b_fe0f/512.gif" alt="😶" width="20" height="20" /> Cloud Dependencies
<!-- markdownlint-enable MD013 -->

While most of my infrastructure and workloads are self-hosted I do rely upon the cloud for certain key parts of my setup.
This saves me from having to worry about three things. (1) Dealing with chicken/egg scenarios, (2) services I critically
need whether my cluster is online or not and (3) The "hit by a bus factor" - what happens to critical apps
(e.g. Email, Password Manager, Photos) that my family relies on when I no longer around.

| Service                                         | Use                                                            | Cost           |
|-------------------------------------------------|----------------------------------------------------------------|----------------|
| [BorgBase](https://www.borgbase.com/)           | Borg Backups                                                   | $80/yr         |
| [Cloudflare](https://www.cloudflare.com/)       | DNS                                                            | Free           |
| [DigitalOcean](https://www.digitalocean.com/)   | VPS: public ingress edge, RustDesk server, tailnet peer relay  | $48/yr         |
| [GitHub](https://github.com/)                   | Hosting this repository and continuous integration/deployments | Free           |
| [healthchecks.io](https://healthchecks.io/)     | Heartbeats monitoring                                          | Free           |
| [Migadu](https://migadu.com/)                   | Email hosting                                                  | $19/yr         |
| [Tailscale](https://tailscale.com/)             | Remote access, VPS to home transport                           | Free           |
|                                                 |                                                                | Total: ~$14/mo |

### Networking

<!-- markdownlint-disable MD033 -->
<details>
  <summary>Click to see a high-level network diagram</summary>

```mermaid
graph TD
    %% Class Definitions
    classDef wan fill:#f87171,stroke:#fff,stroke-width:2px,color:#fff,font-weight:bold;
    classDef core fill:#60a5fa,stroke:#fff,stroke-width:2px,color:#fff,font-weight:bold;
    classDef agg fill:#34d399,stroke:#fff,stroke-width:2px,color:#fff,font-weight:bold;
    classDef switch fill:#a78bfa,stroke:#fff,stroke-width:2px,color:#fff,font-weight:bold;
    classDef device fill:#facc15,stroke:#fff,stroke-width:2px,color:#000,font-weight:bold;
    classDef vlan fill:#1f2937,stroke:#fff,stroke-width:1px,color:#fff,font-size:12px;

    %% Nodes
    WAN[🛜 netia<br/>1Gbps/300Mbps WAN]:::wan
    UCG[📦 UCG Ultra]:::core
    AGG[🔗 USW Pro Max 16 PoE]:::agg
    NAS[💾 NAS<br/>1 Server]:::device
    KUBE[☸️ Kubernetes<br/>3 Nodes]:::device
    SW[🔌 USW Flex 2.5G]:::switch
    DEV[💻 Devices]:::device
    WIFI[📶 WiFi Clients]:::device

    %% Subgraph for VLANs
    subgraph VLANs [VLANs]
        direction TB
        HOME[Home Network<br/>192.168.2.0/24]:::vlan
        IOT["IoT Network<br/>192.168.3.0/24"]:::vlan
        CAMERAS["Cameras Network<br/>192.168.4.0/24"]:::vlan
        KUBERNETES[Kubernetes Network<br/>192.168.42.0/24]:::vlan
        VPN[VPN Network<br/>192.168.69.0/24]:::vlan
        GUEST[Guest Network<br/>192.168.99.0/24]:::vlan
        MGMT[Management Network<br/>192.168.254.0/24]:::vlan
    end

    style VLANs fill:#111,stroke:#fff,stroke-width:2px,rx:0,ry:0,padding:20px;

    %% Links
    WAN -.->|WAN| UCG
    UCG --> AGG
    AGG -- 2x10G LACP --- NAS
    AGG --> DEV
    AGG --> WIFI
    AGG -- 2.5G --- SW
    SW --> KUBE

    %% Style the bonded links thicker
    linkStyle 2 stroke-width:4px,stroke:34d399;

    %% Move VLANs below the graph, to the middle
    KUBE -.-> KUBERNETES
    linkStyle 7 stroke-width:0,opacity:0;
```

</details>
<!-- markdownlint-enable MD033 -->

---

## <img src="https://fonts.gstatic.com/s/e/notoemoji/latest/1f30e/512.gif" alt="🌎" width="20" height="20" /> DNS

In my cluster there are two instances of [ExternalDNS](https://github.com/kubernetes-sigs/external-dns) running.
One for syncing private DNS records to my `UCG Ultra` using [ExternalDNS webhook provider for UniFi](https://github.com/kashalls/external-dns-unifi-webhook),
while another instance syncs public DNS to `Cloudflare`. This setup is managed by creating ingresses with two specific
classes: `internal` for private DNS and `external` for public DNS. The `external-dns` instances then syncs
the DNS records to their respective platforms accordingly.

---

<!-- markdownlint-disable MD013 -->
## <img src="https://fonts.gstatic.com/s/e/notoemoji/latest/2699_fe0f/512.gif" alt="⚙" width="20" height="20" /> Hardware
<!-- markdownlint-enable MD013 -->

<!-- markdownlint-disable MD013 MD033 -->
<details>
  <summary>Click here to see my server rack</summary>

  <img src="https://raw.githubusercontent.com/deedee-ops/home-ops/refs/heads/assets/rack-20260928.jpg" align="center" width="250px" alt="rack" />
</details>
<!-- markdownlint-enable MD013 MD033 -->

| Device                            | Num | OS Disk Size | Data Disk Size                  | Ram  | OS            | Function             |
|-----------------------------------|-----|--------------|---------------------------------|------|---------------|----------------------|
| Intel NUC12WSHi5                  | 3   | 512GB NVME   | 1TB (rook-ceph)                 | 64GB | Talos         | Kubernetes           |
| AMD Ryzen + GB B550I Aorus Pro AX | 1   | 1TB SSD      | 2x26TB ZFS (mirrored) + 4TB SSD | 64GB | TrueNAS SCALE | NFS + Backup Server  |
| JetKVM + AIMOS HDMI KVM Switch    | 1   | -            | -                               | -    | -             | KVM for Kubernetes   |
| UniFi UCG Ultra                   | 1   | -            | -                               | -    | -             | Router               |
| UniFi USW-Pro-Max-16-PoE          | 1   | -            | -                               | -    | -             | 1Gb+2.5Gb PoE Switch |
| UniFi Flex Mini 2.5G              | 1   | -            | -                               | -    | -             | 2.5Gb k8s Switch     |

---

## <img src="https://fonts.gstatic.com/s/e/notoemoji/latest/1f31f/512.gif" alt="🌟" width="20" height="20" /> Stargazers

<!-- markdownlint-disable MD013 MD033 -->
<div align="center">

<a href="https://star-history.com/#deedee-ops/home-ops&Date">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=deedee-ops/home-ops&type=Date&theme=dark" />
    <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=deedee-ops/home-ops&type=Date" />
    <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=deedee-ops/home-ops&type=Date" />
  </picture>
</a>

</div>
<!-- markdownlint-enable MD013 MD033 -->

---

<!-- markdownlint-disable MD013 -->
## <img src="https://fonts.gstatic.com/s/e/notoemoji/latest/1f64f/512.gif" alt="🙏" width="20" height="20" /> Gratitude and Thanks
<!-- markdownlint-enable MD013 -->

Thanks to all the people who donate their time to the [Home Operations](https://discord.gg/home-operations) Discord community.
Be sure to check out [kubesearch.dev](https://kubesearch.dev/) for ideas on how to deploy applications
or get ideas on what you could deploy.

---

<!-- markdownlint-disable MD033 -->
<div align="center">

[![DeepWiki](https://img.shields.io/badge/deepwiki-purple?label=&logo=deepl&style=for-the-badge&logoColor=white)](https://deepwiki.com/deedee-ops/home-ops)

</div>
<!-- markdownlint-enable MD033 -->