Sector: Infrastructure · Consorzio Metis
Multi-site Kubernetes infrastructure
Design and operation of K3s clusters distributed across multiple sites, managed with Rancher on Proxmox virtualization: cross-site networking over a mesh VPN, internal DNS, SSO and high availability.
- Multi-site
- Sites
- K3s + Rancher
- Stack
- SSO
- Access
Professional experience as an employee at Consorzio Metis. Identifying details (host and cluster names, addresses, exact topology) have been omitted.
Context
An organization spread across multiple physical sites needs to run containerized services close to where they are used, while keeping management centralized: one control point, uniform policies, federated access.
What it covers
- K3s clusters (lightweight Kubernetes distribution) located at the various sites, centrally managed through Rancher.
- Underlying virtualization layer on Proxmox VE.
- Cross-site connectivity via a mesh VPN (NetBird), with in-cluster DNS (CoreDNS) and cluster networking (flannel).
- Centralized single sign-on for service access, plus TLS and certificate management for the control plane.
Technical challenges
Kubernetes on top of a mesh VPN. Making cluster networking coexist with a mesh network requires care with routing rule ordering and conflicts between virtual and physical interfaces: one wrong detail and nodes see each other but pods don’t.
Cross-site stability. Agents on remote nodes must reconnect cleanly after network interruptions between sites; internal DNS resolution must stay consistent even when a link goes down.
High availability. Critical services must survive the loss of a node or an entire site, which drives scheduling, replicas and persistence decisions.
Stack
K3s · Rancher · Proxmox VE · NetBird (mesh VPN) · CoreDNS · flannel · SSO/IdP