Proxmox VE 9.2 SDN Fabrics: Routed, Encrypted Networking

Use Proxmox VE 9.2 SDN Fabrics to build a routed, self-healing node network — OSPF or OpenFabric, WireGuard encryption, and route-maps that keep storage private.

On this page
  1. What a fabric actually is
  2. Build a fabric
  3. Keep storage and management traffic where it belongs
  4. The homelab payoff: a full-mesh Ceph network
  5. What’s next

For a long time my cluster’s inter-node network was held together by static routes I’d typed onto each node and then prayed I’d never have to remember. It worked — until a cable or a NIC hiccuped, and then it very much did not, because a static route has no idea the path it points at is dead. It just keeps pointing.

Proxmox VE 9.2’s SDN Fabrics are the grown-up answer. You declare which nodes and interfaces participate and pick a routing protocol, and Proxmox writes the router config on every node for you. The nodes then discover each other, exchange routes, and — the part I actually wanted — reroute automatically when a link goes down. This flagship walks the concepts and the exact build path, and it corrects a misconception I see everywhere about where WireGuard fits. Every menu path and behavior below is from the official Proxmox SDN documentation.

A link dies — the fabric routes around it, no static routes to fixnode1loopbacknode2loopbacknode3loopbacknode4loopbackdirect link downnode2 → node4 → node3: the routing protocol finds the alternate path on its own.
First: make these values your own

Node names, interfaces, and any addresses here are examples. Use your own node names, the real interface names on your hardware, and CIDR ranges from your own address plan (the docs use ranges like 10.0.0.0/24 for loopbacks). If a value looks specific to one machine, it’s a placeholder to change, not a literal to copy.


What a fabric actually is

A fabric is a routed network between your nodes, defined once in the datacenter and pushed to every node. Proxmox VE 9.2 gives you three protocols, and picking the right one is most of the decision:

  • OpenFabric — based on the IS-IS protocol and tuned for spine-leaf data-center layouts. It’s the modern, low-fuss choice for a homelab mesh; router IDs can be IPv4 or IPv6.
  • OSPF — the classic, universally understood link-state protocol. Reach for it if you already speak OSPF or need to interoperate with existing OSPF gear.
  • WireGuard — and here’s the misconception worth killing.
WireGuard is encryption, not routing

It’s tempting to call WireGuard a third routing protocol. It isn’t. The docs are explicit: WireGuard “does not provide dynamic routing by itself,” but is used in conjunction with OSPF or BGP. WireGuard builds the encrypted tunnels between nodes; OSPF or BGP runs on top to distribute the routes. So a “WireGuard fabric” is really an encrypted underlay carrying a dynamic routing protocol — the private road, with OSPF doing the signposting. If you want your inter-node routing encrypted (say, across sites), that’s the combination.


Build a fabric

The whole thing is driven from Datacenter → SDN → Fabrics, and it writes the low-level router config so you never touch FRR by hand.

1Add the fabric5 min

Under Datacenter → SDN → Fabrics, click Add Fabric and choose your protocol (OpenFabric for a fresh homelab mesh). Set the fabric-level basics — router IDs, the loopback prefix, and area settings where the protocol asks for them.

2Add nodes and their interfaces10 min

With the fabric created, add each participating node with the + button, selecting which of its interfaces join the fabric and giving each interface an address in CIDR notation. Proxmox automatically creates the dummy loopback interfaces the routing protocol uses as stable per-node identities.

3Apply — and know what it does2 min

Back on the main SDN panel, hit Apply Configuration. This is the moment it becomes real: Proxmox generates and writes the FRR configuration (/etc/frr/frr.conf) on every node and reloads it. From here the nodes exchange routes and converge on best paths themselves.

Apply is cluster-wide

Applying an SDN change regenerates the routing config on all participating nodes at once, not just the one you’re clicking from. That’s the point — the fabric is a cluster object — but it means you plan the address scheme up front rather than nudging one node at a time.


Keep storage and management traffic where it belongs

Once nodes are exchanging routes, the next question is which routes. You rarely want every network advertised to every peer — your management or Ceph storage network should not leak out to, say, a guest-facing peer. Proxmox VE 9.2 gives you two filtering tools for the BGP/EVPN controller:

  • Prefix Lists (Datacenter → SDN → Prefix Lists) — an ordered, named list of CIDR entries, each permit or deny. This is your “which networks are we even talking about” list.
  • Route Maps (Datacenter → SDN → Route Maps) — ordered entries that match routes against criteria and then permit, deny, or modify them (for example, adjusting a metric) before they’re advertised.

A controller references these for its incoming and outgoing route policy, so you decide exactly which prefixes reach which peers.

A route-map advertises one network and withholds the sensitive onesroutes this node knowsguest netmanagement netstorage / Ceph netroute-mappeersees only the guest netmanagement + storage: denied, never advertised

The homelab payoff: a full-mesh Ceph network

The single most useful place a homelabber meets fabrics is a full-mesh Ceph network. Ceph wants a fast, reliable private network between nodes, and wiring nodes directly to each other (no switch) is a classic way to get 10-gig-plus between three or four boxes cheaply. Historically that meant fiddly static routes on every node; a fabric turns it into a declared object that heals itself if a direct link drops. If you’re building a small Ceph cluster, this is the modern way to lay its network.


What’s next

You’ve got a routed, self-healing, optionally-encrypted network under your cluster. Next in this flagship run we point Proxmox Backup Server 4.2’s brand-new S3 support at a self-hosted object store: PBS 4.2 backing up to MinIO running on your own NAS.


Related posts:

Comments

Comments are powered by GitHub Discussions — sign in with a GitHub account to join the conversation.