HA & config sync

High availability is a single system-wide toggle. Enable it, pick the dedicated HA port, and two mirrored nodes run identical configuration and behave as one system. Failover is layer-2 MAC and IP takeover — no CARP, no virtual IPs, no per-node addresses. Tested and shipping on Business+.

Why not CARP

CARP works, and it adds concepts operators trip over: virtual routers, VHIDs, advskew, and a shared virtual IP layered on top of each node's own address. SecureGuard follows the Astaro/Sophos UTM model instead. There is exactly one IP and one MAC per data segment, and it lives on whichever node is active. Simpler to hold in your head, simpler to configure, and it matches the actual claim: the two boxes are the same system.

How failover works

1. Both nodes render the same cluster MAC and IP for each data port. The MAC is derived deterministically from the port name, so both nodes compute the same value without coordinating.
2. rc.conf pre-addresses the data ports but does not bring them up. The HA controller starts every node passive, with data ports down — so the cluster MAC is present on the network from exactly one node, or none.
3. Nodes heartbeat over the dedicated HA link and mirror connection state with pfsync.
4. On promotion, the controller programs the cluster MAC and IP onto each data port, brings them up, and sends gratuitous ARP (and unsolicited IPv6 neighbor advertisements). Switches relearn the port; because the MAC has not changed, neighbor ARP caches do not even churn.
5. On demotion it brings the data ports back down.

Because pf state is already synced, established sessions survive the cutover rather than being re-established.

Config is mirrored, byte for byte

The whole configuration document is content-addressed and synced by the HA controller, so both nodes hold it identically. Revisions are what make that cheap: syncing config is shipping snapshots.

The one thing that differs is which box is node 1 versus node 2, and that is a local fact recorded at /var/db/rubix/node-id, deliberately kept out of the synced document. The HA link itself is configured by the controller — link-local addresses by node number — not by the mirrored data-plane config. Everything else, including data interfaces, MACs, addresses and rules, is identical.

Mirrored hardware is required: both nodes must have the same interface layout. A pre-commit consistency gate rejects a commit whose interface topology differs from the peer, rather than letting you find out at failover.

Split-brain avoidance

The same MAC on two switch ports is a fault, so promotion is deliberately conservative:

• A node promotes only when the peer has been absent for the failover hold time and it holds the HA link.
• If the HA link drops but peer traffic is still observed, the node stays passive.
• The HA link should be redundant — directly cabled, or an aggregate — so that losing it is rare enough to be treated as a fault rather than a routine event.

Virtual deployments

Failover reprograms interface MAC addresses, so the hypervisor must permit that:

• VMware / ESXi — on the port group, set MAC address changes: Accept and forged transmits: Accept.
• Hyper-V — enable MAC address spoofing on the VM's network adapters.

Without these, the takeover frames are dropped by the virtual switch and a failover presents as a total loss of connectivity with no obvious cause on either node.