Skip to content

Relay Link

How the agent keeps a WireGuard link to its assigned relay, which carries the device's published services. Code: internal/relaylink/ in edge/gateway.

A relay is needed only while the device publishes a service. Without one the keeper does nothing.

The keeper

Run (source.go) acts on three triggers:

Trigger Interval Purpose
Tick 5 min (RetryAfter) Ask Nexus RequestRelay for the current assignment and act on it
Link watch 5 s (WatchEvery) Detect a dead relay (see failover)
Nudge on demand Nexus pushes relay_reassess; the keeper asks again at once

The assignment carries the relay id, public key, host, WireGuard port, the rung-3 tunnel URL and token, and the peer configuration. An assignment missing the id, key, host or port is refused. The host is chosen IPv4 first, then IPv6, then the name.

On each tick decide picks one of:

Condition Action
No link, or the assigned relay changed Climb
Link up, same relay, on rung 2 or 3, last success over 6 h ago Re-probe for a better rung
Link up, same relay, otherwise Nothing: a working link is left alone

Rungs

Rung Transport
1 WireGuard straight to the relay on its assigned WireGuard port
2 WireGuard to the relay on UDP 443
3 WireGuard over a WebSocket tunnel (TLS) to the relay; WireGuard's endpoint is a local 127.0.0.1 port bridged by udpws

There is one interface, wg99, with PersistentKeepalive 25. Each rung counts as up only after a completed WireGuard handshake: a filtered UDP port accepts a send and drops it silently, so nothing less proves the path. Each attempt is capped at 15 s (RungTimeout).

The bare tunnel address on wg99 gets the prefix length of the allowed-IPs range that contains it (hostprefix.go), because OpenBSD wg(4) installs no routes for peers.

Climb

Climb (climb.go) tries rungs in the order given by Order (ladder.go): the remembered rung first when the memory is for the same relay and less than 6 hours old, otherwise 1, 2, 3. It stops at the first rung that handshakes.

When a climb replaces a working rung, the old transport is torn down after the new one is up, never before, and Down removes the interface only if that rung owns it. A rung-3 tunnel left running would hold one of the relay's forward slots for the life of the agent.

Peer propagation. A relay learns a gateway's WireGuard peer from Nexus on its heartbeat (every 30 s). A gateway that climbs to a relay it has no valid memory for (a new assignment, a drain, or a first climb) can arrive before its peer does and fail every rung. A failed climb in that case is retried once after 35 s (PeerPropagation) rather than at the next tick.

Re-probe

On rung 2 or 3, after 6 hours the keeper tries the better rung again, so a site whose firewall changed takes the faster path. The re-probe brings up the better rung, then takes down the current one, then remembers the new rung; if the better rung fails, the current one is re-established. A fruitless re-probe still refreshes the memory's timestamp.

probe.go implements a non-disruptive pre-check (a PRBE UDP packet the relay echoes on the rung's port), but serve.go does not configure a PortProber, so the re-probe acts on the live interface.

Rung 3 tunnel

tunneller.go runs the WebSocket client that bridges WireGuard UDP over TLS.

  • Idempotent start. Start returns the existing tunnel when it is still carrying and the URL and token are unchanged, so a tick on rung 3 does not reconnect. A changed URL or token, or a tunnel that stopped carrying, is replaced.
  • At most one tunnel. Two would bridge to the same WireGuard port and the replies would go to whichever read first.
  • The URL and bearer token come from the assignment and are opaque to the agent; the relay verifies the token.

Remembered rung

/var/lib/edge-agent/relay-rung.json holds one record {working_rung, at, relay}. It survives restarts. A different relay or a record older than 6 hours is ignored, and the next success overwrites it.

Relay failover

The link watch (failover.go) pings the relay's tunnel address 10.80.0.1 through the tunnel (ping -n -q -c 1 -w 2) every 5 s.

  • The relay is declared dead after 30 s without a reply (DeadAfter). If the probe has never answered, the link counts as dead only once WireGuard stops carrying.
  • After a random delay of up to 15 s, the keeper calls RequestRelay naming the relay unreachable, and Nexus assigns another.
  • If Nexus answers with the same relay, it backs off 15 s, doubling to 5 min.
  • A nudge marks the relay unreachable only if the watch agrees.

Moving to another relay also moves the device's published-service DNS records to the new relay's address (done by Nexus); see Relay Architecture.

See also