Skip to content

How It Works

This page draws the whole system as it runs today. Every arrow is a call that exists in code: the label names the RPC or protocol, and the credential it carries where that matters.

For per-component detail see Components.

Component overview

graph LR
    subgraph Users
        Browser[Operator browser]
        Visitor[Visitor of a<br/>published service]
    end

    subgraph Cloudflare
        Portal[portal<br/>SvelteKit on Pages]
        Realtime[realtime Worker<br/>GatewayRoom DO per device]
        DNS[(Cloudflare DNS)]
    end

    subgraph "Scaleway, per environment"
        Nexus[nexus<br/>serverless container]
        Authz[authz<br/>serverless container]
        Bucket[(OTA bucket<br/>digilan-agent-releases-env)]
        Relay[relays<br/>relayd on instances]
    end

    subgraph "Scaleway, shared"
        Zitadel[Zitadel<br/>auth.digilan-edge.com]
        RDB[(digilan-shared RDB<br/>one database per env)]
    end

    subgraph Site
        Agent[edge agent<br/>OpenBSD gateway]
        LAN[LAN host]
    end

    CI[edge/gateway CI]
    LE[Let's Encrypt]

    Browser -->|HTTPS| Portal
    Browser -->|OIDC code + PKCE| Zitadel
    Browser -->|Connect RPC<br/>Bearer Zitadel token| Nexus
    Browser -->|WSS viewer / operator<br/>single-use ticket| Realtime

    Nexus -->|JWKS, Management API| Zitadel
    Nexus -->|Connect RPC, Bearer API key| Authz
    Nexus --> RDB
    Authz --> RDB
    Nexus -->|RealtimePushService.Push<br/>X-Internal-Key| Realtime
    Realtime -->|GatewayInternalService<br/>X-Internal-Key| Nexus

    Agent -->|WSS /gateway/id<br/>api_token| Realtime
    Agent -->|GatewayDeviceService<br/>Bearer api_token| Nexus
    Agent -->|presigned GET| Bucket
    Agent <-->|WireGuard, rung 1/2/3| Relay
    Agent -->|reverse proxy| LAN

    Relay -->|RelayService<br/>Bearer relay token| Nexus
    Nexus -->|gateway records, dns-01 TXT| DNS
    Nexus -->|ACME| LE

    Visitor -->|resolve name| DNS
    Visitor -->|TLS, routed by SNI| Relay

    CI -->|upload signed binary| Bucket
    CI -->|PublishAgentRelease<br/>X-Release-Key| Nexus

What is not on the diagram:

  • Zitadel runs as its own serverless container with its own RDB instance, and sends mail through Scaleway Transactional Email.
  • radius-ctrl and Monsieur Wifi run on dev only. See radius-ctrl and Monsieur Wifi.

Device enrolment

A device enrols with a bootstrap token and lands in the holding zone. Putting it in an organization is a separate admin step. Details: Gateway Enrollment.

sequenceDiagram
    actor Admin as Platform admin
    participant Portal
    participant Nexus
    participant DB as Postgres
    participant Agent as edge agent
    participant DO as realtime DO

    Admin->>Portal: Admin > Enrollment: create bootstrap token
    Portal->>Nexus: PortalAdminService.CreateBootstrapToken
    Nexus->>DB: enrollment_codes row (type bootstrap)
    Nexus-->>Portal: token (shown once)

    Note over Agent: USB first boot, or<br/>edge-agent setup --addr --realtime --token
    Agent->>Nexus: GatewayEnrollmentService.Enroll(token, gateway_info)
    Nexus->>DB: check revoked / expired / max_uses
    Nexus->>DB: device gw-xxxxxxxx, status holding, org NULL,<br/>SHA-256 of api_token, uses + 1
    Nexus-->>Agent: gateway_id + api_token (plaintext, once)
    Agent->>Agent: write identity.json

    Agent->>Nexus: GetConfig, GetDesiredState (Bearer api_token)
    Agent->>DO: WSS /gateway/{id}?token=api_token
    DO->>Nexus: ValidateGatewayToken
    Nexus-->>DO: valid
    DO-->>Agent: socket accepted

    Admin->>Portal: Admin > Devices: assign to an org
    Portal->>Nexus: PortalGatewayService.AssignGateway
    Nexus->>DB: org_id set, status assigned

A device prepared ahead of time (CreateDevice, CloneDevice) takes the holding-zone unit over with ClaimGateway instead of AssignGateway. See Device Lifecycle.

Config push, apply, confirm

Saving creates a draft. Applying pushes it through realtime. The operator must confirm within 60 s, or the agent reverts it. Details: Config Lifecycle.

sequenceDiagram
    actor User
    participant Portal
    participant Nexus
    participant DB as Postgres
    participant RT as realtime Worker
    participant DO as GatewayRoom DO
    participant Agent

    User->>Portal: Save
    Portal->>Nexus: SaveGatewayConfig
    Nexus->>DB: gateway_configs version N, status draft

    User->>Portal: Apply
    Portal->>Nexus: ApplyGatewayConfig
    Nexus->>DB: status pending, devices.desired_config_version = N
    Nexus->>RT: RealtimePushService.Push(apply_config)
    RT->>DO: /push/{device_id}
    DO->>Agent: apply_config {version, hash, config}
    Note over Nexus: device offline: Push answers offline,<br/>the config waits for the next GetConfig

    Agent->>Agent: render and apply pf, interfaces, services
    Agent->>Agent: arm 60 s rollback guard
    Agent->>DO: config_report {status applied}
    DO->>Nexus: GatewayInternalService.ConfigReport
    Nexus->>DB: status awaiting_confirm

    User->>Portal: Confirm (60 s countdown)
    Portal->>Nexus: ConfirmGatewayConfig
    Nexus->>DB: status applied, devices.config_version = N
    Nexus->>RT: Push(command confirm_config)
    RT->>DO: /push/{device_id}
    DO->>Agent: confirm_config
    Agent->>Agent: disarm guard

    Note over Agent,Nexus: While armed, the agent also polls GetDesiredState,<br/>so a confirm whose push was lost is still seen.
    Note over Agent: no confirm in 60 s: restore the previous state,<br/>config_report {status rolled_back}

Relays and published services

A relay forwards a TLS connection by reading the SNI, without decrypting it. It sends the bytes through the gateway's WireGuard tunnel to the gateway, and the gateway terminates TLS with its own certificate. Details: Relay Architecture, Published Services.

Relay bring-up

sequenceDiagram
    participant TF as tofu apply
    participant Nexus
    participant Relay as relayd
    participant LE as Let's Encrypt
    participant CF as Cloudflare DNS

    TF->>Nexus: GatewayInternalService.RegisterRelayToken<br/>(relay-token.sh, X-Internal-Key)
    TF->>Relay: create instance, cloud-init with the bootstrap token
    Relay->>Nexus: RelayEnrollmentService.Enroll(bootstrap token, addresses, WG key)
    Nexus-->>Relay: relay_id + api_token
    loop every 30 s
        Relay->>Nexus: RelayService.Heartbeat(health, sessions, tunnel_url)
        Nexus-->>Relay: whole set of WG peers, SNI routes, rung-3 tokens
    end
    Relay->>Nexus: RelayService.RequestCertificate(CSR, no names)
    Nexus->>CF: TXT for tunnel.relay-id.suffix
    Nexus->>LE: ACME dns-01 order
    Nexus-->>Relay: chain + names + renew_after
    Note over Relay: tunnel_url is reported only once<br/>the certificate is served

Gateway attach and a visitor request

sequenceDiagram
    participant Agent as edge agent
    participant Nexus
    participant CF as Cloudflare DNS
    participant Relay
    actor Visitor
    participant LAN as LAN host

    Agent->>Nexus: GatewayDeviceService.RequestRelay(WG public key, role)
    Nexus->>Nexus: Assign: keep the current relay if still fit, else the emptiest
    Nexus->>Nexus: record session + tunnel address
    Nexus->>CF: upsert gw-id.suffix and *.gw-id.suffix to the relay IP (TTL 60)
    Nexus-->>Agent: relay address, WG key, port, tunnel_url, tunnel token
    Agent->>Relay: WireGuard: UDP 51820, else UDP 443, else WSS tunnel.relay-id.suffix/t
    Agent->>Nexus: GatewayDeviceService.RequestCertificate(CSR)
    Nexus-->>Agent: certificate for gw-id.suffix + *.gw-id.suffix

    Visitor->>CF: resolve cam1.gw-id.suffix
    CF-->>Visitor: relay IP
    Visitor->>Relay: TLS ClientHello, SNI cam1.gw-id.suffix
    Relay->>Agent: splice to the gateway tunnel address :443 (no decryption)
    Agent->>Agent: terminate TLS, check the access code if required
    Agent->>LAN: proxy to target address:port

OTA release path

Each tag is built once and signed with the shared ed25519 key. The same build is published to every target environment's bucket on the beta channel. A manual job per environment promotes it to stable. Details: OTA Agent Updates.

sequenceDiagram
    participant Tag as git tag (edge/gateway)
    participant CI
    participant Bucket as OTA bucket (per env)
    participant Nexus as nexus (per env)
    participant Agent

    Tag->>CI: build:amd64, build:arm64
    CI->>CI: go build, sign ed25519, verify with the public key
    CI->>Bucket: publish:<env>:<arch>: upload agent/<tag>/edge-agent-openbsd-<arch>
    CI->>Nexus: PublishAgentRelease(channel beta, sha256, signature, object key)
    Note over CI: manual promote:stable:<env>:<arch>
    CI->>Nexus: PublishAgentRelease(same object, channel stable)

    Note over Agent: at startup, on connect, every 30 min,<br/>and on an update_agent nudge
    Agent->>Nexus: GetDesiredState
    Nexus->>Nexus: assigned only, pin or latest on the device channel,<br/>never a channel downgrade
    Nexus-->>Agent: version, presigned URL, sha256, signature
    Agent->>Bucket: GET presigned URL
    Agent->>Agent: verify sha256 + signature, swap binary,<br/>arm watchdog, restart
    Agent->>Nexus: agent_update phases (WS, relayed by the DO)
    Note over Agent: new binary does not confirm: watchdog restores the old one

Login and authorization

Zitadel handles identity only. Every permission is an acl row in authz, keyed on the raw Zitadel sub. The JWT interceptor on nexus authenticates the caller, and each handler checks its own permission. See Authz model.

sequenceDiagram
    actor User
    participant Portal as portal (browser)
    participant Z as Zitadel
    participant Nexus
    participant Authz
    participant DB as Postgres (acl)

    User->>Portal: open portal
    Portal->>Z: authorize (code + PKCE), scope openid profile email<br/>+ project roles
    Z->>User: login (second factor optional)
    Z-->>Portal: code, then access token
    Portal->>Nexus: Connect RPC, Authorization: Bearer token
    Nexus->>Z: JWKS (issuer/oauth/v2/keys), cached
    Nexus->>Nexus: verify signature, issuer, audience = project id, sub into context
    Nexus->>Authz: IsAllowed(sub, target, id, action), Bearer API key
    Authz->>DB: acl join role_actions
    Authz-->>Nexus: allowed / denied
    Nexus-->>Portal: response or PermissionDenied

    Note over Portal,Nexus: Live views: IssueViewerTicket, then WSS ?role=viewer&ticket=…<br/>The console uses IssueShellTicket (action console) and ?role=operator.

Sign-up is closed on the Zitadel instance. An account exists only when an operator invites one (PortalUsersService.Invite, which nexus performs through the Zitadel Management API with its service token) or an admin creates one in the Zitadel console.

radius-ctrl and Monsieur Wifi

Both are deployed by modules/environment only where their image variables are set. Today that is dev only. The stg and prod roots do not pass those variables.

graph LR
    subgraph radius-ctrl
        SPA[radius SPA<br/>Pages, dev.radius.digilan-edge.com]
        API[radius-api-env<br/>.NET container]
        FR[FreeRADIUS VM<br/>UDP 1812/1813]
    end
    subgraph "Monsieur Wifi"
        Go[mrwifi-go-env<br/>Connect API]
        PHP[mrwifi-php-env<br/>legacy Laravel]
    end
    Z[Zitadel]
    A[authz]
    RDB[(shared RDB)]

    SPA -->|Bearer Zitadel token| API
    API -->|JWKS, audience = project id| Z
    API --> RDB
    FR -->|SQL, radius-env db| RDB
    FR -->|CA and CRL sync, provisioning token| API
    Go -->|validates Zitadel tokens| Z
    Go -->|IsAllowed| A
    Go --> RDB
    PHP --> RDB
    PHP -->|OIDC client| Z

The portal dashboard links out to both, through VITE_RADIUS_URL and VITE_MRWIFI_URL. See Radius architecture and Monsieur Wifi.