Data plane

Lace runs overlay networks over a routed IPv6 underlay, using SRv6 to carry traffic between nodes. Each node acts as a router between the underlay and the overlays it hosts — and that router is implemented entirely in eBPF. This page covers the underlay and encapsulation, the interfaces Lace builds on a node, where the eBPF programs hook in, and how a packet moves through them.

Underlay

The underlay is the physical network between nodes. Lace requires only L3 (routed) IPv6 reachability between them — no special fabric. Each node owns an SRv6 locator (an IPv6 prefix); every encapsulated packet is addressed into the locator of the node hosting its destination, so the underlay only has to route each node’s locator to that node.

Today Lace establishes that routing itself rather than relying on the fabric (see UnderlayEndpoint). It only does so for nodes whose underlay address is directly reachable on-link, so nodes on the same L2 segment work out of the box. For a node reachable only through a router, Lace installs no route and traffic falls back to the node’s default (or a custom) route — that path works only if you make the underlay aware of each node’s locator prefix and manage the routing yourself.

SRv6 encapsulation

SRv6 (Segment Routing over IPv6) expresses a path as a list of IPv6 addresses called segments, each one an instruction to the node that owns it. Lace uses it as a plain encapsulation with a single segment: the original packet is wrapped in an outer IPv6 header addressed to a SID (segment identifier) on the destination node. Because there is only one segment, Lace omits the routing extension header that would otherwise carry the segment list (SRv6’s reduced encapsulation): the SID rides in the outer destination address alone, so the encapsulation adds only an outer IPv6 header and leaves more of the underlay MTU for pods.

Pod addresses are meaningful only inside their network, so cross-node traffic cannot ride the underlay as-is. What gets wrapped depends on the network type:

L3 networks encapsulate the pod’s IP packet.
L2 networks encapsulate the whole Ethernet frame, so MAC-level semantics are preserved across nodes.

A SID is an IPv6 address built from three parts:

the destination node’s locator prefix — routes the encapsulated packet to that node across the underlay,
a function — what the node should do on arrival, namely decapsulate,
the network id — which overlay the inner packet belongs to, so the far node delivers it into the right network.

The destination node matches the outer address against its own locator, decapsulates, and hands the inner packet to the local pod. Traffic between two pods on the same node stays local and is never encapsulated.

UnderlayEndpoint

UnderlayEndpoint is how a node makes itself reachable on the underlay; one exists per node.

Lifecycle — created and owned by the controller, one per Node and bound to it. When a node joins, the controller allocates its locator prefix (unique cluster-wide and fixed for the node’s lifetime) and records the resource; it is garbage-collected with the Node.
Contents — the node’s locator prefix, its IPv6 underlay address (the next hop for traffic encapsulated to that node), and the node’s internal addresses that participate in NodePort.
Use — every node watches all UnderlayEndpoints and installs a host route to each other node’s locator prefix via that node’s underlay address, but only when that address is directly reachable on-link. That route is what carries an encapsulated packet across the underlay to the node owning the destination SID; for nodes reached through a router it is left to the fabric (see Underlay). The NodePort addresses let a NodePort answer on every node (see Services).

Node interfaces

Each network is realised on a node as a small set of Linux interfaces:

VRF — one per network. eBPF forwards all overlay traffic itself, so the kernel’s router never carries it in the pod-to-pod or pod-to-host direction. The VRF handles isolation the other way: a network’s interfaces add connected routes — for L3, the bridge’s link-local and its gateway subnet — and the VRF keeps them in its own table instead of the host’s main table, so the host stack can’t route into the overlay on its own. It keeps each network’s routes from leaking into the main table, between overlays, or between an overlay and the host. Both L3 and L2 networks get one.
Bridge — one per L3 network, enslaved to that network’s VRF. An L3 network’s prefix is split into a host-local subnet per node; the bridge is the L2 segment for that subnet, forwarding between the pods on this node. It also carries the subnet’s gateway IP and a stable gateway MAC, which pods use as their default gateway. L2 networks have no bridge — their L2 domain spans nodes and is handled in eBPF.
Pod veths — one pair per pod. The container end lives in the pod’s namespace; the host end is attached to the network’s bridge (L3) or directly to its VRF (L2). The pod-side MTU is reduced from the underlay MTU to leave room for the SRv6 encapsulation overhead.
lc-host / lc-host-peer — a single veth pair for traffic that originates on the host itself (node-local health probes, a host process reaching a pod or ClusterIP). Host routes for pod and service prefixes point out lc-host; the traffic arrives on lc-host-peer’s ingress, the one place the in-kernel redirect to a pod is legal.
Underlay interfaces — the node’s uplinks, identified as the interfaces carrying a default route. The primary uplink carries SRv6 traffic to and from other nodes; eBPF attaches to each managed underlay interface so encapsulated overlay traffic and other north-south ingress are all processed on entry.

Where eBPF attaches

Every Lace program attaches at the TC ingress hook of its interface — never egress. This is deliberate: the in-kernel redirect that hands a packet straight into a pod’s veth is only permitted from an ingress hook, so the whole pipeline is built around ingress.

That single rule places the programs as follows:

Pod host veth, ingress — a pod’s egress. Traffic leaving a pod is processed here, with separate programs for L3 and L2 networks.
Underlay uplink, ingress — traffic arriving from other nodes and north-south. SRv6 is decapsulated here before the packet continues through the pipeline.
lc-host-peer, ingress — host-originated traffic, redirected to the destination pod.

Connection tracking

Lace tracks every managed flow. The first packet of a new flow runs the full pipeline, and the decision it reaches is recorded as connection-tracking entries that match the flow in both directions:

the forward direction — so the rest of the flow’s packets take the fast path and skip the pipeline,
the return direction — so replies are recognised and follow the same path in reverse.

Because return traffic is matched against the entry created on the forward path, the data plane needs neither routing symmetry nor a matching reverse policy for it.

The pipeline

The entry programs hand off to a chain of small, self-contained stages, each tail-called from the last. A new flow runs through all of them; an established one is short-circuited at the fast path:

Network classification — determine which overlay the packet belongs to and resolve its source and destination context.
Fast path — look the flow up in connection tracking; a hit forwards it by the recorded decision and skips everything below.
Routing — resolve the destination and the topology between networks to decide where the packet goes.
Service routing — translate a service VIP to one of its backing endpoints.
Policy — enforce reachability between the source and destination segments.
Delivery — carry out the decision: encap in SRv6 to the node hosting a remote pod, local redirect to another pod on this node, or hand to the host stack (egress, optionally masqueraded). The flow is committed to connection tracking on the way out.

Every final accept or deny decision on a new flow is emitted as a structured trace event. Reading those traces and inspecting connection tracking is covered in Node state.