How networking works
Every instance gets its address, its neighbors, and its reachability from a virtual private cloud (VPC) that you own. This page explains what the network objects are, how they nest, what the platform decides for you, and what happens to a packet between the guest and the wire.
The objects you own
Networking is built from seven tenant-visible objects. Each one belongs to a project, and every project belongs to a tenant. You never name your tenant when you use the CLI: it is carried in your credentials.
| Object | What it is | Parent |
|---|---|---|
| VPC | An isolated network with its own address plan | Project |
| Subnet | A block carved out of the VPC that instances attach to | VPC |
| Route table | A named set of routes | VPC |
| Route | One destination CIDR and where to send it | Route table |
| Firewall rule | One allow or deny match | VPC |
| NAT gateway | A public source address for outbound traffic | VPC |
| Floating IP | A public address you attach to one instance NIC | Project |
A VPC carries a 24-bit Geneve virtual network identifier (VNI) that the platform
assigns at create time. Tenant VNIs come from the range at or above 4096; values
below that are reserved for platform use. The VNI is what keeps two tenants both
using 10.0.0.0/16 from ever seeing each other's packets. It is stamped into the
outer header of every encapsulated frame and bound before the dataplane engine
sees the packet.
Every object is addressed by UUID. Nothing in the v1 API resolves a name to an object, so you carry the UUIDs of the objects you create.
These objects are created, read, and deleted. There is no update operation on a VPC, subnet, route table, route, firewall rule, NAT gateway, or floating IP. Changing one means deleting it and creating its replacement.
Address plans nest strictly
A VPC declares an IPv4 block, an IPv6 block, or both. At least one is required; a VPC with neither is rejected, because the dataplane has no way to represent it.
Each subnet then carves blocks out of the parent. Three rules are enforced when the subnet is created, not when the first packet moves:
- A subnet block must sit inside the VPC's block of the same family. A
10.0.1.0/24subnet inside a10.0.0.0/16VPC is valid; inside a10.1.0.0/16VPC it is rejected. - A subnet cannot declare a family the VPC does not have. An IPv4-only VPC cannot host an IPv6 subnet.
- Two subnets in the same VPC cannot overlap in either family.
Subnets are where instances attach. An instance names one subnet as its primary when you create it, and the platform allocates that instance a NIC with a MAC address and one address per family the subnet carries. The MAC is locally-administered, generated by the control plane, and unique across the whole installation, because the dataplane resolves each frame by its VNI and MAC pair.
Addresses are assigned, not requested
No DHCP server runs on your network. The control plane picks the address when it creates the NIC, and the compute node hosting the instance synthesizes the DHCP reply from that assignment when the guest asks for one. The reply always carries the address the platform already recorded, and a request asking for a different address is refused.
Three addresses in every IPv4 subnet are never handed out:
- the network address,
- the network address plus one, which is the virtual gateway,
- the broadcast address.
The virtual gateway is the default gateway your guest sees. It is not a device anywhere on the network. It is a set of rules inside the compute node's dataplane that answers ARP for that address and terminates traffic sent to it, which means the gateway has no failure mode independent of the compute node itself.
The synthesized reply carries a default lease of 86400 seconds, an MTU of 1500,
the virtual gateway as both the router and the sole DNS server, and a domain name
of your VPC's name followed by .vpc.local. The lease and additional raw DHCP
options are tunable per VPC and per MAC address through the API; tritonctl has
no command for that surface.
Routing is per subnet, and an absent route drops
Creating a VPC also creates its route table, named main, in the same operation.
Every new subnet starts out associated with that table. The main table cannot be
deleted on its own; it goes away with the VPC.
A route pairs a destination CIDR with a target. Four targets are available to you:
| Target | Behavior |
|---|---|
| Virtual gateway | Hand the packet to the VPC's virtual gateway |
| NAT gateway | Forward to a NAT gateway in the same VPC |
| Blackhole | Drop the packet |
| Reject | Drop the packet |
A fifth target, floating IP, exists in the data model but is reserved for routes the platform installs. Creating one through the API is rejected. Blackhole and Reject are distinct in the data model and describe different intents, but the dataplane compiles both to a drop today; neither returns an ICMP unreachable.
Lookup is longest-prefix match: a /24 route wins over a /0 route for an address
both cover. Two properties of this layer are worth internalizing.
A packet with no matching route is dropped. Outbound routing is default-deny. A newly created route table is empty, so a fresh VPC gives its instances no path off the VPC until you add a route or attach a floating IP.
Traffic inside the VPC does not depend on your routes. The platform adds permits covering the port's own subnet blocks, so instances in a subnet can always reach each other even when the route table is empty. An explicit route you write that covers the same prefix takes precedence over the synthesized permit.
Several other entries appear in the compiled table that never show up in your route listings, because they describe the wire rather than your intent: a route covering the local subnet for cross-compute-node delivery, a host route for the instance metadata address when metadata is enabled, and a default route out the compute node's external link when a floating IP is attached to that NIC. They are recomputed on every apply and never stored.
Route tables other than main can be created and deleted, but nothing associates
a subnet with one. A subnet's route table is fixed at create time to the VPC's
main table, so a second route table holds routes that no packet consults.
Firewall rules are per VPC and stateful
A firewall rule belongs to a VPC, and every NIC in that VPC inherits every rule. There are no security groups to attach and no per-instance rule sets.
A rule matches on direction (inbound or outbound), protocol (any, tcp, udp,
icmp4, or icmp6), a source CIDR, a destination CIDR, and inclusive source and
destination port ranges. An omitted CIDR matches anything. Port ranges are ignored
for protocols other than TCP and UDP. The API also accepts an ICMP type and code
filter, valid only when the protocol is icmp4 or icmp6; tritonctl does not
set it.
Every rule carries a required priority, a 16-bit number. Higher values are
evaluated first. Rules that share a priority are evaluated oldest first.
Allow rules are stateful. The dataplane installs a flow entry so return traffic is recognized without re-evaluating the rule table, and for TCP it runs a connection state machine that drops out-of-state packets.
Outbound traffic is allowed by default. No rule is needed for an instance to initiate connections; what stops outbound traffic is the routing layer, not this one. Inbound is where the default depends on what you have configured.
A VPC with zero firewall rules accepts all inbound traffic. The moment a single rule exists, the inbound default flips to deny and only traffic matching an allow rule gets in. Adding one narrow rule to an empty VPC therefore tightens that VPC completely, not slightly.
Firewall rules are implemented only on the in-memory store. On an installation backed by FoundationDB, creating, reading, or deleting a rule fails with a server error, and every VPC lists zero rules, which puts every VPC in the inbound default-allow posture described above. Treat inbound filtering as unavailable until this lands, and do not place an instance where an open inbound path is unacceptable.
Reaching the outside
Two objects give a VPC a path off itself, and they solve different problems.
A floating IP is a public address owned by your project, not by any instance. You allocate it, then attach it to one NIC. The attachment is one-to-one NAT: outbound packets from that NIC get their source rewritten to the floating IP, and inbound packets to the floating IP get their destination rewritten to the NIC's private address. Attaching also gives that NIC a default route out the compute node's external link, which is what makes the address usable for outbound traffic and not for inbound traffic alone.
Because the address belongs to the project, deleting an instance detaches the floating IP but never releases it, and attaching it to a replacement instance keeps the address stable. Attaching to a second NIC replaces the binding in a single transaction rather than passing through a detached state, and moving one between compute nodes always releases the old binding before the new one is allowed to claim it. Deleting a floating IP requires it to be detached first.
A NAT gateway is a public source address for a VPC's outbound traffic. It reserves one public address when you create it, and it carries no traffic until a route names it as a target. Unlike a floating IP, a NAT gateway does not terminate on the compute node hosting your instance: the control plane places it on an edge cluster and the dataplane forwards matching traffic there.
Both take an address family when you create them, ipv4 or ipv6, and both
default to ipv4.
The path a packet takes
Every instance NIC corresponds to one port in the compute node's kernel dataplane. Outbound packets traverse five layers in a fixed order, and inbound packets traverse the same five in reverse. This diagram shows where a packet can leave the chain and where it can die.
Four things in that chain deserve attention.
ARP and DHCP never reach the network. The gateway layer recognizes them and answers locally, which is why the virtual gateway is always up and never a bottleneck.
The anti-spoof check requires the source MAC and source IP together to be the pair the platform assigned to that NIC. A packet with the right MAC and a forged IP matches neither the IPv4 nor the IPv6 permit and is dropped by the layer's default-deny. This is what stops one instance from impersonating another inside a shared VPC.
Two instances on the same compute node never touch the physical network. The dataplane recognizes that the destination's underlay address is its own, rewrites the inner destination MAC, and delivers the frame directly with no encapsulation and no round trip to the switch.
An in-VPC destination the compute node has not yet resolved to a peer is dropped rather than sent outward. The boundary tested is the whole VPC block, not the local subnet, so an instance with a floating IP talking to a sibling subnet is never diverted out the public link while that resolution is in flight.
The mapping from a peer's private address to the compute node holding it is pushed to each node by the control plane. It is never learned from the outer header of an arriving packet.
What you decide and what the platform decides
| You decide | The platform decides |
|---|---|
| VPC and subnet address plans | The VNI, and which compute node hosts an instance |
| Which subnet an instance attaches to | The NIC's MAC address and IP addresses |
| Route tables and the routes in them | Routes for the local subnet, metadata, and floating IP egress |
| Firewall rules and their priorities | The stateful return path and the empty-VPC default |
| Whether to allocate a floating IP or a NAT gateway | Which public address you get, and where it terminates |
Changes converge, they do not apply instantly
A network change is not a synchronous edit to a running dataplane. The control plane records your intent and stamps a monotonically increasing generation on every affected port. The compute node fetches the recomputed configuration for that port and applies it, and the kernel refuses anything arriving at a generation it has already passed. Concurrent changes to the same port each bump the generation and each recompute the whole port, so the newest one wins rather than the last one to arrive.
A NAT gateway reports this convergence in its own record: alongside the generation the control plane wants, it carries the highest generation any realizer has confirmed applying, and a per-realizer breakdown. A newly created object is a statement of intent; those two generations meeting is the signal that the wire agrees.
Current limits
These are properties of the shipped code, not permanent design decisions.
- Firewall rules exist only on the in-memory store. A FoundationDB-backed installation cannot create, read, or delete them.
- Firewall rules apply to every NIC in the VPC. There is no way to scope a rule to a subset of instances.
- A subnet's route table cannot be changed after the subnet is created, so route
tables other than
mainhold routes that nothing consults. - Network objects cannot be updated; they can only be created and deleted.
- An instance's NICs are fixed at create time. There is no operation to attach or
detach a NIC afterward, and
tritonctl instance createattaches exactly one. - Blackhole and Reject behave identically today. Neither sends an ICMP response.
- The
tritonctlcommands that create a VPC or a floating IP, and the commands that list them, send only a project selector while the server requires both a tenant and a project selector, so those calls return400 MissingScope. - Listing firewall rules, NAT gateways, or route tables requires a VPC, and
listing routes requires a route table, even though the corresponding
tritonctlflags are optional.
See also
- Creating an instance for attaching an instance to a subnet.
- Networking commands for the flags and fields behind every object on this page.
- Your first instance for a guided path through a working setup.