Skip to main content

Enroll compute nodes and converge

After genesis, tritonadm setup converge brings up the rest of the control plane, including the booter. Every machine other than the founder then network-boots from that booter. You install nothing by hand. This page runs the first converge, takes the declared machines from powered off to approved, and lets the control plane grow onto them.

Before you start, confirm all of the following:

  • Genesis completed with == bootstrap complete, and tritonadm cn list shows the founder as approved.
  • The answers document declares a booter (booter.enabled). The admin NIC of each machine is on the admin layer-2 segment.
  • You can reach the BMC or the console of each machine, to set it to network boot once.
  • Each machine boots UEFI with Secure Boot off. The enrollment platform image is not signed for Secure Boot.

Step 1: Converge​

On the founder, as root:

tritonadm setup converge

Without --answers, converge reads the committed intent from the control plane. Without --channel, it reads the channel genesis recorded and prints channel <URL> (cluster config channel.upstream_url). It wakes the intent reconciler and prints each step as it changes: ClickHouse, DNS and certificate re-proof, object storage, the in-cluster channel, the booter, the boot policy, and the platform image staging. Every step is re-entrant, so you can run the command more than once. It is also the forward path for a genesis that stopped after the pivot.

At this point the control-plane-seats step reports below policy, because no other machine is enrolled yet. That counts as converged. The run ends with == converge complete: intent revision <N> at fixed point ==.

If the command stops with resolve the release channel for converge, see Troubleshooting an install.

Step 2: Check that the booter serves​

On the founder, as root:

tritonadm infra zone list --kind booter
tritonadm pi list
tritonadm cn boot policy show

The booter zone is active. The enrollment platform image is on the staging list. The boot policy names it as enrollment_pi. Convergence stages the platform image that the answers document names. If pi list is empty, see Troubleshooting an install.

Auto-enrollment is on by default. Admission is decided by the boot policy allowlist of MAC prefixes and service tags. Convergence fills the allowlist with the admin NIC MACs of the nodes the answers document declares. A machine that matches an entry is admitted and moves to a cn_range address. A machine that matches no entry is recorded as discovered, and the booter serves it nothing. This is what keeps two clusters safe on one admin segment.

warning

With auto-enrollment on and an empty allowlist, the boot policy admits every machine that network-boots on the admin segment. The allowlist stays empty when no declared node has a concrete admin MAC, because a node matched by selector contributes none. Turn auto-enrollment off to close this, then authorize each machine by hand.

On the founder, in the global zone, as root:

tritonadm cn boot policy set --auto-enroll false

Step 3: Network-boot each machine​

Set the machine to boot from the network once. Then reset it. Setup writes a local boot tree, so one network boot is sufficient.

From the BMC, the Redfish one-time boot override is two requests:

PATCH https://<BMC>/redfish/v1/Systems/System.Embedded.1
  {"Boot":{"BootSourceOverrideEnabled":"Once","BootSourceOverrideTarget":"Pxe"}}
POST  https://<BMC>/redfish/v1/Systems/System.Embedded.1/Actions/ComputerSystem.Reset
  {"ResetType":"ForceRestart"}
warning

Do not include BootSourceOverrideMode in the boot-override request. On iDRAC9 that field stages a BIOS configuration job through the Lifecycle Controller. When that controller is busy, the job stays in Scheduling, and the host stops in POST with the override unused. The two-field form above creates no job.

The System.Embedded.1 path is Dell's. Other BMCs name the system resource differently. The Boot object is the same.

Watch the roster fill. On the founder, in the global zone, as root:

tritonadm cn boot declared

The command joins the declared nodes with live state. It shows which machines netbooted, which enrolled, and which have not appeared. A machine that booted takes a join_dhcp_range lease and announces its identity. It then appears in tritonadm cn list as awaiting_setup.

A machine that never takes a lease is almost never a booter problem. See Troubleshooting an install.

Step 4: Approve each machine for setup​

Setup is the one step that erases disks, so it is a separate operator action for each machine. First read the disk inventory and the proposed layout. On the founder, in the global zone, as root:

tritonadm cn setup plan <CN>

<CN> is the hostname, the UUID, or a UUID prefix. The proposal is advisory. The layout you pass to approve is the layout that setup builds.

Approve setup. On the founder, in the global zone, as root:

danger

Setup approval destroys every disk that the confirmed layout uses. The machine builds its zones pool with zpool create -f. The -f flag overrides the ZFS refusal to reuse a disk that belongs to another pool. There is no undo.

tritonadm cn setup approve <CN> --layout mirror --hostname <HOSTNAME>

--layout is required. It accepts single, mirror, raidz1, raidz2, or raidz3. Use the value that the answers document declares in storage.layout for that node. --width, --spares, --exclude, --no-cache, and --compress refine the pool. Omit them to let the layout heuristic select the values. --hostname sets the node name that the machine keeps. It must be an RFC 1123 label.

The command starts the setup saga. Watch it. On the founder, in the global zone, as root:

tritonadm cn show <CN>
tritonadm operation list

state reads setting_up while the install runs, then approved. The machine reboots into its local boot tree, and its agent registers. A machine that returns to awaiting_setup records the reason in last_setup_error. Read it with tritonadm cn show <CN> --json.

Repeat this step for every declared machine that will hold a control-plane seat. The other machines can enroll later.

Step 5: Grow the control plane​

The FoundationDB reconciler in tritond adds one seat at a time, on an approved compute node that holds no seat, until the declared seat count is reached. It acts only while a standing approval window for grow is open. Open one on the founder, as root:

tritonadm fdb approvals standing open --operation grow

The window lasts 3600 seconds unless you pass --expires-in. Watch the growth:

tritonadm setup converge

The control-plane-seats step names each member it grows and the compute node it grows it on. This is the step that turns a single-seat FoundationDB into a three-seat cluster.

Step 6: Check the result​

On the founder, in the global zone, as root:

tritonadm fdb status
tritonadm intent show
tritonadm operation list
tritonadm cn list

fdb status reports the declared seat count, quorum, and fault tolerance. operation list is a summary. For anything that looks wrong, read the saga with tritonadm operation show <OPERATION> --json. Walk steps to the first step that is not succeeded.

Next steps​