Skip to main content

Enroll compute nodes and converge

After genesis, every machine other than the founder network-boots from the cluster's own booter. You install nothing by hand. This page takes the declared machines from powered off to approved. Then it converges the committed intent onto them.

Before you start, confirm all of the following:

  • Genesis completed, and tritonadm cn list shows the founder as approved.
  • The answers document declares a booter (booter.enabled). The admin NIC of each machine is on the admin layer-2 segment.
  • You can reach the BMC or the console of each machine, to set it to network boot once.
  • Each machine boots UEFI with Secure Boot off. The enrollment platform image is not signed for Secure Boot.

Step 1: Check that the booter serves​

On the founder, as root:

tritonadm infra zone list --kind booter
tritonadm pi list
tritonadm cn boot policy show

The booter zone is online. The enrollment platform image is on the staging list. The boot policy names it as enrollment_pi. Genesis stages the platform image that the answers document names. If pi list is empty, see Troubleshooting an install.

Auto-enrollment is on by default. The declared node roster decides admission. A machine whose identity matches a declaration is admitted and moves to a cn_range address. A machine that matches no declaration is recorded as discovered, and the booter serves it nothing. This is what keeps two clusters safe on one admin segment. For the strictest posture, turn automatic admission off and authorize each machine by hand:

tritonadm cn boot policy set --auto-enroll false

Step 2: Network-boot each machine​

Set the machine to boot from the network once. Then reset it. Setup writes a local boot tree, so one network boot is sufficient.

From the BMC, the Redfish one-time boot override is two requests:

PATCH https://<BMC>/redfish/v1/Systems/System.Embedded.1
  {"Boot":{"BootSourceOverrideEnabled":"Once","BootSourceOverrideTarget":"Pxe"}}
POST  https://<BMC>/redfish/v1/Systems/System.Embedded.1/Actions/ComputerSystem.Reset
  {"ResetType":"ForceRestart"}
warning

Do not include BootSourceOverrideMode in the boot-override request. On iDRAC9 that field stages a BIOS configuration job through the Lifecycle Controller. When that controller is busy, the job stays in Scheduling, and the host stops in POST with the override unused. The two-field form above creates no job.

The System.Embedded.1 path is Dell's. Other BMCs name the system resource differently. The Boot object is the same.

Watch the roster fill. On the founder:

tritonadm cn boot declared

The command joins the declared nodes with live state. It shows which machines netbooted, which enrolled, and which have not appeared. A machine that booted takes a join_dhcp_range lease and announces its identity. It then appears in tritonadm cn list as awaiting_setup.

A machine that never takes a lease is almost never a booter problem. See Troubleshooting an install.

Step 3: Approve each machine for setup​

Setup is the one step that erases disks, so it is a separate operator action for each machine. First read the disk inventory and the proposed layout:

tritonadm cn setup plan <CN>

<CN> is the hostname, the UUID, or a UUID prefix. The proposal is advisory. The layout you pass to approve is the layout that setup builds.

danger

Setup approval destroys every disk that the confirmed layout uses. The machine builds its zones pool with zpool create -f. The -f flag overrides the ZFS refusal to reuse a disk that belongs to another pool. There is no undo.

tritonadm cn setup approve <CN> --layout mirror --hostname <HOSTNAME>

--layout is required. It accepts single, mirror, raidz1, raidz2, or raidz3. Use the value that the answers document declares in storage.layout for that node. --width, --spares, --exclude, --no-cache, and --compress refine the pool. Omit them to let the layout heuristic select the values. --hostname sets the node name that the machine keeps. It must be an RFC 1123 label.

The command starts the setup saga. Watch it:

tritonadm cn show <CN>
tritonadm operation list

state reads setting_up while the install runs, then approved. The machine reboots into its local boot tree, and its agent registers. A machine that returns to awaiting_setup records the reason in last_setup_error. Read it with tritonadm cn show <CN> --json.

Repeat this step for every declared machine that will hold a control-plane seat. Do this before you converge. The other machines can enroll later.

Step 4: Converge​

When the machines that will hold seats are approved, run this on the founder:

tritonadm setup converge

Convergence drives the observed state toward the committed intent: ClickHouse, DNS and certificate re-proof, object storage, the booter, and the boot policy. Every step is re-entrant, so you can run the command more than once. It is also the forward path for a genesis that stopped after the pivot. This step turns a single-seat FoundationDB into a three-seat cluster.

converge needs the release channel, and genesis does not record it. If the command stops with resolve the release channel for converge, set the upstream once:

tritonadm config set channel.upstream_url https://tritoncloud.nyc3.digitaloceanspaces.com/channels/edge.json

The same setting lets tritonadm channel refresh mirror the upstream into the cluster later.

Step 5: Check the result​

tritonadm fdb status
tritonadm intent show
tritonadm operation list
tritonadm cn list

fdb status reports the declared seat count, quorum, and fault tolerance. operation list is a summary. For anything that looks wrong, read the saga with tritonadm operation show <OPERATION> --json. Walk steps to the first step that is not succeeded.

Next steps​