Skip to main content

Enroll a compute node

This page takes one machine from a cold power-on to a compute node (CN) in the approved state.

Two separate state machines run during enrollment. You check both:

  • Cn.state is per server. Its five values are pending, approved, disabled, awaiting_setup, and setting_up.
  • BootRecord.state is per MAC address. Its four values are discovered, enrolling, enrolled, and decommissioned.

The netboot path drives both. The claim-code path at the end of this page drives only Cn.state.

Before you start​

You need all of the following:

  • A head node that runs tritond, and a triton-booter zone on the machine's admin layer-2 segment. The booter's [ur] config block is optional and absent by default. Without it, the booter does not bind the ur-enrollment AMQP listener, and no bare machine can announce itself.
  • A [ur.setup] block on the booter with install_sh_url, agent_endpoint, platform_stamp, platform_image_sha256, boot_image_stamp, and boot_image_sha256 set. The on-CN installer verifies the running platform image stamp and both artifact digests before it touches a disk.
  • The enrollment platform image staged on the booter.
  • Operator credentials for tritonadm. Setup approval requires the operator action CnSetupApprove. Of the API key scopes, only full reaches it.

tritonadm cn boot covers boot records and the boot policy fields enrollment_pi, auto_enroll, and the enrollment kernel arguments. tritonadm cn setup covers the layout preview and setup approval. Three steps have no CLI verb and use the tritond HTTP API: editing the boot policy's allowlist and admin_network, authorizing a discovered boot record, and setting rescue mode. Load a session into your shell first so that the curl examples below work.

On an operator workstation with a configured tritonadm:

eval "$(tritonadm env)"

That command exports TRITONADM_ENDPOINT and TRITONADM_ACCESS_TOKEN. Every API call below sends the token as Authorization: Bearer.

Every <CN> below is the compute node's hostname, UUID, or a UUID prefix.

Step 1: Set the site boot policy​

The booter serves an enrolling record only when enrollment_pi is set, passes the platform-image stamp charset check, and is staged on the booter. Auto-enrollment also requires admin_network, because tritond never invents an address on a physical layer-2 segment.

On a cluster installed from an answers document, genesis writes the policy from the declared roster and stages the enrollment platform image (tritonadm pi list). On such a cluster, read the policy and skip to step 2 when it is already correct.

On the operator workstation, as the user with the configured tritonadm, read the current policy:

tritonadm cn boot policy show

To change only enrollment_pi, auto_enroll, or the enrollment kernel arguments, use the CLI. It reads the current policy and writes it back whole, so every field that you do not give is preserved. On the operator workstation:

tritonadm cn boot policy set --enrollment-pi <PI_STAMP> --auto-enroll true

To change allowlist or admin_network, edit the whole document through the API. On the operator workstation, save the current policy as JSON:

tritonadm cn boot policy show -j > boot-policy.json
danger

PUT /v1/system/hardware/cn-boot-policy replaces the whole policy. A field that your file leaves out is reset to its default. An omitted declared_nodes or pi_staging becomes an empty list, which wipes the answers-declared roster and the staging requests. An omitted auto_enroll becomes true. Start from the saved policy, not from a fresh file.

Edit boot-policy.json. The fields that enrollment reads look like this:

{
  "auto_enroll": true,
  "allowlist": [{ "kind": "mac_prefix", "value": "0c:c4:7a" }],
  "enrollment_pi": "20260622T191148Z",
  "enrollment_kernel_args": [],
  "admin_network": {
    "ipv4_block": "10.88.88.0/24",
    "provision_start_ipv4": "10.88.88.100",
    "provision_end_ipv4": "10.88.88.199",
    "gateway": "10.88.88.1",
    "resolvers": ["10.88.88.1"]
  }
}

Replace the addresses and the platform-image stamp with your own. Keep the declared_nodes and pi_staging arrays that the saved file carries.

FieldMeaning
auto_enrollDefaults to true. Reachability of the admin DHCP segment is the enrollment trust boundary. Set false to require a per-MAC authorize.
allowlistConstrains auto-enroll. An empty list with auto_enroll on is unconstrained. Entries are {"kind": "mac_prefix", ...} or {"kind": "service_tag", ...}. A mac_prefix value must be in canonical form: lowercase hex in colon-separated octets. A service_tag value must be 1 to 64 ASCII alphanumeric characters. The server rejects any other value when you apply the policy.
enrollment_piPlatform-image stamp served to enrolling nodes. The booter refuses enrollment renders while this is unset.
enrollment_kernel_argsExtra kernel arguments appended to enrollment boots.
admin_networkAddress pool. Authorization reserves the lowest free address and persists the matching networking spec.

Apply it from the same shell:

curl -sS -X PUT \
  -H "Authorization: Bearer $TRITONADM_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  --data @boot-policy.json \
  "$TRITONADM_ENDPOINT/v1/system/hardware/cn-boot-policy"

The response is the stored policy. Read it back with tritonadm cn boot policy show.

Auto-enrollment never authorizes a disk wipe. That approval is always a separate, explicit operator action (step 5).

Step 2: Power the machine on and confirm its boot record​

Connect the machine's admin NIC to the booter's segment. Set the machine to network boot. With answer_only_pxe = true (the shipped default), the booter answers only clients whose DHCP option 60 starts with PXEClient or HTTPClient, or that set option 77 to iPXE.

The booter never sends a DHCP NAK. Every negative outcome is silence. A machine that never gets an address is the expected symptom of an unauthorized MAC, a discovered or decommissioned record, or a record with no assigned address.

Check the boot records. On the operator workstation:

tritonadm cn boot list
tritonadm cn boot show <MAC>

On a cluster installed from an answers document, tritonadm cn boot declared joins the declared roster with live state. The output shows which declared machines have booted, which have enrolled, and which have not announced themselves yet.

Find the record for your machine's MAC and read its state:

  • enrolling means auto-enroll admitted it. The lowest free address from admin_network is reserved on the record. The booter serves the enrollment platform image.
  • discovered means the booter saw the machine but did not admit it. The booter serves it nothing. This happens when auto_enroll is false or the allowlist does not match.

When needed, promote a discovered record by hand. On the operator workstation, in the shell that ran tritonadm env:

curl -sS -X POST \
  -H "Authorization: Bearer $TRITONADM_ACCESS_TOKEN" \
  "$TRITONADM_ENDPOINT/v1/system/hardware/cn-boot/<MAC>/authorize"

Any 12-hex-digit form of <MAC> works, with or without :, -, or . separators. The server canonicalizes it to lowercase colon form. That is the form the record itself carries: 0c:c4:7a:11:22:33.

If the machine has a BMC, a one-time boot override is two Redfish requests. First, send a PATCH of System.Embedded.1 (Dell's path) with {"Boot":{"BootSourceOverrideEnabled":"Once","BootSourceOverrideTarget":"Pxe"}}. Then send a ComputerSystem.Reset with {"ResetType":"ForceRestart"}. Do not include BootSourceOverrideMode. On iDRAC9 it stages a Lifecycle Controller job that can park the host in POST. The booter binary also carries a redfish subcommand that stages a netboot without a console. In the booter zone, as root:

triton-booter redfish --host <BMC_HOST> --user <BMC_USER> \
  --password-file /data/etc/bmc.pw ensure-uefi
triton-booter redfish --host <BMC_HOST> --user <BMC_USER> \
  --password-file /data/etc/bmc.pw set-onetime-boot --target uefi-http
triton-booter redfish --host <BMC_HOST> --user <BMC_USER> \
  --password-file /data/etc/bmc.pw power-cycle

The command reads the password from a file, never from argv. Add --accept-invalid-certs for a self-signed BMC certificate.

Step 3: Confirm the node announced itself​

The enrollment platform image boots with a stock ur-agent that connects to the booter's AMQP listener. The booter validates the machine's identity. It rejects the nil UUID and the all-0xFF value that an unprogrammed SMBIOS reports. Then it announces the machine to tritond. That creates a Cn record in awaiting_setup. The record is inert: it has no claim code, no long-poller, and no credential.

On the operator workstation:

tritonadm cn list

The default columns are SHORTID, HOSTNAME, STATE, ROLE, ADMIN_IP, SEEN, AGE, and TRAITS. Your new machine appears with STATE of awaiting_setup. An empty fleet prints the header row only. Add -l for the full SERVER_UUID and the SETUP_PROGRESS column.

warning

tritonadm cn list --state accepts only pending, approved, and disabled. It cannot express awaiting_setup or setting_up. A node in the netboot enrollment flow is invisible to a filtered list. Run the command unfiltered.

Record the hostname or SHORTID. Every remaining step takes it as <CN>.

Step 4: Preview the disk layout​

Read the announced disk inventory and the layout that the on-CN disklayout heuristic would select with no keyword. On the operator workstation:

tritonadm cn setup plan <CN>

The command prints the announced disks with their sizes, the proposed layout, the supported layouts, and the setup approve command to run. It reads GET /v1/system/hardware/cns/{cn}/setup-layout-proposal, which always returns 200. A machine that announced no usable storage disk prints No proposal: and the reason, so you can read the inventory and see why approval would fail.

The proposal is advisory. The API sees only each disk's name and its whole-GB size from sysinfo. It has no solid-state signal and no view of /etc/mnttab. It classifies every disk as spinning, non-removable, and unmounted. The on-CN disklayout is authoritative and can select differently on a mixed HDD/SSD machine. The installer builds only the explicit choice that you confirm in the next step.

Step 5: Approve the node for setup​

This is the only step that authorizes a disk wipe. It is the only path that can move a node to setting_up.

danger

Approving setup destroys all data on every disk that the confirmed layout consumes. The on-CN installer runs zpool create -f to build the zones pool. The -f flag overrides ZFS's refusal to reuse a disk that already belongs to a pool. The installer does not touch disks listed in disk_exclude. The one protection in the installer is an existing pool named zones. If the org.smartos:server_uuid property on its zones/var dataset does not match the approved server UUID, setup refuses and exits. It does not destroy that pool. No other pool on the machine is protected.

On an operator workstation:

tritonadm cn setup approve <CN> --layout raidz2 --width 8 --spares 1 \
  --compress lz4 --hostname cn-a1-03

The command does not ask for confirmation.

FlagAPI fieldMeaning
--layoutdisk_layoutRequired. One of single, mirror, raidz1, raidz2, raidz3.
--widthdisk_widthFixed stripe width. Omit to let disklayout select.
--sparesdisk_sparesFixed spare count. 0 means exactly zero spares.
--excludedisk_excludeDisk name to keep out of the pool. Repeatable.
--no-cachedisk_cache: falseDisable the L2ARC cache assignment.
--compressdisk_compressOne of off, on, lz4, zstd, gzip, or gzip-1 through gzip-9. Omit for the pool default.
--hostnamehostnameNodename written to the machine's node.config. Omit to keep the announced name.

The command sends these as the body of POST /v1/system/hardware/cns/{cn}/approve-setup. --spares 0 pins a zero-spare pool, which differs from omitting the flag. The server validates the compression value before it mints any credential.

hostname must be an RFC-1123 label: 1 to 63 ASCII letters, digits, or hyphens, and it must not start or end with a hyphen. This is a security gate. The value goes into node.config, which the platform config loader evaluates at every boot.

The API enforces the layout constraints before it mints any credential. It rejects a disk_width below 2 for every keyword. It groups disks into size classes. The largest class is the storage set that the constraints below count. Every smaller class becomes L2ARC cache when disk_cache is on.

KeywordConstraint
singleUses one disk, the smallest in the storage class. disk_width is not applied. A fixed non-zero disk_spares is rejected.
mirrorNeeds at least 2 storage disks. An explicit disk_width runs from 2 up to the storage disk count minus spares.
raidz1Explicit disk_width runs from 3 up to the storage disk count. An omitted width is auto-selected in the range 3 to 6.
raidz2Explicit disk_width runs from 7 up to the storage disk count. An omitted width is auto-selected in the range 7 to 12. With exactly 5 or 6 storage disks, both bounds pin to the disk count.
raidz3Explicit disk_width runs from 9 up to the storage disk count. An omitted width is auto-selected in the range 9 to 20. With exactly 7 or 8 storage disks, both bounds pin to the disk count.

The automatic width search selects the narrowest width in its range that divides the available disks evenly. When disk_spares is omitted, the search starts at the smaller of 2 and the storage disk count divided by 7 for raidz1, 12 for raidz2, or 15 for raidz3, rounded down. It then increases the spare count until one width tiles. With an explicit disk_width and no disk_spares, the spares are the disks left over after whole stripes.

On success, the node is in setting_up and the command prints:

Approved setup for CN <SERVER_UUID> (<CN_HOSTNAME>); layout raidz2 — watch with: tritonadm cn show <SERVER_UUID> (state setting_up until the install reports; failures land in `setup error`)

Approval starts no saga. The operation_id in the -j output is an audit correlation id with no operation record behind it, so tritonadm operation show cannot follow it. Watch the node with tritonadm cn show as step 6 describes.

Common rejections:

  • 409 means the node is not in awaiting_setup. The server checks the state before any layout work.
  • 400 means the layout, hostname, or compression value failed validation. The layout message is disklayout's own wording.
  • 429 means the per-source-IP rate limit is drained. Wait and retry.

Step 6: Watch the install​

Approval stamps a per-attempt setup epoch and a deadline 45 minutes out. The booter takes the node's consume-once credential bundle only when it holds a live AMQP connection to the machine. Then it ships the setup script.

The script runs in this order:

  1. Sets the wall clock. It uses ntp_server when the booter config sets it. Otherwise it uses the booter's /time endpoint. With neither, it fails closed, because a trustworthy clock is a prerequisite for the signature checks and for the ticket keys that the node received.
  2. Verifies that the running platform image reports the configured stamp. Verifies the SHA-256 of both the boot image and the platform image.
  3. Runs disklayout with your confirmed keyword and flags.
  4. Enforces a pool capacity floor of at least twice physical RAM.
  5. Runs zpool create -n with the exact argument list as a dry run. Then runs the destructive zpool create -f -o ashift=12 for the pool named zones.
  6. Stamps org.smartos:server_uuid on zones/var before any fallible step runs. Then creates the remaining datasets, dump, and swap.
  7. Installs a local boot tree. Writes node.config. Installs tritonagent with its credentials. Runs installboot on every pool device.
  8. Exits 113, which is ur's reboot convention. The machine reboots into its newly installed local boot.

Poll the node. On the operator workstation:

tritonadm cn show <CN>

While the install runs, state reads setting_up and setup_progress shows the reported progress.

If the install fails, the node returns to awaiting_setup. The server records the reason on the record and deletes the bound API key. tritonadm cn show prints the reason on its setup_error line. In -j output, the field is last_setup_error. The reporting endpoint accepts at most 512 bytes and rejects a longer report with a 400.

If a node never reports either outcome, the reconciler reverts it when its deadline passes. The recorded reason is cn setup exceeded its deadline without reporting complete/failed; reverted by reconciler. Fix the cause and repeat step 5.

Step 7: Confirm the node is approved​

On success, the node moves to approved. In the same atomic unit, every enrolling boot record among the machine's MACs changes to enrolled. Each record is bound to the server UUID with its assigned platform image. Later boots serve that image instead of the enrollment one. On the operator workstation:

tritonadm cn list

STATE reads approved and ADMIN_IP is populated. tritonadm cn show <CN> prints approved_at, last_seen, bound_api_key_id, the node's traits and labels, and the full sysinfo blob.

The per-CN API key is never shown to you. The server delivers it to tritonagent directly. You see only bound_api_key_id, which lets you correlate audit events.

Step 8: Set the placement role​

Every CN starts as tenant. Change it when the node serves north/south edge traffic. On the operator workstation:

tritonadm cn label set <CN> --role edge

Accepted values are tenant, edge, and both.

Enrolling a machine that already runs SmartOS​

A machine that already runs a SmartOS global zone skips netboot. It uses the claim-code path, which drives only Cn.state.

In the machine's global zone, as root, install the operator CLI from the channel the cluster follows:

curl -fsSL https://tritoncloud.nyc3.digitaloceanspaces.com/install.sh | TRITON_CHANNEL=<CHANNEL> sh

<CHANNEL> is the channel name, for example edge. Without it the script fetches stable.

Point the agent at tritond before you install it. tritonadm install does not write the agent's configuration, and the SMF start method exits without TRITONAGENT_ENDPOINT. In the machine's global zone, as root:

mkdir -p /opt/triton/tritonagent/etc
echo 'TRITONAGENT_ENDPOINT=<TRITOND_URL>' > /opt/triton/tritonagent/etc/agent.env

<TRITOND_URL> is the tritond URL that this machine reaches, in the form http://<TRITOND_IP>:8080. The founder's own /opt/triton/tritonagent/etc/agent.env carries the value that genesis wrote.

Install the agent. In the machine's global zone, as root:

/opt/triton/bin/tritonadm install tritonagent --channel <CHANNEL>

The install enables site/tritonagent. If you installed the agent before you wrote agent.env, the service is in maintenance. Write the file, then run svcadm clear site/tritonagent. To change the endpoint later, edit agent.env and run svcadm restart site/tritonagent.

On an admin network with no route to the internet, set upstream_channel_url in the booter's [ur.setup] block. The booter then mirrors the signature-verified script and serves it at /install.sh on its own HTTP listener. It also serves /channel.json and the tritonadm and tritonagent tarballs that the script needs.

The agent registers anonymously with POST /internal/v1/agent/register. That creates a Cn record in pending with a six-character claim code. The code uses Crockford base32, displays as XXX-XXX, and expires after one hour. The agent prints it to standard error between two === lines. SMF sends that stream to /var/log/tritonagent/agent.out:

REGISTRATION CLAIM CODE: <CODE> (operator: tritonadm cn approve <CODE>)

The agent also mirrors the code to /var/lib/tritonagent/claim-code. The log is authoritative. The file write is best effort.

The agent then waits at most one hour for approval before it exits.

On an operator workstation:

tritonadm cn approve <CODE>

The server accepts both XXX-XXX and XXXXXX, in either case. On success the command prints Approved CN <SERVER_UUID>; bound api key id <KEY_ID>.

The approval endpoint is rate limited per source IP. An unknown, expired, or already-approved code returns 404. The server merges these three cases so that a caller cannot enumerate valid codes. A drained rate-limit bucket returns 429.

Troubleshooting​

SymptomWhere to look
Machine gets no DHCP addressThe booter never NAKs. Check that the boot record is enrolling or enrolled and has an address. Check that answer_only_pxe matches your client. Check that the record is not discovered or decommissioned. If the record is correct and the switch still shows no DISCOVER reaching the booter, check native-VLAN broadcast on the booter host's LACP bond. See Troubleshooting an install.
Machine parks in POST after a Redfish boot overrideOn iDRAC9, a request that carries BootSourceOverrideMode stages a Lifecycle Controller job. Clear the job queue. Reset again with the two-field override.
Node stays setting_up with last_seen older than approved_atThe enrollment image's agent session to the booter dropped. Reset the machine so that it network-boots again. The installer recognizes a built pool and does not wipe it a second time.
Boot record is enrolling but nothing bootsenrollment_pi must be set, valid, and staged on the booter. An enrolled record needs its assigned platform image staged.
Machine boots but no Cn record appearsThe booter rejects the nil and all-0xFF server UUIDs as identities. With auto_enroll set to false, a new server UUID also needs an open enrollment window. Otherwise announce returns 403.
tritonadm cn setup approve returns 409The node is not in awaiting_setup. Read its current state with tritonadm cn show <CN>.
tritonadm cn setup approve returns 400The layout failed validation against the announced disks, or the server rejected the hostname or compression value.
Node reverted to awaiting_setupRead the setup_error line of tritonadm cn show <CN>.
Node needs recovery accessRescue boot mode overrides every lifecycle gate, including decommissioned. Set it with POST /v1/system/hardware/cn-boot/<MAC>/rescue. Clear it with POST /v1/system/hardware/cn-boot/<MAC>/rescue/clear.

Next steps​