Skip to main content

Enroll a compute node

This page takes one machine from a cold power-on to a compute node (CN) in the approved state.

Two separate state machines run during enrollment. You check both:

  • Cn.state is per server. Its five values are pending, approved, disabled, awaiting_setup, and setting_up.
  • BootRecord.state is per MAC address. Its four values are discovered, enrolling, enrolled, and decommissioned.

The netboot path drives both. The claim-code path at the end of this page drives only Cn.state.

Before you start​

You need all of the following:

  • A head node that runs tritond, and a triton-booter zone on the machine's admin layer-2 segment. The booter's [ur] config block is optional and absent by default. Without it, the booter does not bind the ur-enrollment AMQP listener, and no bare machine can announce itself.
  • A [ur.setup] block on the booter with install_sh_url, agent_endpoint, platform_stamp, platform_image_sha256, boot_image_stamp, and boot_image_sha256 set. The on-CN installer verifies the running platform image stamp and both artifact digests before it touches a disk.
  • The enrollment platform image staged on the booter.
  • Operator credentials for tritonadm. Setup approval requires the operator action CnSetupApprove. No scoped API key can hold that action.

tritonadm cn boot covers boot records and the boot policy fields enrollment_pi, auto_enroll, and the enrollment kernel arguments. tritonadm cn setup covers the layout preview and setup approval. For now, you can edit the allowlist and admin_network fields of the boot policy only through the tritond HTTP API. Load a session into your shell first so that the curl examples below work.

On an operator workstation with a configured tritonadm:

eval "$(tritonadm env)"

That command exports TRITONADM_ENDPOINT and TRITONADM_ACCESS_TOKEN. Every API call below sends the token as Authorization: Bearer.

Step 1: Set the site boot policy​

The booter serves an enrolling record only when enrollment_pi is set, passes the platform-image stamp charset check, and is staged on the booter. Auto-enrollment also requires admin_network, because tritond never invents an address on a physical layer-2 segment.

Write the policy file:

{
  "auto_enroll": true,
  "allowlist": [{ "kind": "mac_prefix", "value": "0c:c4:7a" }],
  "enrollment_pi": "20260622T191148Z",
  "enrollment_kernel_args": [],
  "admin_network": {
    "ipv4_block": "10.88.88.0/24",
    "provision_start_ipv4": "10.88.88.100",
    "provision_end_ipv4": "10.88.88.199",
    "gateway": "10.88.88.1",
    "resolvers": ["10.88.88.1"]
  }
}

Replace the addresses and the platform-image stamp with your own.

FieldMeaning
auto_enrollDefaults to true. Reachability of the admin DHCP segment is the enrollment trust boundary. Set false to require a per-MAC authorize.
allowlistConstrains auto-enroll. An empty list with auto_enroll on is unconstrained. Entries are {"kind": "mac_prefix", ...} or {"kind": "service_tag", ...}. A mac_prefix value must be in canonical form: lowercase hex in colon-separated octets. A service_tag value must be 1 to 64 ASCII alphanumeric characters. The server rejects any other value when you apply the policy.
enrollment_piPlatform-image stamp served to enrolling nodes. The booter refuses enrollment renders while this is unset.
enrollment_kernel_argsExtra kernel arguments appended to enrollment boots.
admin_networkAddress pool. Authorization reserves the lowest free address and persists the matching networking spec.

Apply it:

curl -sS -X PUT \
  -H "Authorization: Bearer $TRITONADM_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  --data @boot-policy.json \
  "$TRITONADM_ENDPOINT/v2/cn-boot-policy"

Read it back. Then change the three CLI-editable fields without a JSON body:

tritonadm cn boot policy show
tritonadm cn boot policy set --enrollment-pi <PI_STAMP> --auto-enroll true

set preserves every field that you do not give. On a cluster installed from an answers document, genesis writes the policy from the declared roster and stages the enrollment platform image (tritonadm pi list). On such a cluster, step 1 is a check, not a write.

Auto-enrollment never authorizes a disk wipe. That approval is always a separate, explicit operator action (step 5).

Step 2: Power the machine on and confirm its boot record​

Connect the machine's admin NIC to the booter's segment. Set the machine to network boot. With answer_only_pxe = true (the shipped default), the booter answers only clients whose DHCP option 60 starts with PXEClient or HTTPClient, or that set option 77 to iPXE.

The booter never sends a DHCP NAK. Every negative outcome is silence. A machine that never gets an address is the expected symptom of an unauthorized MAC, a discovered or decommissioned record, or a record with no assigned address.

Check the boot records:

tritonadm cn boot list
tritonadm cn boot show <MAC>

On a cluster installed from an answers document, tritonadm cn boot declared joins the declared roster with live state. The output shows which declared machines have booted, which have enrolled, and which have not announced themselves yet.

Find the record for your machine's MAC and read its state:

  • enrolling means auto-enroll admitted it. The lowest free address from admin_network is reserved on the record. The booter serves the enrollment platform image.
  • discovered means the booter saw the machine but did not admit it. The booter serves it nothing. This happens when auto_enroll is false or the allowlist does not match.

When needed, promote a discovered record by hand:

curl -sS -X POST \
  -H "Authorization: Bearer $TRITONADM_ACCESS_TOKEN" \
  "$TRITONADM_ENDPOINT/v2/cn-boot/<MAC>/authorize"

Any 12-hex-digit form of <MAC> works, with or without :, -, ., or space separators. The server canonicalizes it to lowercase colon form. That is the form the record itself carries: 0c:c4:7a:11:22:33.

If the machine has a BMC, a one-time boot override is two Redfish requests. First, send a PATCH of System.Embedded.1 (Dell's path) with {"Boot":{"BootSourceOverrideEnabled":"Once","BootSourceOverrideTarget":"Pxe"}}. Then send a ComputerSystem.Reset with {"ResetType":"ForceRestart"}. Do not include BootSourceOverrideMode. On iDRAC9 it stages a Lifecycle Controller job that can park the host in POST. The booter binary also carries a redfish subcommand that stages a netboot without a console. In the booter zone, as root:

triton-booter redfish --host <BMC_HOST> --user <BMC_USER> \
  --password-file /data/etc/bmc.pw ensure-uefi
triton-booter redfish --host <BMC_HOST> --user <BMC_USER> \
  --password-file /data/etc/bmc.pw set-onetime-boot --target uefi-http
triton-booter redfish --host <BMC_HOST> --user <BMC_USER> \
  --password-file /data/etc/bmc.pw power-cycle

The command reads the password from a file, never from argv. Add --accept-invalid-certs for a self-signed BMC certificate.

Step 3: Confirm the node announced itself​

The enrollment platform image boots with a stock ur-agent that connects to the booter's AMQP listener. The booter validates the machine's identity. It rejects the nil UUID and the all-0xFF value that an unprogrammed SMBIOS reports. Then it announces the machine to tritond. That creates a Cn record in awaiting_setup. The record is inert: it has no claim code, no long-poller, and no credential.

On an operator workstation:

tritonadm cn list

The output columns are SERVER_UUID, HOSTNAME, STATE, ROLE, ADMIN_IP, and REGISTERED_AT. Your new machine appears with STATE of awaiting_setup. An empty fleet prints (no compute nodes).

warning

tritonadm cn list --state accepts only pending, approved, and disabled. It cannot express awaiting_setup or setting_up. A node in the netboot enrollment flow is invisible to a filtered list. Run the command unfiltered.

Record the SERVER_UUID. Every remaining step is keyed by it.

Step 4: Preview the disk layout​

Read the announced disk inventory and the layout that the on-CN disklayout heuristic would select with no keyword:

tritonadm cn setup plan <SERVER_UUID>

The same data is available raw from GET /v1/cns/<SERVER_UUID>/setup-layout-proposal.

The response carries disks, an optional proposal, an optional error, and supported_layouts. It always returns 200. This includes a machine that announced no usable storage disk, so you can read the inventory and see why approval would fail.

The proposal is advisory. The API sees only each disk's name and its whole-GB size from sysinfo. It has no solid-state signal and no view of /etc/mnttab. It classifies every disk as spinning, non-removable, and unmounted. The on-CN disklayout is authoritative and can select differently on a mixed HDD/SSD machine. The installer builds only the explicit choice that you confirm in the next step.

Step 5: Approve the node for setup​

This is the only step that authorizes a disk wipe. It is the only path that can move a node to setting_up.

danger

Approving setup destroys all data on every disk that the confirmed layout consumes. The on-CN installer runs zpool create -f to build the zones pool. The -f flag overrides ZFS's refusal to reuse a disk that already belongs to a pool. The installer does not touch disks listed in disk_exclude. The one protection in the installer is an existing pool named zones. If the org.smartos:server_uuid property on its zones/var dataset does not match the approved server UUID, setup refuses and exits. It does not destroy that pool. No other pool on the machine is protected.

Approve with the CLI:

tritonadm cn setup approve <SERVER_UUID> --layout raidz2 --width 8 --spares 1 \
  --compress lz4 --hostname cn-a1-03
FlagAPI fieldMeaning
--layoutdisk_layoutRequired. One of single, mirror, raidz1, raidz2, raidz3.
--widthdisk_widthFixed stripe width. Omit to let disklayout select.
--sparesdisk_sparesFixed spare count. 0 means exactly zero spares.
--excludedisk_excludeDisk name to keep out of the pool. Repeatable.
--no-cachedisk_cache: falseDisable the L2ARC cache assignment.
--compressdisk_compresslz4, zstd, or off. The API also accepts the gzip levels.
--hostnamehostnameNodename written to the machine's node.config.

The equivalent request body, for the API directly:

{
  "disk_layout": "raidz2",
  "disk_width": 8,
  "disk_spares": 1,
  "disk_exclude": [],
  "disk_cache": true,
  "disk_compress": "lz4",
  "hostname": "cn-a1-03"
}
FieldRequiredMeaning
disk_layoutYesOne of single, mirror, raidz1, raidz2, raidz3. No default. A body that omits it returns 400, so setup never runs on the on-CN heuristic alone.
disk_widthNoFixed stripe width. Omit to let disklayout select.
disk_sparesNoFixed spare count. 0 means a fixed zero-spare pool, which differs from omitting the field.
disk_excludeNoDisk names kept out of the pool. Defaults to an empty list.
disk_cacheNoDefaults to true. false disables the L2ARC cache device.
disk_compressNoOne of off, on, lz4, zstd, gzip, or gzip-1 through gzip-9. Omit for the pool default.
hostnameNoOperator-chosen nodename written to the machine's node.config. Omit to keep the announced name.

The body rejects unknown fields. A misspelled key such as disk_excludes returns 400. The API does not drop your intent and consume a disk that you meant to keep.

hostname must be an RFC-1123 label: 1 to 63 ASCII letters, digits, or hyphens, and it must not start or end with a hyphen. This is a security gate. The value goes into node.config, which the platform config loader evaluates at every boot.

The API enforces the layout constraints before it mints any credential. It rejects a disk_width below 2 for every keyword. It groups disks into size classes. The largest class is the storage set that the constraints below count. Every smaller class becomes L2ARC cache when disk_cache is on.

KeywordConstraint
singleUses one disk, the smallest in the storage class. disk_width is not applied. A fixed non-zero disk_spares is rejected.
mirrorNeeds at least 2 storage disks. An explicit disk_width runs from 2 up to the storage disk count minus spares.
raidz1Explicit disk_width runs from 3 up to the storage disk count. An omitted width is auto-selected in the range 3 to 6.
raidz2Explicit disk_width runs from 7 up to the storage disk count. An omitted width is auto-selected in the range 7 to 12. With exactly 5 or 6 storage disks, both bounds pin to the disk count.
raidz3Explicit disk_width runs from 9 up to the storage disk count. An omitted width is auto-selected in the range 9 to 20. With exactly 7 or 8 storage disks, both bounds pin to the disk count.

The automatic width search selects the narrowest width in its range that divides the available disks evenly. When disk_spares is omitted, it increases the spare count until one width tiles.

Submit it:

curl -sS -X POST \
  -H "Authorization: Bearer $TRITONADM_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  --data @approve-setup.json \
  "$TRITONADM_ENDPOINT/v1/cns/<SERVER_UUID>/approve-setup"

The response carries the updated CN, now in setting_up, and an operation_id. Approval starts the cn-setup saga. You can follow it step by step with tritonadm operation show <OPERATION> --json. The durable Cn.state column is still the state machine that the rest of this page watches.

Common rejections:

  • 409 means the node is not in awaiting_setup. The server checks the state before any layout work.
  • 400 means the layout, hostname, or compression value failed validation. The layout message is disklayout's own wording.

Step 6: Watch the install​

Approval stamps a per-attempt setup epoch and a deadline 45 minutes out. The booter takes the node's consume-once credential bundle only when it holds a live AMQP connection to the machine. Then it ships the setup script.

The script runs in this order:

  1. Sets the wall clock. It uses ntp_server when the booter config sets it. Otherwise it uses the booter's /time endpoint. With neither, it fails closed, because a trustworthy clock is a prerequisite for the signature checks and for the ticket keys that the node received.
  2. Verifies that the running platform image reports the configured stamp. Verifies the SHA-256 of both the boot image and the platform image.
  3. Runs disklayout with your confirmed keyword and flags.
  4. Enforces a pool capacity floor of at least twice physical RAM.
  5. Runs zpool create -n with the exact argument list as a dry run. Then runs the destructive zpool create -f -o ashift=12 for the pool named zones.
  6. Stamps org.smartos:server_uuid on zones/var before any fallible step runs. Then creates the remaining datasets, dump, and swap.
  7. Installs a local boot tree. Writes node.config. Installs tritonagent with its credentials. Runs installboot on every pool device.
  8. Exits 113, which is ur's reboot convention. The machine reboots into its newly installed local boot.

Poll the node:

tritonadm cn show <SERVER_UUID>

While the install runs, state reads setting_up.

If the install fails, the node returns to awaiting_setup. The server records the reason on the record and deletes the bound API key. The plain output of tritonadm cn show does not print that reason. Read it with:

tritonadm cn show <SERVER_UUID> --json

The reason is in last_setup_error. The reporting endpoint accepts at most 512 bytes. It rejects a longer report with a 400.

If a node never reports either outcome, the reconciler reverts it when its deadline passes. The recorded reason is cn setup exceeded its deadline without reporting complete/failed; reverted by reconciler. Fix the cause and repeat step 5.

Step 7: Confirm the node is approved​

On success, the node moves to approved. In the same atomic unit, every enrolling boot record among the machine's MACs changes to enrolled. Each record is bound to the server UUID with its assigned platform image. Later boots serve that image instead of the enrollment one.

tritonadm cn list

STATE reads approved and ADMIN_IP is populated. tritonadm cn show against the server UUID prints approved_at, last_seen, bound_api_key_id, and the full sysinfo blob.

The per-CN API key is never shown to you. The server delivers it to tritonagent directly. You see only bound_api_key_id, which lets you correlate audit events.

Step 8: Set the placement role​

Every CN starts as tenant. Change it when the node serves north/south edge traffic:

tritonadm cn label set <SERVER_UUID> --role edge

Accepted values are tenant, edge, and both.

Enrolling a machine that already runs SmartOS​

A machine that already runs a SmartOS global zone skips netboot. It uses the claim-code path, which drives only Cn.state.

In the machine's global zone, as root, install the operator CLI and then the agent:

curl -fsSL https://tritoncloud.nyc3.digitaloceanspaces.com/install.sh | sh
/opt/triton/bin/tritonadm install tritonagent

On an admin network with no route to the internet, set upstream_channel_url in the booter's [ur.setup] block. The booter then mirrors the signature-verified script and serves it at /install.sh on its own HTTP listener. It also serves /channel.json and the tritonadm and tritonagent tarballs that the script needs.

The agent registers anonymously. That creates a Cn record in pending with a six-character claim code. The code uses Crockford base32, displays as XXX-XXX, and expires after one hour. The agent prints it to the console and mirrors it to /var/lib/tritonagent/claim-code. The console log is authoritative. The file write is best effort.

The agent then waits at most one hour for approval before it exits.

On an operator workstation:

tritonadm cn approve <CODE>

The command accepts both XXX-XXX and XXXXXX. On success it prints Approved CN <uuid>; bound api key id <id>.

The console banner that the agent prints names the CLI by its former name and a --code flag. The current command is tritonadm cn approve with the code as a positional argument, as shown above.

The approval endpoint is rate limited per source IP. An unknown, expired, or already-approved code returns 404. The server merges these three cases so that a caller cannot enumerate valid codes. A drained rate-limit bucket returns 429.

Troubleshooting​

SymptomWhere to look
Machine gets no DHCP addressThe booter never NAKs. Check that the boot record is enrolling or enrolled and has an address. Check that answer_only_pxe matches your client. Check that the record is not discovered or decommissioned. If the record is correct and the switch still shows no DISCOVER reaching the booter, check native-VLAN broadcast on the booter host's LACP bond. See Troubleshooting an install.
Machine parks in POST after a Redfish boot overrideOn iDRAC9, a request that carries BootSourceOverrideMode stages a Lifecycle Controller job. Clear the job queue. Reset again with the two-field override.
Node stays setting_up with last_seen older than approved_atThe enrollment image's agent session to the booter dropped. Reset the machine so that it network-boots again. The installer recognizes a built pool and does not wipe it a second time.
Boot record is enrolling but nothing bootsenrollment_pi must be set, valid, and staged on the booter. An enrolled record needs its assigned platform image staged.
Machine boots but no Cn record appearsThe booter rejects the nil and all-0xFF server UUIDs as identities. With auto_enroll set to false, a new server UUID also needs an open enrollment window. Otherwise announce returns 403.
approve-setup returns 409The node is not in awaiting_setup. Read its current state with tritonadm cn show.
approve-setup returns 400The layout failed validation against the announced disks, or the server rejected the hostname or compression value.
Node reverted to awaiting_setupRead last_setup_error from tritonadm cn show <SERVER_UUID> --json.
Node needs recovery accessRescue boot mode overrides every lifecycle gate, including decommissioned. Set it with POST /v2/cn-boot/<MAC>/rescue. Clear it with POST /v2/cn-boot/<MAC>/rescue/clear.

Next steps​