Hardware requirements
This page lists the hardware for an on-premises Triton Cloud install, for a one-host evaluation and for a production cluster. The last column says whether the installer refuses hardware that falls short. A requirement the installer does not check fails later, at enrollment or at the first instance.
Minimums
| Item | Evaluation | Production | Checked by the installer |
|---|---|---|---|
| Hosts | 1 | 3 | Warning only. See Host count. |
| CPU | x86_64 with hardware virtualization enabled in firmware: Intel VT-x with EPT, or AMD-V with NPT | Same | Architecture yes, virtualization no |
| Memory | 32 GiB | Sized to the workload. 128 GiB or more per host. | No |
| Disks | 2 SSDs, mirrored, 350 GiB usable | 2 or more NVMe drives per host, 560 GiB usable on a founder with 128 GiB of memory. See Minimum pool size. | Eligibility only. Enrollment refuses a pool under twice memory on every host but the founder. See Disks. |
| Network interfaces | 1 GbE or faster | 2 × 10 GbE per host | No |
| Jumbo frames | Not needed | MTU 9000 on the underlay VLAN, on every host and switch port | No. See Network. |
| Out-of-band management | Not needed | A BMC on each host that can set a one-time network boot | No |
Host count
The control plane places its members on seats, and the fault tolerance in the answers document sets the number of seats it needs. The installer refuses an answers document whose seats are fewer than its fault tolerance needs. It only warns when there are fewer control-plane-eligible hosts than seats.
| Hosts | Control-plane fault tolerance | FoundationDB redundancy | Object storage |
|---|---|---|---|
| 1 | None | Single | One member, replication factor 1 |
| 3 | One host | Double, 3 coordinators | 3 members, replication factor 3 |
| 5 | Two hosts | Triple, 5 coordinators | 3 or 5 members |
A one-host evaluation uses a host-private admin network. That mode allows exactly one host and no fault tolerance. To grow past one host, install again with an answers document that declares the admin network on a switch.
CPU
Hosts are x86_64 only. The release channel publishes tritonadm and the
platform agents for x86_64-unknown-illumos and no other target.
The installer does not detect hardware virtualization. A host with VT-x or AMD-V disabled in firmware enrolls normally and fails when it boots its first instance. Check the firmware setting on every host before you install.
Live migration requires the target host to have every CPU feature of the source host. Keep the hosts of a live-migration pool on one CPU generation. Cold migration has no CPU check.
Memory
The installer has no memory minimum. Size each host for the control-plane zones it may hold, plus the instances it runs.
| Zone | Memory | Disk quota | Where it runs |
|---|---|---|---|
| FoundationDB | 4 GiB, 1 GiB on one host | 20 GiB, 10 GiB on one host | The founder, then one member per seat. See Host count. |
tritond | 4 GiB, 2 GiB on one host | 20 GiB | The founder |
| ClickHouse | 8 GiB, 2 GiB on one host | 40 GiB, 20 GiB on one host | The founder, when the answers document enables it |
mantad | 4 GiB, 2 GiB on one host | 100 GiB | The founder, then one host at a time up to the member count in the answers document |
| booter | 2 GiB | 40 GiB | The founder, when the answers document configures it |
| dnsedge | 1 GiB | 10 GiB | The founder, when the answers document has a dns block |
| portal | 2 GiB | 25 GiB | A host the operator picks |
| notifyd | 1 GiB | 10 GiB | Hosts the operator picks |
| mantafs master | 4 GiB | 20 GiB | Up to 5 hosts, one per host |
| mantafs storage | 8 GiB | 200 GiB, or its disk_gb | Up to 64 hosts, one per host |
On a multi-host install, the founder first runs a FoundationDB member sized at one eighth of host memory, from 2 to 8 GiB. The installer then replaces it with the 4 GiB member above.
By default tritonagent reserves 80% of each host's physical memory for
instances, in the bhyve memory reservoir (reservoir.percent_default).
Disks
The installer builds the zones pool from the disks it finds. It never uses
a removable (USB) disk, a disk already in a pool, or the boot media. It warns,
and does not refuse, when a vdev mixes SSDs with HDDs, mixes buses, or mixes
disk sizes more than 10% apart.
Host pools are created with ashift=12.
| Layout | Disks per vdev at host enrollment |
|---|---|
mirror | 2 |
raidz1 | 3 to 6 |
raidz2 | 7 to 12 |
raidz3 | 9 to 20 |
MantaBlock block storage refuses a drive with a volatile write cache, or one whose cache state it cannot read. Its drives need power-loss protection.
Minimum pool size
The table gives the minimum usable pool for each host by role and memory.
The founder is the first host, which tritonadm installs. Any other host
that holds a FoundationDB seat or a mantad member is a control-plane host.
Each figure is the disk budget plus 25%, which keeps the pool under 80% full before any instance. On hosts other than the founder, the budget is at least twice memory, the size enrollment checks. When hosts share one hardware build, size that build for the largest column it may fill.
| Host memory | Founder | Other control-plane host | Compute-only host |
|---|---|---|---|
| 32 GiB, one-host evaluation | 350 GiB (380 GB) | Not applicable | Not applicable |
| 128 GiB | 560 GiB (610 GB) | 470 GiB (510 GB) | 320 GiB (350 GB) |
| 256 GiB | 800 GiB (860 GB) | 770 GiB (830 GB) | 640 GiB (690 GB) |
| 384 GiB | 1030 GiB (1110 GB) | 1060 GiB (1140 GB) | 960 GiB (1040 GB) |
| 512 GiB | 1260 GiB (1360 GB) | 1350 GiB (1450 GB) | 1280 GiB (1380 GB) |
| 768 GiB | 1730 GiB (1860 GB) | 1940 GiB (2090 GB) | 1920 GiB (2070 GB) |
The figure in parentheses is the size of each drive in a 2-drive mirror. Drives are sold in decimal gigabytes (GB), and the pool is measured in binary gibibytes (GiB). A mirror of two 200 GB drives holds 186 GiB, which is too small for every row.
Add the disks of the instances each host runs. Add the quota of each operator-placed zone, such as portal, notifyd, or mantafs, that a host holds. A mantafs storage zone alone needs 200 GiB.
What enrollment checks
When a host other than the founder enrolls, enrollment fails with
disk_space if the pool is smaller than twice the host's memory. The check
uses the raw size of the data drives: one drive per mirror vdev, and the data
drives of each raidz vdev. Spares and cache devices don't count, and ZFS
metadata, raidz padding, and the 3% of the pool that ZFS holds back are not
subtracted. The founder gets the same eligibility checks but no size check.
No step checks pool space before it places a control-plane zone. When the pool fills, zone writes fail. On hosts other than the founder, swap writes past the reserved quarter fail too.
Disk budget
Every host holds a swap volume, a dump volume, crash dumps, platform images,
and /opt. The founder and the other hosts size swap and dump differently.
| Item | Founder | Other hosts |
|---|---|---|
| Swap | A volume the size of memory, all of it reserved | A volume the size of memory, one quarter of it reserved. Budget the whole volume, because swap grows into it under memory pressure. |
| Dump | Twice the kernel dump estimate from dumpadm -e. Budget 15% of memory. | Half of memory, all of it reserved |
| Crash dumps | Room in /var/crash for one expanded dump. Budget 25% of memory. | Same |
Platform images, /opt, and cores | 30 GiB, including the 10 GiB quota on cores/global. Each platform release adds about 2 GiB, and old releases stay on disk. | 20 GiB. cores has no quota. |
| Control-plane zones | 230 GiB for the six zones the installer can create, or 200 GiB on one host | 120 GiB on a control-plane host, for a FoundationDB seat and a mantad member. None on a compute-only host. |
The budget includes the ZFS metadata on reserved volumes, about 6% of the swap volume and 3% of the dump volume. Zone disk figures are the quotas in Memory. A new zone starts at the size of its image, about 2 GiB, and grows toward its quota.
The other hosts take their swap reservation from swap_spec under
[ur.setup] in the booter configuration, 0.25x by default. A value of
1x or more, or a fixed Ng size above memory, reserves the whole volume.
Network
The answers document declares three networks, admin, external, and
underlay. See Requirements. Each takes an MTU
from 1500 to 9000. The default is 1500.
Instance traffic between hosts is Geneve over IPv6 on the underlay network. Encapsulation adds 70 bytes, so an instance's interface MTU is the underlay MTU minus 70. A 1500-byte underlay gives instances 1430 bytes. An underlay at 9000 carries instance interfaces at 1500 and above.
Set jumbo frames on the switch ports and the host links before the
aggregate exists. Changing the MTU of a live link aggregation on a host fails
with link busy, and a failure at boot leaves the host without an admin
address.
The switch must forward IPv6 on the underlay VLAN. If a host's admin link is an LACP aggregate, the switch must deliver native-VLAN broadcast to that aggregate, or netbooting hosts get no DHCP lease. See Troubleshooting an install.
Boot firmware
The booter serves network boot to both UEFI and legacy BIOS clients through iPXE. Hosts boot the platform image over the network at every boot and need no boot disk.