How instance disks work
Every instance has a boot disk and can have one data disk. This page explains where those disks live, what a storage class is, what your silo decides for you, and what you can change after the instance exists.
Two kinds of disk
A disk is either local or replicated.
A local disk is a ZFS volume on the compute node that runs the instance. It is as fast as that node's pool, and it exists in one place only. If the node is lost, the disk is lost with it.
A replicated disk is a MantaBlock volume. Each block of the disk is written to two or three compute nodes, so the disk does not live on one node alone. The instance reads and writes the volume through its compute node's MantaBlock engine, whether or not that node holds a copy itself.
Storage classes
You choose a disk's storage by naming its storage class:
| Storage class | Kind | Where the write journals live |
|---|---|---|
local | Local | Not applicable: a ZFS volume on the instance's node |
replicated_standard | Replicated | In files on each node's pool |
replicated_performance | Replicated | On power-loss-protected NVMe |
The class does not set the number of copies. Your silo sets that, as 2 or 3 for
every replicated disk, and the default is 3. A create request that names
replicas= or tier= is refused, because the silo decides both. Name the
class instead.
A disk that asks for replicated storage without a class gets
replicated_performance when the silo offers it, and replicated_standard
otherwise. tritonctl instance create --data-disk 100G is such a request.
What your silo decides
Your silo's provisioning policy lists the storage classes its tenants can use,
and the copy count for replicated disks. A silo that never set the list offers
local only. A class the silo does not offer is refused at create time.
The policy is readable by any member of the silo at
GET /v1/silos/{silo_id}/provisioning-policy, in its storage_classes field.
tritonctl has no verb for it; read it with tritonctl api. Only an operator
can change it.
Replicated disks also need a separate permission. Creating an instance with a
replicated disk is checked against the ReplicatedDiskCreate action as well as
InstanceCreate, so a role can be kept to local disks.
Replicated storage needs enough compute nodes to hold the copies. When too few eligible nodes are available at create time, the create is refused.
Disks belong to their instance
Disks are created with the instance and deleted with it. There is no API to
create a disk on its own, attach a disk to another instance, detach one, delete
one, or snapshot one. You choose the disks with --disk and --data-disk on
tritonctl instance create, as described in
Disk specs.
The boot disk is named boot. The data disk is named data0. Both names are
unique within the instance, not across the project, which is why
tritonctl disk show needs --instance to find a disk by name.
The boot disk holds the image. A replicated boot disk is imported from the
image's source, so it needs a bhyve image that carries a source URL and a
valid SHA-256 checksum. Other images take a local boot disk only.
When you delete an instance, the delete removes its replicated volumes after the instance is torn down on its compute node. Each volume is fenced first, so it is never served again, then removed once every node that holds a copy has freed it.
Resizing
tritonctl disk resize grows a disk. It cannot shrink one, and the new size is
the total size, not an increase. The instance must be running or stopped.
A running guest sees the new capacity after a reboot. A stopped one sees it on its next start. Only the block device grows: the filesystem inside it grows when the guest's cloud-init grows it at boot.
Only local disks can be resized today.
When the platform moves a disk
Two operator actions change where a disk lives without any request from you.
An operator can move a local boot disk onto a MantaBlock volume. The copy runs
while the instance runs. A running instance then stops for the last changes
and starts again from the volume; a stopped one stays stopped. Until that switch, the disk reads as local;
afterward it reads as replicated_standard or replicated_performance and
gains a copies field.
An operator can also migrate an instance to another compute node. When every disk of the instance is replicated, the migration runs live to a node that holds a copy of every disk. No disk data is copied. The guest pauses while each volume's coordinator moves to the new node. Until the new node owns the guest, a handoff that stalls can send the guest back to the old node.
Metering
Each disk is metered by its provisioned size under its storage class, from create to delete. A resize or a change of class updates the record. Usage rolls up hourly into GiB-hours per storage class.
Current limits
| Limit | Detail |
|---|---|
| Data disks per instance | At most one. |
| Local data disks | Not available. A data disk is replicated. |
| Replicated boot disk with a data disk | Not available. |
| Data disk size | A whole number of GiB, from 1 GiB to 16 TiB. |
| Boot disk size | At most 16 TiB, and at least the image's content size. |
| Resize | Grow only, and local disks only. |
| Attach, detach, delete, snapshot | Not available. Disks live and die with their instance. |
The messages for each refused request are listed under Disk specs.