Skip to main content

Store objects with an S3 client

This guide configures the AWS CLI to use Triton Cloud object storage, then creates a bucket, uploads and downloads an object, and shares it with a presigned URL. Any S3 client that signs with Signature Version 4 and supports path-style addressing works the same way.

Before you start, you need:

  • An access key ID and secret access key for a storage user in your tenant. The operator of your cloud issues these. A tenant cannot create them. See How object storage works.
  • The S3 endpoint URL of your cloud, also from your operator. It is https://s3.<DOMAIN>:7443 when the cloud issues certificates, and http://<IP>:7443 on a private address when it does not.
  • The region your operator configured for the object store. It is us-east-1 unless they set another.
  • The AWS CLI installed on your workstation.

Step 1: Add a profile for Triton Cloud​

On your workstation, add a profile to ~/.aws/credentials:

~/.aws/credentials
[triton]
aws_access_key_id = <ACCESS_KEY_ID>
aws_secret_access_key = <SECRET_ACCESS_KEY>

Then add the matching profile to ~/.aws/config:

~/.aws/config
[profile triton]
region = <REGION>
request_checksum_calculation = when_required
s3 =
    addressing_style = path

addressing_style = path is required. The object store reads the bucket from the URL path and does not support virtual-hosted-style requests.

request_checksum_calculation = when_required stops recent AWS CLI versions from sending uploads with the STREAMING-UNSIGNED-PAYLOAD-TRAILER encoding, which the object store refuses unless the operator turned it on.

Set the profile and endpoint for the rest of this shell session:

export AWS_PROFILE=triton
export S3_ENDPOINT=https://s3.<DOMAIN>:7443

Step 2: Check the credentials​

List the buckets you can see:

aws s3 ls --endpoint-url "$S3_ENDPOINT"

An empty result with no error means the key and endpoint work. See If a request fails for the errors at this step.

Step 3: Create a bucket​

aws s3 mb s3://<BUCKET> --endpoint-url "$S3_ENDPOINT"

<BUCKET> is 3 to 63 characters of lowercase letters, digits, -, and ., starting and ending with a letter or digit. The full rules are in the S3 API reference.

A scoped access key can create a bucket only if its scope grants full on that bucket name.

Step 4: Upload an object​

aws s3 cp ./report.csv s3://<BUCKET>/reports/report.csv --endpoint-url "$S3_ENDPOINT"

A single PUT accepts up to 5 GiB by default. The AWS CLI switches to a multipart upload for large files on its own. The object store requires every part except the last to be at least 5 MiB, and allows at most 10,000 parts.

To upload a directory:

aws s3 sync ./reports s3://<BUCKET>/reports --endpoint-url "$S3_ENDPOINT"

Step 5: List and download​

aws s3 ls s3://<BUCKET>/reports/ --endpoint-url "$S3_ENDPOINT"
aws s3 cp s3://<BUCKET>/reports/report.csv ./report-copy.csv --endpoint-url "$S3_ENDPOINT"

Step 6: Share an object with a presigned URL​

A presigned URL lets someone without a key download one object until the URL expires:

aws s3 presign s3://<BUCKET>/reports/report.csv --expires-in 3600 --endpoint-url "$S3_ENDPOINT"

The command prints the URL. --expires-in is in seconds, from 1 to 604800 (7 days). A URL with a longer expiry is refused with 403 AccessDenied.

Confirm usage from tritonctl​

tritonctl shows the bucket, its object count, and its size, without an access key:

tritonctl storage buckets

If a request fails​

ResponseCause
403, InvalidAccessKeyIdThe access key ID is unknown to the object store. Check ~/.aws/credentials, and check with tritonctl storage access-keys --user <STORAGE_USER> that the key is listed.
403, SignatureDoesNotMatchThe secret access key is wrong.
403, RequestTimeTooSkewedYour workstation clock differs from the server's by more than 15 minutes.
403, AccessDenied, STREAMING-UNSIGNED-PAYLOAD-TRAILER is not enabled on this cluster; ...The client sent an unsigned streaming upload. Set request_checksum_calculation = when_required in the profile.
403, AccessDenied on mbThe key is scoped and lacks full on that bucket name, or an inline policy on the storage user denies s3:CreateBucket.
400, InvalidBucketNameThe bucket name breaks the naming rules.
400, InvalidLocationConstraintThe profile's region differs from the object store's region.
409, BucketAlreadyExistsAnother owner holds that bucket name. Choose another.
501, NotImplementedThe operation is not implemented, or the client sent a virtual-hosted-style request. Check that the profile sets addressing_style = path.
507, InsufficientStorageThe write would exceed your workspace quota. Ask your operator to raise it.
503, SlowDownYou exceeded the per-workspace request rate. Retry with backoff.

Clean up​

Delete the bucket when you no longer need it.

danger

This deletes the bucket and every object in it. There is no undo.

aws s3 rb s3://<BUCKET> --force --endpoint-url "$S3_ENDPOINT"

See also​