Singularity

Reference

Verify a signed launch and acquire separate signing, authorization and Most credentials.

Bootstrap

singularity-bootstrap verifies a signed launch manifest, proves its workload identity to Skarbiec, materializes three credentials as unlinked owner-only files, and replaces itself with the named executable under a controlled environment. No wrapper process remains to wait for a child; the inherited descriptors own the credentials' lifetime.

Invocation

singularity-bootstrap \
  --manifest <file> \
  --manifest-signature <file> \
  --trust-root <file> \
  --runtime-root <absolute-directory>

The corresponding variables are SINGULARITY_BOOTSTRAP_MANIFEST, SINGULARITY_BOOTSTRAP_MANIFEST_SIG, SINGULARITY_BOOTSTRAP_TRUST_ROOT and SINGULARITY_RUNTIME_ROOT. Manifest, signature, trust root and workload private key must be owner-only files. On successful exec the same process becomes the being, whose eventual exit status reaches the supervisor; a failed exec returns its operating-system cause.

Signed launch version

The signature covers the exact manifest bytes after the byte prefix SINGULARITY-BOOTSTRAP-MANIFEST\0v2\0. The trust root is an Ed25519 public key. The manifest must satisfy these checks:

  • version is singularity.bootstrap.v2.
  • It is not expired, and it expires after it was issued. How long it lives is the issuer's signed expires_at; the consumer sets no ceiling of its own.
  • The workload public key, executable, code and policy digests, and all three capability IDs are 64 lowercase hexadecimal characters.
  • Agent, role, environment, host and workload identities are nonempty, trimmed, NUL-free and bounded. The policy sequence is positive.
  • Broker socket, workload private key and executable paths are absolute.
  • The three capability IDs are distinct. Every capability targets singularity-bootstrap.
  • capabilities.brama binds purpose singularity.brama.bootstrap to a brama: resource.
  • capabilities.brama_bearer binds purpose singularity.brama.authorization to a brama: resource.
  • capabilities.most binds purpose singularity.most.bootstrap to a most: resource.

The actual executable must match executable_digest. The private signing key must derive the manifest's public key. Identity signing does not replace the separate Brama caller authorization.

Migrating a launch from version 1

The supervising deployment must issue the additional workload-bound Brama authorization capability, include it as capabilities.brama_bearer, set version 2 and sign the resulting bytes under the version-2 domain. Regenerate and resign the complete launch ticket; editing an issued ticket invalidates its signature. A version-1 ticket or signature is not accepted by the version-2 consumer. The Skarbiec redemption wire protocol remains version 1.

Redemption and cleanup

For each capability, bootstrap generates a fresh nonce and sends a skarbiec.redeem.v1 request with operation redeem to the broker's Unix socket. Its proof is the workload key's Ed25519 signature over SKARBIEC-WORKLOAD-PROOF\0v1\0, then the capability ID, nonce, workload ID and redeem, each ended by a NUL byte, then the authorization ID, which is empty for a bootstrap capability: exactly what Skarbiec verifies and Brama signs. Only an ok control response followed by exactly secret_len bytes is accepted.

A control line is read to its newline and a credential to the length the broker states; nothing is cut and no clock decides. Local failures name the broker socket, failed operation and underlying I/O or JSON cause. A control-read failure includes the observed byte count; truncated credentials and trailing data name the declared length without printing secret bytes. A broker refusal reports its observed wire version and status; Skarbiec's log names the authorization check that refused it.

The declared credential length is reserved with a fallible allocation before reading its bytes. An allocation refusal reports the broker, declared length and allocator cause, rather than deliberately aborting on an infallible allocation. The serialized redemption request is erased on scope exit, including connection failures.

The fresh singularity-<uuid> runtime directory has mode 0700. Each credential is opened as a new owner-only file and unlinked before any secret byte is written. The retained descriptor carries the credential; write, sync or metadata failures cannot leave a named secret file. An unlink failure stops materialization before writing the secret. Closing the descriptor releases the unlinked file.

Creation, unlink, anonymous-descriptor write, sync and metadata inspection failures name the exact operation, destination and operating-system cause without credential contents. A rejected descriptor reports its observed regular-file status and permission mode. If unlink fails, only the empty file can remain because no secret bytes have been written.

Cleanup unlinks leftover credential pathnames without opening or following them. It no longer allocates a file-sized overwrite buffer. Failure to remove the runtime directory refuses execution with the path and operating-system cause; cleanup during another refusal reports its own failure on stderr without replacing the original cause.

Child environment

The launcher starts with env_clear(). It restores a verified absolute HOME, a controlled PATH beginning with that owner's .local/bin and .stado/bin, and LANG/LC_ALL. The home must be owned by the current Unix principal and must not be group- or world-writable.

The manifest supplies SINGULARITY_AGENT_ID, SINGULARITY_ROLE, SINGULARITY_ENVIRONMENT, SINGULARITY_HOST, SINGULARITY_WORKLOAD_ID, SINGULARITY_WORKLOAD_PUBLIC_KEY, SINGULARITY_EXECUTABLE_SHA256, SINGULARITY_CODE_SHA256, SINGULARITY_POLICY_SHA256 and SINGULARITY_POLICY_SEQUENCE.

The three inherited descriptors are named by SINGULARITY_BRAMA_HMAC_FD, SINGULARITY_BRAMA_BEARER_FD and SINGULARITY_MOST_TOKEN_FD; no materialized credential pathname is exported. The being validates distinct, unlinked, nonempty owner-only files before reading them, then restores close-on-exec. Only an explicit Singularity child handoff inherits them; ordinary tools do not. The signed singularity_args array selects the command and its options, including an ecosystem policy path when launching the portfolio. The remaining parent environment is not inherited.

The consumer checks each inherited file's observed length before converting it to an addressable allocation size and reserves that allocation fallibly. Inspection, allocation and read errors name the descriptor declaration, operation and cause, retaining the observed length where available without exposing credential bytes.

Invalid inherited files report regular-file status, link count, owner and effective owner, permissions and length. Metadata inspection and close-on-exec failures identify the descriptor declaration, descriptor number and operating-system cause.

Issuing a launch ticket

singularity ticket writes what this bootstrap starts from. Every input is explicit; nothing is read from the environment or defaulted.

singularity ticket key --holder supervisor|workload --out <absolute-private-key> --public-out <absolute-public-key>
singularity ticket sign \
  --agent-id <id> --role <role> --environment <env> --host <host> --workload-id <id> \
  --workload-key <workload-private-key> --broker-socket <absolute-socket> \
  --executable <absolute-singularity> --code-digest <sha256> \
  --policy-file <ecosystem-policy> --policy-sequence <n> --expires-in-seconds <n> \
  --brama-capability <id> --brama-resource brama:<field> \
  --brama-bearer-capability <id> --brama-bearer-resource brama:<field> \
  --most-capability <id> --most-resource most:<field> \
  --supervisor-key <supervisor-private-key> --trust-root <supervisor-public-key> \
  --manifest-out <absolute-file> --signature-out <absolute-file> -- <singularity args>

ticket key reads a fresh seed from /dev/urandom and writes the private key as a new owner-only hex file and its public half as a new owner-only file in the encoding its reader requires (an existing file is refused). --holder supervisor writes the public half as hex: it is the trust root this bootstrap reads with --trust-root. --holder workload writes it as a PEM public key: skarbiec grant issue --workload-public-key-file refuses anything else (workload public key must be a PEM public key) and Skarbiec verifies every redemption proof against it with openssl pkeyutl -verify. The three capability ids come from skarbiec grant capability. ticket sign fills the manifest from its flags, takes the executable's and the policy file's SHA-256, checks the manifest against the rules above before writing anything, signs the bytes under the version-2 domain with the supervisor key, writes manifest and signature as new owner-only files, and then runs the same check this bootstrap runs on its input. A ticket that check refuses is removed when cleanup succeeds, and the command retains singularity-bootstrap refused the ticket written: <reason>, including any cleanup failure. Both verbs answer JSON (or --text): the key paths and public hex, or the ticket paths, digests and validity window.

The issuer encodes both key halves before publishing either. If publishing the public half fails, it removes only the private file created by that invocation; an existing destination remains unchanged. Write and sync failures name the path and underlying cause and remove the partial file when cleanup succeeds. Ticket signing follows the same ownership rule for manifest and signature. Cleanup attempts every owned file and reports any removal failure alongside the original cause, rather than claiming that nothing remains. These are failure-cleanup guarantees, not an atomic transaction across two output paths. The real CLI regression in tests/bootstrap/ticket.rs refuses an occupied public destination, checks retained bytes and absence of the new private file, then retries successfully and verifies the produced PEM with OpenSSL.

singularity ticket launch is the managed start a service declaration runs. A ticket expires within minutes and names a workload key and three capabilities, so each start makes its own: it writes a fresh workload key under a new owner-only ticket-<uuid> directory of --runtime-root, registers its PEM public half with <--skarbiec> grant issue <agent-id> --capabilities <--grant-capabilities> --ttl-seconds <--grant-ttl-seconds> --workload-public-key-file, issues the three capabilities with grant capability --agent <agent-id> --purpose <purpose> --resource <resource> --target singularity-bootstrap --ttl <--capability-ttl-seconds> --max-uses <--capability-max-uses>, signs and checks the ticket exactly as ticket sign does, and then runs this bootstrap on it, which replaces the process with the being. It takes ticket sign's identity, executable, policy, resource, supervisor and -- inputs; the workload key, capability ids and ticket paths are its own. Every refusal names the step: <skarbiec> could not be started, skarbiec grant issue refused: <Skarbiec's sentence>, skarbiec grant capability for <purpose> answered no capability_id, or the bootstrap's own refusal. No ticket is signed before all three capabilities exist.

Deployment files

deploy/capabilities/ contains systemd units and drop-ins, a launchd template, environment examples, sysusers/tmpfiles definitions and the reusable deployment preflight. Those declarations do not prove that a workload can redeem its capabilities or use Brama; managed-launch qualification must exercise those consumers.

Settings