Single node or cluster
What you win with a cluster.
One ISO installs both. Everything not in a row, the API, CLI, Terraform provider and console, is the same.
-
Single node
One machine runs everything. For a lab, an edge site or a first look. No fault tolerance.
Install guide -
Hyperconverged
3 to 5 machines, each running everything. Survives a machine failing; adds object storage, the managed database and Auto Scaling.
Install guide
| Service or feature | Single node | Hyperconverged |
|---|---|---|
| Platform services | ||
| Compute | Yes KVM instances | Yes across the nodes |
| Block storage | Yes LVM-thin pools | Yes Ceph, 3 copies, plus LVM-thin |
| Networking | Yes OVN | Yes OVN, replicated databases |
| Load balancing | Yes application load balancers | Yes plus Auto Scaling targets |
| DNS | Yes hosted zones | Yes hosted zones |
| Object storage (S3) | No needs Ceph | Yes Ceph object gateway |
| Machine images | Yes no bucket import or export | Yes plus bucket import and export |
| Auto Scaling | No not offered | Yes groups, policies, refresh |
| Identity | Yes orgs, projects, policies | Yes the same |
| Secrets Manager | Yes versions, rotation | Yes OpenBao as a raft cluster |
| SSH access | Yes short-lived certificates | Yes the same |
| Client VPN | Yes WireGuard endpoints | Yes the same |
| Observability and alerts | Yes runs on the node | Yes the same |
| Audit | Yes hash-chained events | Yes the same |
| Managed services | ||
| Stratum Database | No not offered | Yes PostgreSQL; others planned |
| Stratum Kubernetes | No not offered | Planned no tenant API yet |
| Managed backup | Partly volumes only | Partly volumes only |
| Messaging | Planned topics and queues | Planned topics and queues |
| Resilience and operations | ||
| Control plane | No one node, no virtual addresses | Yes 3 to 5 nodes |
| Replicated cluster state | No one copy | Yes Postgres, OpenBao, OVN |
| Ceph storage and its capacity chart | No no Ceph | Yes capacity chart in the console |
| Moving instances between nodes | No one node | Yes cold and live migration |
| A node fails | No one machine, no fault tolerance | Yes survives 1 of 3 (2 of 5) |
| Upgrades, and what they do to instances | Yes an OS release restarts instances | Yes rolling, tested on 3 nodes |
| Backup of the cloud’s own state | Yes opt-in | Partly opt-in, restore not yet run end to end |
| Adding nodes | No install again as a cluster | Yes up to 5 nodes in all |
| Compute-only and storage-only nodes | No one node | Planned planned, later |
| Removing or replacing a node | No one node | Planned planned for the next release |
| Minimum hardware | ||
| Machines | 1 | 3 to 5 |
| Each machine | 2 cores, 8 GiB, 131 GiB disk | the same, plus TPM 2.0 and a Ceph disk |
Notes behind the table
- Object storage (S3)
- Ceph cannot form on one node, so a single node has no S3. On a cluster Ceph is set up after the install, the gateway after that.
- Stratum Database
- PostgreSQL today; Valkey, MongoDB and MariaDB are planned. Needs object storage (the WAL archive), Secrets Manager, Auto Scaling and machine images; its engine image is built after the install. Snapshots, point-in-time restore, Multi-AZ and blue/green upgrades.
- Stratum Kubernetes
- Planned. The service is registered and the cluster builds the Kubernetes node image recipe, but there is no tenant API yet. Not planned for one node.
- Managed backup
- Plans, vaults, recovery points and restore for volumes; an operator sets the backup target first. Databases and buckets are not covered. Ceph volumes can use incremental chains.
- Messaging
- No spec yet, and which shapes get it is not decided.
- Control plane
- Every control-plane node runs it. Six virtual addresses (API, Postgres, OpenBao, OVN north and south, metrics query) each follow their own leader. A single node serves the console at :8088 and the API at :8443 on itself.
- Replicated cluster state
- Postgres runs under Patroni with a synchronous standby; OpenBao and the OVN databases are raft clusters. A single node has plain Postgres.
- Ceph storage and its capacity chart
- The Ceph page charts raw, used and available capacity, and per pool, over the last hour, day or week.
- Moving instances between nodes
- Live migration needs Ceph-backed instances and the same CPU model on both nodes; cold migration stops the guest. Drain a node to move everything off it.
- A node fails
- The virtual addresses move in seconds. Instances on Ceph restart on a survivor in roughly two to three minutes by the code’s timers (not measured); instances on a node’s local pool do not restart. With two of three nodes down the cluster loses quorum.
- Upgrades, and what they do to instances
- Container and host releases keep instances running; an OS release restarts the node and its instances unless you drain it first. A cluster rolls one node at a time, the leader’s last, gated on health. 0.7.0 starts the upgrade chain, so the first upgrade between releases will be 0.7.0 to 0.8.0.
- Backup of the cloud’s own state
- Off until you set a target. A single node takes its own backup before an upgrade and can restore it in place. A cluster restores into a fresh first node and the others rejoin; a full-site restore has not run on the ISO-installed flow, and Ceph comes back empty.
- Adding nodes
- A single node never grows into a cluster. A cluster takes a join code, the same ISO on the new machine, and an approval; one planned for three takes a fourth and fifth.
- Compute-only and storage-only nodes
- Every node that joins is a full node (control plane, compute and Ceph), so a cluster stops at 5 nodes; more come with these.
- Removing or replacing a node
- A lost node can be decommissioned, which cuts its access; taking it out of the databases and Ceph is manual and replacing a dead node has not been run.
- Machines
- Three is the smallest cluster that survives losing one; two is worse than one machine.
- Each machine
- Single node: x86-64 with VT-x or AMD-V, UEFI, 2 cores (4 recommended), 8 GiB (16 recommended), one disk of 131 GiB (111 GiB with the whole-disk layout), one wired port. A cluster node adds a TPM 2.0 for OpenBao’s unseal key, a blank disk for Ceph and jumbo frames between nodes; Ceph and the control plane take about 4 GiB per OSD and 16 GiB on a control-plane node from instances.