# mvirt 0.11.0

migrations: yes
wire: unchanged
rev: 6263c478b749f0a88577d6ed7f71644db3a5998c

v0.11.0 — the control plane can actually lose a member

The release that makes a three-voter control plane real. 0.10.0 could not grow
an existing deployment at all; growing one then found three further defects
that a group formed from nothing cannot reach, each of which cost the dev fleet
an outage. All four are fixed, each with a test that fails without it, and the
result was proven end to end: three voters, the leader stopped, another member
serving within six seconds and committing writes while it was down.

Control plane — growth and failover

- mvdb: the raft log's purge boundary can never outrun the applied index that
  is durable in the materialized view. A snapshot install is not crash-atomic
  across the two redb files — openraft forwards the install to its worker
  without awaiting it and purges the log inline — so a death in that window
  left a store no restart could boot. That is what turned mvirt.io's growth
  into a quorum outage. A shared watermark, a clamped purge, an install that
  writes both files in order, and a boot check that repairs what is repairable
  and refuses to discard a voter's log. (#100)
- A learner is promoted to voter only once its own replica is running and
  caught up, not merely once its host is Online. Promoting replicas that are
  not running takes the quorum, and the write that would have started them is
  the slice that needs that quorum — a deadlock that does not end on its own.
- An interrupted membership change is completed rather than mistaken for done.
  A change commits twice and nothing retries the second, so a group could sit
  in a joint configuration with the member list of an HA deployment and the
  fault tolerance of a single voter.
- The startup seeds follow leadership rather than how the process was
  configured. The bootstrap host of a grown group could not restart after a
  failover: it ran the seeds as a follower and died on the first one.
- Server leaves are P-256. BoringSSL never offers ed25519 for verifying, so
  Envoy could not complete a handshake with an Ed25519 leaf and reported every
  health check as a network failure — with the control plane healthy the whole
  time. (#101)
- GET /v1/nodes carries each host's replica state, so a growth can be watched
  without ssh.

Object storage

- ADR-0074: pluggable object-storage backends — the entity and its migration,
  the provider trait with a Bunny implementation, the level reconciler, admin
  REST and console.
- ADR-0075: the S3 proxy is the default path in front of provider buckets.

Upgrade

Control-plane host first, then the others one at a time. This release migrates
the store, so a control-plane host cannot roll back by booting the previous
generation — the way back is a store restore. Nodes roll back normally.


## Upgrading

Point your flake's `mvirt` input at this release and redeploy the control-plane
host first, then each node in turn:

```
nix run .#colmena -- apply boot --reboot --on <control-plane host>
nix run .#colmena -- apply boot --reboot --on <node>   # one at a time
```

**This release migrates the store.** Once the control plane has started, the
previous version will refuse to run against it — migrations are forward-only, so
rolling the control-plane host back means restoring a backup, with data loss back
to the backup point. The control plane writes a backup immediately before it
applies migrations; nodes roll back freely by booting the previous generation.
