# mvirt 0.14.12

migrations: no
wire: unchanged
rev: da5b47f19dd596058c1a19098fd4408586bf4f2b

0.14.12 — the guest that wedged on its first write, and the console that would have shown it

A VM booted from a vhost-user disk mounted its root filesystem, remounted it r/w,
and then stopped dead about three and a half seconds in. From outside it looked
alive: it answered ICMP, and it sent a TCP reset on the ports its security group
allowed. Nothing ever started — no cloud-init, no sshd, no ssh.

The serial console says exactly what happened, once it is possible to read it:

  WARNING: CPU: 1 PID: 198 at drivers/virtio/virtio_ring.c:580
           virtqueue_add_split+0x486/0x7b0
  Comm: jbd2/vda1-8
   virtblk_add_req [virtio_blk] → virtio_queue_rqs → blk_mq_flush_plug_list
   → jbd2_journal_commit_transaction → kjournald2

That WARN fires on `total_sg > vq->split.vring.num && !vq->indirect`. virtio-blk
frames every request with a header descriptor and a status descriptor, so N data
segments need N+2 slots in the ring. cloud-hypervisor gives a vhost-user disk a
128-descriptor ring by default and the stor-agent advertised `seg_max = 128`, so
the first ext4 journal commit built a 130-descriptor chain for a 128-slot ring.
The driver refused it, the write never completed, and jbd2 blocked forever with
the filesystem already mounted — which is why the guest looked healthy and did
nothing.

- mvirt-stor-agent advertises `VIRTIO_RING_F_INDIRECT_DESC`. Its descriptor walk
  (`virtio_queue::DescriptorChain`) has always followed indirect tables; it simply
  never said so.
- `seg_max` drops to 126, two below the smallest ring the VMM can hand us, so a
  driver that does not use indirect descriptors stays in bounds as well.

**The console is the other half of this release.** None of the above was visible.
mvirt-vmm connected to the serial socket only when a console client attached, and
cloud-hypervisor buffers nothing — so firmware output, GRUB, the entire kernel
boot and any panic were dropped before anyone could ask for them. Attaching in a
retry loop from the first millisecond did not help, and neither did socat on the
node itself. The pod path has always recorded to `console.log`; the VM path does
now:

- the hypervisor owns the serial socket from VM start and tees it into
  `console.log`;
- keystrokes reach the guest through that recorder, because the socket accepts
  exactly one peer;
- the console RPC serves history first and then live output, so a VM that failed
  an hour ago can still be asked why.

A VM started by an older daemon keeps the previous direct-connect behaviour.

refs #216


## Upgrading

Point your flake's `mvirt` input at this release and redeploy the control-plane
host first, then each node in turn:

```
nix run .#colmena -- apply boot --reboot --on <control-plane host>
nix run .#colmena -- apply boot --reboot --on <node>   # one at a time
```

This release adds no migrations, so the control-plane host rolls back the same way
a node does: boot the previous NixOS generation.