QEMU serial std output diverges on archlinux guest

Issue

I'm trying to bootstrap some installation automation of a freshly downloaded ISO in QEMU. I create a clean img to install to and kick off QEMU like this:

$ qemu-img create -f qcow2 out/main.img 15G
$ qemu-system-x86_64
  -m 8G \
  -serial stdio \
  -cdrom out/linux.iso \
  -drive file=out/main.img,if=virtio \
  -netdev user,id=net0 \
  -device e1000,netdev=net0

and I can see Arch boot up. At first both the display and the terminal are in sync, but they soon diverge after this the GRUB boot up screen.

I'm not sure what piece I'm missing to get this to work. I've seen some people suggest adding -append "root=/dev/sda console=ttyS0" to your QEMU arguments, but (from what I can tell) while it requires you to extract the kernel and the initram from the ISO (which should be easy enough as mounting and copy/pasting the correct files) but it also expects you to already have an installed system on /dev/sda (which is what I'm trying to bootstrap).

At this point I don't know what to search for next, how do I get the full terminal session in my current terminal and not just in my display?

Solution

In this case, it was as @Peter Maydell commented; this is not a QEMU question. QEMU was doing exactly what it was supposed to do, but Arch had to be told to utilize the serial console as its primary means of communication.

Two samples of how this can be done

bash via console

pipe_dir="$(mktemp -d)"
mkfifo "${pipe_dir}/pipe.in" "${pipe_dir}/pipe.out"
function cleanup {
    rm -rfv "${pipe_dir}"
}
trap cleanup EXIT

qemu-system-x86_64 \
  -m 8G \
  -display none \
  -serial stdio \
  -drive file=./out/linux.iso,index=0,media=cdrom \
  -drive file=./out/main.img,if=virtio &

sleep 2s
printf "\t" > "${pipe_dir}/pipe.in"
sleep 2s
printf " console=ttyS0,115200" > "${pipe_dir}/pipe.in"
sleep 2s
echo  > "${pipe_dir}/pipe.in"

# Whatever other interactions you want go here...

wait

expect via console

set timeout -1
spawn qemu-system-x86_64 \
  -m 8G \
  -display none \
  -serial stdio \
  -drive file=./out/linux.iso,index=0,media=cdrom \
  -drive file=./out/main.img,if=virtio

sleep 1
send \t
sleep 1
send " console=ttyS0,115200"
sleep 1
send \n

In theory this should be fine, but in practice I still had difficulty interacting with the console and sending characters over to login correctly. I'm sure there is probably more user-error on my part than not.

A better solution (again contextual to Arch and not QEMU specifically) was to use a cloud-init script that included my SSH public key. Interactions with the VM were stable, reliable, and easily reproducible.

bash with cloud-init/ssh

$ touch ./out/meta-data
$ cat > ./out/user-data <<EOF
#cloud-config
users:
  - name: root
    ssh_authorized_keys:
        - $(cat ${HOME}/.ssh/id_ed25519.pub)
EOF
$ xorriso -as genisoimage -output ./out/cloud-init.iso \
    -volid CIDATA -joliet -rock ./out/meta-data ./out/user-data
$ qemu-system-x86_64 \
    -m 8G \
    -drive file=./out/linux.iso,index=0,media=cdrom \
    -drive file=./out/cloud-init.iso,index=1,media=cdrom \
    -drive file=./out/main.img,if=virtio \
    -net user,hostfwd=tcp::10022-:22 \
    -net nic &

$ function qemu-ssh {
    ssh -q -o ConnectTimeout=5 -o StrictHostKeyChecking=no -o "UserKnownHostsFile /dev/null" -p 10022 root@localhost ${@}
  }

$ printf 'Waiting for SSH to go live (this will take a while)...'
$ until qemu-ssh exit; do
    printf '.'
  done

# This convenience function starts an interactive
# session when supplied with no additional arguments
# but your automation can go here
$ qemu-ssh 

Answered By - Paul FREAKN Baker