petr-korobeinikov.com

Faking systemd inside a docker container, with testinfra on top

· docker, systemd, testinfra

Our teams lean heavily on ansible for templating and the final configuration of virtual machines. That brings up the question of how to make sure deployment scenarios actually work and stay correct.

This post is about faking systemd inside a docker container and validating the end result with the testinfra framework.

Working with molecule will be covered in other posts.

The problem

  • Give the team a simple, fast, reproducible tool for locally verifying that ansible playbooks and roles are correct.
  • Provide ssh access to the container so ansible can run against it.
  • Most importantly, make it possible to use the ansible.builtin.systemd module.

Picking an approach

Our goal is fast iteration on a developer’s laptop. So:

  • virtualbox + vagrant:
    • slow VM startup;
    • doesn’t work for the lucky folks on M1.
  • multipass:
    • fast, but the VM start is still noticeable;
    • requires a small bit of extra setup for ssh access.
  • docker:
    • lightning-fast container startup;
    • lets you “bake in” a reusable sshd configuration once;
    • not designed to use systemd as the init process.

In practice we use two variants:

  • multipass + multipass-compose when we need real virtual machines;
  • docker when we need to verify changes quickly.

Let’s look at the docker-image solution.

The naive approach

The obvious move is to launch systemd as the container’s init process. And, of course, it doesn’t work like that.

# syntax=docker/dockerfile:1

FROM ubuntu:focal

ENV DEBIAN_FRONTEND=noninteractive

RUN <<EOF
apt update

apt install -y --no-install-recommends \
    systemd
EOF


CMD ["/bin/systemd"]

The container starts, but the moment you reach for systemctl you’ll get:

System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down

A solution that meets the requirements

Let’s prepare an image with the following installed and configured:

  • the systemd package (effectively only needed to create the directory layout);
  • gdraheim/docker-systemctl-replacement as a stub for systemctl and journalctl;
  • a user that belongs to the sudo group.

The entry point is sshd.

# syntax=docker/dockerfile:1

FROM ubuntu:focal

ENV DEBIAN_FRONTEND=noninteractive

RUN <<EOF
apt update

apt install -y --no-install-recommends \
    openssh-server \
    sudo \
    gnupg \
    systemd \
    python3

useradd -rm -d /home/ubuntu -s /bin/bash -g root -G sudo -u 1000 ubuntu
echo 'ubuntu:ubuntu' | chpasswd

service ssh start
EOF

COPY systemctl3.py /usr/bin/systemctl
COPY journalctl3.py /usr/bin/journalctl

RUN chmod +x /usr/bin/systemctl /usr/bin/journalctl

EXPOSE 22

CMD ["/usr/sbin/sshd","-D"]

While running, sshd will ignore SIGTERM when stopping the container, so you should run it either with --init or with an explicit --stop-signal.

Once the container is up, you can ssh into it — which is exactly what we need to run ansible and testinfra.

ssh \
  -o UserKnownHostsFile=/dev/null \
  -o ControlMaster=auto -o \
  ControlPersist=60s \
    ubuntu@localhost

testinfra

When checking service state via the Service module, testinfra runs a series of probes to determine which init subsystem is in use on the current machine.

To explicitly tell it we’re using systemd, you need a fairly harmless and compact monkeypatch:

import testinfra.modules.service

testinfra.modules.service.Service.get_module_class = classmethod(
  lambda *args, **kwargs: testinfra.modules.service.SystemdService
)

This explicitly tells testinfra to lean on systemctl.

Wrap-up

What did we end up with?

  • A fast-starting container with a systemd stand-in and ssh access.
  • A simple way to verify ansible playbooks and roles.
  • The ability to run testinfra tests.
  • Reproducibility and portability across developer machines.

You can see the whole thing put together on my github:

Good luck with ansible and your testing!

References