1===================
2Testing BPF on s390
3===================
4
51. Introduction
6***************
7
8IBM Z are mainframe computers, which are descendants of IBM System/360 from
9year 1964. They are supported by the Linux kernel under the name "s390". This
10document describes how to test BPF in an s390 QEMU guest.
11
122. One-time setup
13*****************
14
15The following is required to build and run the test suite:
16
17  * s390 GCC
18  * s390 development headers and libraries
19  * Clang with BPF support
20  * QEMU with s390 support
21  * Disk image with s390 rootfs
22
23Debian supports installing compiler and libraries for s390 out of the box.
24Users of other distros may use debootstrap in order to set up a Debian chroot::
25
26  sudo debootstrap \
27    --variant=minbase \
28    --include=sudo \
29    testing \
30    ./s390-toolchain
31  sudo mount --rbind /dev ./s390-toolchain/dev
32  sudo mount --rbind /proc ./s390-toolchain/proc
33  sudo mount --rbind /sys ./s390-toolchain/sys
34  sudo chroot ./s390-toolchain
35
36Once on Debian, the build prerequisites can be installed as follows::
37
38  sudo dpkg --add-architecture s390x
39  sudo apt-get update
40  sudo apt-get install \
41    bc \
42    bison \
43    cmake \
44    debootstrap \
45    dwarves \
46    flex \
47    g++ \
48    gcc \
49    g++-s390x-linux-gnu \
50    gcc-s390x-linux-gnu \
51    gdb-multiarch \
52    git \
53    make \
54    python3 \
55    qemu-system-misc \
56    qemu-utils \
57    rsync \
58    libcap-dev:s390x \
59    libelf-dev:s390x \
60    libncurses-dev
61
62Latest Clang targeting BPF can be installed as follows::
63
64  git clone https://github.com/llvm/llvm-project.git
65  ln -s ../../clang llvm-project/llvm/tools/
66  mkdir llvm-project-build
67  cd llvm-project-build
68  cmake \
69    -DLLVM_TARGETS_TO_BUILD=BPF \
70    -DCMAKE_BUILD_TYPE=Release \
71    -DCMAKE_INSTALL_PREFIX=/opt/clang-bpf \
72    ../llvm-project/llvm
73  make
74  sudo make install
75  export PATH=/opt/clang-bpf/bin:$PATH
76
77The disk image can be prepared using a loopback mount and debootstrap::
78
79  qemu-img create -f raw ./s390.img 1G
80  sudo losetup -f ./s390.img
81  sudo mkfs.ext4 /dev/loopX
82  mkdir ./s390.rootfs
83  sudo mount /dev/loopX ./s390.rootfs
84  sudo debootstrap \
85    --foreign \
86    --arch=s390x \
87    --variant=minbase \
88    --include=" \
89      iproute2, \
90      iputils-ping, \
91      isc-dhcp-client, \
92      kmod, \
93      libcap2, \
94      libelf1, \
95      netcat, \
96      procps" \
97    testing \
98    ./s390.rootfs
99  sudo umount ./s390.rootfs
100  sudo losetup -d /dev/loopX
101
1023. Compilation
103**************
104
105In addition to the usual Kconfig options required to run the BPF test suite, it
106is also helpful to select::
107
108  CONFIG_NET_9P=y
109  CONFIG_9P_FS=y
110  CONFIG_NET_9P_VIRTIO=y
111  CONFIG_VIRTIO_PCI=y
112
113as that would enable a very easy way to share files with the s390 virtual
114machine.
115
116Compiling kernel, modules and testsuite, as well as preparing gdb scripts to
117simplify debugging, can be done using the following commands::
118
119  make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- menuconfig
120  make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- bzImage modules scripts_gdb
121  make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- \
122    -C tools/testing/selftests \
123    TARGETS=bpf \
124    INSTALL_PATH=$PWD/tools/testing/selftests/kselftest_install \
125    install
126
1274. Running the test suite
128*************************
129
130The virtual machine can be started as follows::
131
132  qemu-system-s390x \
133    -cpu max,zpci=on \
134    -smp 2 \
135    -m 4G \
136    -kernel linux/arch/s390/boot/compressed/vmlinux \
137    -drive file=./s390.img,if=virtio,format=raw \
138    -nographic \
139    -append 'root=/dev/vda rw console=ttyS1' \
140    -virtfs local,path=./linux,security_model=none,mount_tag=linux \
141    -object rng-random,filename=/dev/urandom,id=rng0 \
142    -device virtio-rng-ccw,rng=rng0 \
143    -netdev user,id=net0 \
144    -device virtio-net-ccw,netdev=net0
145
146When using this on a real IBM Z, ``-enable-kvm`` may be added for better
147performance. When starting the virtual machine for the first time, disk image
148setup must be finalized using the following command::
149
150  /debootstrap/debootstrap --second-stage
151
152Directory with the code built on the host as well as ``/proc`` and ``/sys``
153need to be mounted as follows::
154
155  mkdir -p /linux
156  mount -t 9p linux /linux
157  mount -t proc proc /proc
158  mount -t sysfs sys /sys
159
160After that, the test suite can be run using the following commands::
161
162  cd /linux/tools/testing/selftests/kselftest_install
163  ./run_kselftest.sh
164
165As usual, tests can be also run individually::
166
167  cd /linux/tools/testing/selftests/bpf
168  ./test_verifier
169
1705. Debugging
171************
172
173It is possible to debug the s390 kernel using QEMU GDB stub, which is activated
174by passing ``-s`` to QEMU.
175
176It is preferable to turn KASLR off, so that gdb would know where to find the
177kernel image in memory, by building the kernel with::
178
179  RANDOMIZE_BASE=n
180
181GDB can then be attached using the following command::
182
183  gdb-multiarch -ex 'target remote localhost:1234' ./vmlinux
184
1856. Network
186**********
187
188In case one needs to use the network in the virtual machine in order to e.g.
189install additional packages, it can be configured using::
190
191  dhclient eth0
192
1937. Links
194********
195
196This document is a compilation of techniques, whose more comprehensive
197descriptions can be found by following these links:
198
199- `Debootstrap <https://wiki.debian.org/EmDebian/CrossDebootstrap>`_
200- `Multiarch <https://wiki.debian.org/Multiarch/HOWTO>`_
201- `Building LLVM <https://llvm.org/docs/CMake.html>`_
202- `Cross-compiling the kernel <https://wiki.gentoo.org/wiki/Embedded_Handbook/General/Cross-compiling_the_kernel>`_
203- `QEMU s390x Guest Support <https://wiki.qemu.org/Documentation/Platforms/S390X>`_
204- `Plan 9 folder sharing over Virtio <https://wiki.qemu.org/Documentation/9psetup>`_
205- `Using GDB with QEMU <https://wiki.osdev.org/Kernel_Debugging#Use_GDB_with_QEMU>`_
206