1.. SPDX-License-Identifier: GPL-2.0 2 3================= 4kunit_tool How-To 5================= 6 7What is kunit_tool? 8=================== 9 10kunit_tool is a script (``tools/testing/kunit/kunit.py``) that aids in building 11the Linux kernel as UML (`User Mode Linux 12<http://user-mode-linux.sourceforge.net/>`_), running KUnit tests, parsing 13the test results and displaying them in a user friendly manner. 14 15kunit_tool addresses the problem of being able to run tests without needing a 16virtual machine or actual hardware with User Mode Linux. User Mode Linux is a 17Linux architecture, like ARM or x86; however, unlike other architectures it 18compiles the kernel as a standalone Linux executable that can be run like any 19other program directly inside of a host operating system. To be clear, it does 20not require any virtualization support: it is just a regular program. 21 22What is a .kunitconfig? 23======================= 24 25It's just a defconfig that kunit_tool looks for in the build directory 26(``.kunit`` by default). kunit_tool uses it to generate a .config as you might 27expect. In addition, it verifies that the generated .config contains the CONFIG 28options in the .kunitconfig; the reason it does this is so that it is easy to 29be sure that a CONFIG that enables a test actually ends up in the .config. 30 31It's also possible to pass a separate .kunitconfig fragment to kunit_tool, 32which is useful if you have several different groups of tests you wish 33to run independently, or if you want to use pre-defined test configs for 34certain subsystems. 35 36Getting Started with kunit_tool 37=============================== 38 39If a kunitconfig is present at the root directory, all you have to do is: 40 41.. code-block:: bash 42 43 ./tools/testing/kunit/kunit.py run 44 45However, you most likely want to use it with the following options: 46 47.. code-block:: bash 48 49 ./tools/testing/kunit/kunit.py run --timeout=30 --jobs=`nproc --all` 50 51- ``--timeout`` sets a maximum amount of time to allow tests to run. 52- ``--jobs`` sets the number of threads to use to build the kernel. 53 54.. note:: 55 This command will work even without a .kunitconfig file: if no 56 .kunitconfig is present, a default one will be used instead. 57 58If you wish to use a different .kunitconfig file (such as one provided for 59testing a particular subsystem), you can pass it as an option. 60 61.. code-block:: bash 62 63 ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4/.kunitconfig 64 65For a list of all the flags supported by kunit_tool, you can run: 66 67.. code-block:: bash 68 69 ./tools/testing/kunit/kunit.py run --help 70 71Configuring, Building, and Running Tests 72======================================== 73 74It's also possible to run just parts of the KUnit build process independently, 75which is useful if you want to make manual changes to part of the process. 76 77A .config can be generated from a .kunitconfig by using the ``config`` argument 78when running kunit_tool: 79 80.. code-block:: bash 81 82 ./tools/testing/kunit/kunit.py config 83 84Similarly, if you just want to build a KUnit kernel from the current .config, 85you can use the ``build`` argument: 86 87.. code-block:: bash 88 89 ./tools/testing/kunit/kunit.py build 90 91And, if you already have a built UML kernel with built-in KUnit tests, you can 92run the kernel and display the test results with the ``exec`` argument: 93 94.. code-block:: bash 95 96 ./tools/testing/kunit/kunit.py exec 97 98The ``run`` command which is discussed above is equivalent to running all three 99of these in sequence. 100 101All of these commands accept a number of optional command-line arguments. The 102``--help`` flag will give a complete list of these, or keep reading this page 103for a guide to some of the more useful ones. 104 105Parsing Test Results 106==================== 107 108KUnit tests output their results in TAP (Test Anything Protocol) format. 109kunit_tool will, when running tests, parse this output and print a summary 110which is much more pleasant to read. If you wish to look at the raw test 111results in TAP format, you can pass the ``--raw_output`` argument. 112 113.. code-block:: bash 114 115 ./tools/testing/kunit/kunit.py run --raw_output 116 117The raw output from test runs may contain other, non-KUnit kernel log 118lines. You can see just KUnit output with ``--raw_output=kunit``: 119 120.. code-block:: bash 121 122 ./tools/testing/kunit/kunit.py run --raw_output=kunit 123 124If you have KUnit results in their raw TAP format, you can parse them and print 125the human-readable summary with the ``parse`` command for kunit_tool. This 126accepts a filename for an argument, or will read from standard input. 127 128.. code-block:: bash 129 130 # Reading from a file 131 ./tools/testing/kunit/kunit.py parse /var/log/dmesg 132 # Reading from stdin 133 dmesg | ./tools/testing/kunit/kunit.py parse 134 135This is very useful if you wish to run tests in a configuration not supported 136by kunit_tool (such as on real hardware, or an unsupported architecture). 137 138Filtering Tests 139=============== 140 141It's possible to run only a subset of the tests built into a kernel by passing 142a filter to the ``exec`` or ``run`` commands. For example, if you only wanted 143to run KUnit resource tests, you could use: 144 145.. code-block:: bash 146 147 ./tools/testing/kunit/kunit.py run 'kunit-resource*' 148 149This uses the standard glob format for wildcards. 150 151Running Tests on QEMU 152===================== 153 154kunit_tool supports running tests on QEMU as well as via UML (as mentioned 155elsewhere). The default way of running tests on QEMU requires two flags: 156 157``--arch`` 158 Selects a collection of configs (Kconfig as well as QEMU configs 159 options, etc) that allow KUnit tests to be run on the specified 160 architecture in a minimal way; this is usually not much slower than 161 using UML. The architecture argument is the same as the name of the 162 option passed to the ``ARCH`` variable used by Kbuild. Not all 163 architectures are currently supported by this flag, but can be handled 164 by the ``--qemu_config`` discussed later. If ``um`` is passed (or this 165 this flag is ignored) the tests will run via UML. Non-UML architectures, 166 e.g. i386, x86_64, arm, um, etc. Non-UML run on QEMU. 167 168``--cross_compile`` 169 Specifies the use of a toolchain by Kbuild. The argument passed here is 170 the same passed to the ``CROSS_COMPILE`` variable used by Kbuild. As a 171 reminder this will be the prefix for the toolchain binaries such as gcc 172 for example ``sparc64-linux-gnu-`` if you have the sparc toolchain 173 installed on your system, or 174 ``$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux-`` 175 if you have downloaded the microblaze toolchain from the 0-day website 176 to a directory in your home directory called ``toolchains``. 177 178In many cases it is likely that you may want to run an architecture which is 179not supported by the ``--arch`` flag, or you may want to just run KUnit tests 180on QEMU using a non-default configuration. For this use case, you can write 181your own QemuConfig. These QemuConfigs are written in Python. They must have an 182import line ``from ..qemu_config import QemuArchParams`` at the top of the file 183and the file must contain a variable called ``QEMU_ARCH`` that has an instance 184of ``QemuArchParams`` assigned to it. An example can be seen in 185``tools/testing/kunit/qemu_configs/x86_64.py``. 186 187Once you have a QemuConfig you can pass it into kunit_tool using the 188``--qemu_config`` flag; when used this flag replaces the ``--arch`` flag. If we 189were to do this with the ``x86_64.py`` example from above, the invocation would 190look something like this: 191 192.. code-block:: bash 193 194 ./tools/testing/kunit/kunit.py run \ 195 --timeout=60 \ 196 --jobs=12 \ 197 --qemu_config=./tools/testing/kunit/qemu_configs/x86_64.py 198 199Other Useful Options 200==================== 201 202kunit_tool has a number of other command-line arguments which can be useful 203when adapting it to fit your environment or needs. 204 205Some of the more useful ones are: 206 207``--help`` 208 Lists all of the available options. Note that different commands 209 (``config``, ``build``, ``run``, etc) will have different supported 210 options. Place ``--help`` before the command to list common options, 211 and after the command for options specific to that command. 212 213``--build_dir`` 214 Specifies the build directory that kunit_tool will use. This is where 215 the .kunitconfig file is located, as well as where the .config and 216 compiled kernel will be placed. Defaults to ``.kunit``. 217 218``--make_options`` 219 Specifies additional options to pass to ``make`` when compiling a 220 kernel (with the ``build`` or ``run`` commands). For example, to enable 221 compiler warnings, you can pass ``--make_options W=1``. 222 223``--alltests`` 224 Builds a UML kernel with all config options enabled using ``make 225 allyesconfig``. This allows you to run as many tests as is possible, 226 but is very slow and prone to breakage as new options are added or 227 modified. In most cases, enabling all tests which have satisfied 228 dependencies by adding ``CONFIG_KUNIT_ALL_TESTS=1`` to your 229 .kunitconfig is preferable. 230 231There are several other options (and new ones are often added), so do check 232``--help`` if you're looking for something not mentioned here. 233