1# SPDX-License-Identifier: GPL-2.0+
2#
3# Copyright (C) 2018, Bin Meng <bmeng.cn@gmail.com>
4
5VirtIO Support
6==============
7
8This document describes the information about U-Boot support for VirtIO [1]
9devices, including supported boards, build instructions, driver details etc.
10
11What's VirtIO?
12--------------
13VirtIO is a virtualization standard for network and disk device drivers where
14just the guest's device driver "knows" it is running in a virtual environment,
15and cooperates with the hypervisor. This enables guests to get high performance
16network and disk operations, and gives most of the performance benefits of
17paravirtualization. In the U-Boot case, the guest is U-Boot itself, while the
18virtual environment are normally QEMU [2] targets like ARM, RISC-V and x86.
19
20Status
21------
22VirtIO can use various different buses, aka transports as described in the
23spec. While VirtIO devices are commonly implemented as PCI devices on x86,
24embedded devices models like ARM/RISC-V, which does not normally come with
25PCI support might use simple memory mapped device (MMIO) instead of the PCI
26device. The memory mapped virtio device behaviour is based on the PCI device
27specification. Therefore most operations including device initialization,
28queues configuration and buffer transfers are nearly identical. Both MMIO
29and PCI transport options are supported in U-Boot.
30
31The VirtIO spec defines a lots of VirtIO device types, however at present only
32network and block device, the most two commonly used devices, are supported.
33
34The following QEMU targets are supported.
35
36  - qemu_arm_defconfig
37  - qemu_arm64_defconfig
38  - qemu-riscv32_defconfig
39  - qemu-riscv64_defconfig
40  - qemu-x86_defconfig
41  - qemu-x86_64_defconfig
42
43Note ARM and RISC-V targets are configured with VirtIO MMIO transport driver,
44and on x86 it's the PCI transport driver.
45
46Build Instructions
47------------------
48Building U-Boot for pre-configured QEMU targets is no different from others.
49For example, we can do the following with the CROSS_COMPILE environment
50variable being properly set to a working toolchain for ARM:
51
52  $ make qemu_arm_defconfig
53  $ make
54
55You can even create a QEMU ARM target with VirtIO devices showing up on both
56MMIO and PCI buses. In this case, you can enable the PCI transport driver
57from 'make menuconfig':
58
59Device Drivers  --->
60	...
61	VirtIO Drivers  --->
62		...
63		[*] PCI driver for virtio devices
64
65Other drivers are at the same location and can be tuned to suit the needs.
66
67Requirements
68------------
69It is required that QEMU v2.5.0+ should be used to test U-Boot VirtIO support
70on QEMU ARM and x86, and v2.12.0+ on QEMU RISC-V.
71
72Testing
73-------
74The following QEMU command line is used to get U-Boot up and running with
75VirtIO net and block devices on ARM.
76
77  $ qemu-system-arm -nographic -machine virt -bios u-boot.bin \
78    -netdev tap,ifname=tap0,id=net0 \
79    -device virtio-net-device,netdev=net0 \
80    -drive if=none,file=test.img,format=raw,id=hd0 \
81    -device virtio-blk-device,drive=hd0
82
83On x86, command is slightly different to create PCI VirtIO devices.
84
85  $ qemu-system-i386 -nographic -bios u-boot.rom \
86    -netdev tap,ifname=tap0,id=net0 \
87    -device virtio-net-pci,netdev=net0 \
88    -drive if=none,file=test.img,format=raw,id=hd0 \
89    -device virtio-blk-pci,drive=hd0
90
91Additional net and block devices can be created by appending more '-device'
92parameters. It is also possible to specify both MMIO and PCI VirtIO devices.
93For example, the following commnad creates 3 VirtIO devices, with 1 on MMIO
94and 2 on PCI bus.
95
96  $ qemu-system-arm -nographic -machine virt -bios u-boot.bin \
97    -netdev tap,ifname=tap0,id=net0 \
98    -device virtio-net-pci,netdev=net0 \
99    -drive if=none,file=test0.img,format=raw,id=hd0 \
100    -device virtio-blk-device,drive=hd0 \
101    -drive if=none,file=test1.img,format=raw,id=hd1 \
102    -device virtio-blk-pci,drive=hd1
103
104By default QEMU creates VirtIO legacy devices by default. To create non-legacy
105(aka modern) devices, pass additional device property/value pairs like below:
106
107  $ qemu-system-i386 -nographic -bios u-boot.rom \
108    -netdev tap,ifname=tap0,id=net0 \
109    -device virtio-net-pci,netdev=net0,disable-legacy=true,disable-modern=false \
110    -drive if=none,file=test.img,format=raw,id=hd0 \
111    -device virtio-blk-pci,drive=hd0,disable-legacy=true,disable-modern=false
112
113A 'virtio' command is provided in U-Boot shell.
114
115  => virtio
116  virtio - virtio block devices sub-system
117
118  Usage:
119  virtio scan - initialize virtio bus
120  virtio info - show all available virtio block devices
121  virtio device [dev] - show or set current virtio block device
122  virtio part [dev] - print partition table of one or all virtio block devices
123  virtio read addr blk# cnt - read `cnt' blocks starting at block
124       `blk#' to memory address `addr'
125  virtio write addr blk# cnt - write `cnt' blocks starting at block
126       `blk#' from memory address `addr'
127
128To probe all the VirtIO devices, type:
129
130  => virtio scan
131
132Then we can show the connected block device details by:
133
134  => virtio info
135  Device 0: QEMU VirtIO Block Device
136              Type: Hard Disk
137              Capacity: 4096.0 MB = 4.0 GB (8388608 x 512)
138
139And list the directories and files on the disk by:
140
141  => ls virtio 0 /
142  <DIR>       4096 .
143  <DIR>       4096 ..
144  <DIR>      16384 lost+found
145  <DIR>       4096 dev
146  <DIR>       4096 proc
147  <DIR>       4096 sys
148  <DIR>       4096 var
149  <DIR>       4096 etc
150  <DIR>       4096 usr
151  <SYM>          7 bin
152  <SYM>          8 sbin
153  <SYM>          7 lib
154  <SYM>          9 lib64
155  <DIR>       4096 run
156  <DIR>       4096 boot
157  <DIR>       4096 home
158  <DIR>       4096 media
159  <DIR>       4096 mnt
160  <DIR>       4096 opt
161  <DIR>       4096 root
162  <DIR>       4096 srv
163  <DIR>       4096 tmp
164                 0 .autorelabel
165
166Driver Internals
167----------------
168There are 3 level of drivers in the VirtIO driver family.
169
170	+---------------------------------------+
171	|	 virtio device drivers		|
172	|    +-------------+ +------------+	|
173	|    | virtio-net  | | virtio-blk |	|
174	|    +-------------+ +------------+	|
175	+---------------------------------------+
176	+---------------------------------------+
177	|	virtio transport drivers	|
178	|    +-------------+ +------------+	|
179	|    | virtio-mmio | | virtio-pci |	|
180	|    +-------------+ +------------+	|
181	+---------------------------------------+
182		+----------------------+
183		| virtio uclass driver |
184		+----------------------+
185
186The root one is the virtio uclass driver (virtio-uclass.c), which does lots of
187common stuff for the transport drivers (virtio_mmio.c, virtio_pci.c). The real
188virtio device is discovered in the transport driver's probe() method, and its
189device ID is saved in the virtio uclass's private data of the transport device.
190Then in the virtio uclass's post_probe() method, the real virtio device driver
191(virtio_net.c, virtio_blk.c) is bound if there is a match on the device ID.
192
193The child_post_bind(), child_pre_probe() and child_post_probe() methods of the
194virtio uclass driver help bring the virtio device driver online. They do things
195like acknowledging device, feature negotiation, etc, which are really common
196for all virtio devices.
197
198The transport drivers provide a set of ops (struct dm_virtio_ops) for the real
199virtio device driver to call. These ops APIs's parameter is designed to remind
200the caller to pass the correct 'struct udevice' id of the virtio device, eg:
201
202int virtio_get_status(struct udevice *vdev, u8 *status)
203
204So the parameter 'vdev' indicates the device should be the real virtio device.
205But we also have an API like:
206
207struct virtqueue *vring_create_virtqueue(unsigned int index, unsigned int num,
208					 unsigned int vring_align,
209					 struct udevice *udev)
210
211Here the parameter 'udev' indicates the device should be the transport device.
212Similar naming is applied in other functions that are even not APIs, eg:
213
214static int virtio_uclass_post_probe(struct udevice *udev)
215static int virtio_uclass_child_pre_probe(struct udevice *vdev)
216
217So it's easy to tell which device these functions are operating on.
218
219Development Flow
220----------------
221At present only VirtIO network card (device ID 1) and block device (device
222ID 2) are supported. If you want to develop new driver for new devices,
223please follow the guideline below.
224
2251. add new device ID in virtio.h
226#define VIRTIO_ID_XXX		X
227
2282. update VIRTIO_ID_MAX_NUM to be the largest device ID plus 1
229
2303. add new driver name string in virtio.h
231#define VIRTIO_XXX_DRV_NAME	"virtio-xxx"
232
2334. create a new driver with name set to the name string above
234U_BOOT_DRIVER(virtio_xxx) = {
235	.name = VIRTIO_XXX_DRV_NAME,
236	...
237	.remove = virtio_reset,
238	.flags = DM_FLAG_ACTIVE_DMA,
239}
240
241Note the driver needs to provide the remove method and normally this can be
242hooked to virtio_reset(). The driver flags should contain DM_FLAG_ACTIVE_DMA
243for the remove method to be called before jumping to OS.
244
2455. provide bind() method in the driver, where virtio_driver_features_init()
246   should be called for driver to negotiate feature support with the device.
247
2486. do funny stuff with the driver
249
250References
251----------
252[1] http://docs.oasis-open.org/virtio/virtio/v1.0/virtio-v1.0.pdf
253[2] https://www.qemu.org
254