1% RTDS Scheduler
2% Revision 1
3
4\clearpage
5
6# Basics
7---------------- ----------------------------------------------------
8         Status: **Experimental**
9
10      Component: Hypervisor
11---------------- ----------------------------------------------------
12
13# Overview
14
15RTDS is one of the virtual CPU (vCPU) scheduler available in the Xen
16hypervisor.
17
18RTDS is a real-time scheduler, so its purpose is enabling
19**deterministic** scheduling of the virtual machine's vCPUs. It has
20been originally developed in the context of the RT-Xen project.
21
22# User details
23
24RTDS is not in use by default. In order to use it as the Xen scheduler
25the following parameter should be passed to the hypervisor at boot:
26
27    `sched=rtds`
28
29Once the system is live, for creating a cpupool with RTDS as its
30scheduler, either compile a cpupool configuration file, as described
31in `docs/man/xlcpupool.cfg.pod.5` (and as exemplified in
32`tools/examples/cpupool`), or use just `xl` directly:
33
34    xl cpupool-create name=\"pool-rt\" sched=\"rtds\" cpus=[4,5,6,8]
35
36For checking or changing a VM's scheduling parameters from xl, do
37as follows:
38    * `xl sched-rtds -d vm-rt -v all`
39    * `xl sched-rtds -d vm-rt -v all -p 10000 -b 2500`
40
41It is possible, for a multiple vCPUs VM, to change the parameters of
42each vCPU individually:
43    * `xl sched-rtds -d vm-rt -v 0 -p 20000 -b 10000 -e 1 -v 1 -p 45000 -b 12000 -e 0`
44
45# Technical details
46
47Implementation entirely lives in the hypervisor. Xen has a pluggable,
48hook based, architecture for schedulers. Thanks to this, RTDS code
49is all contained in `xen/common/sched_rtds.c`.
50
51In libxl, the availability of the RTDS scheduler is advertised by
52the presence of the LIBXL_HAVE_SCHED_RTDS symbol. The ability of
53specifying different scheduling parameters for each vcpu has been
54introduced later, and is available if the following symbols are defined:
55    * LIBXL_HAVE_VCPU_SCHED_PARAMS,
56    * LIBXL_HAVE_SCHED_RTDS_VCPU_PARAMS,
57    * LIBXL_HAVE_SCHED_RTDS_VCPU_EXTRA.
58
59# Limitations
60
61RTDS is a special purpose scheduling. This is by design, and not at
62all a limitation, but it is certainly something to keep in mind when
63thinking about using it. The purpose of the scheduler is enabling
64deterministic and statically analyzable behavior (as per the
65real-time academic literature), according to the scheduling parameters
66assigned to each vCPU.
67
68Using RTDS a the Xen scheduler, and/or for general purpose workloads
69is definitely possible, but the vCPU scheduling parameters (of both
70Domain0 and of the various VMs) would probably require tweaking, with
71respect to their default values.
72
73# Testing
74
75Any change done in RTDS must be tested by doing the following:
76
77* create a cpupool with RTDS as its scheduler,
78* create a few virtual machines a move them in and out of the pool,
79* create a few virtual machines, directly inside the pool, and verify
80  that they boot and can run some basic workload (e.g., login into them
81  and run simple commands),
82* shutdown/reboot the virtual machines,
83
84The fact that the system boots fine when passing `sched=rtds` to Xen
85should also be verified.
86
87Finally, to check that the scheduler is working properly (although only
88at a macroscopic level), the following should be done:
89
90* create a VM with 1 vCPU and put it in the RTDS cpupool,
91* set the scheduling parameters such as it has a 50% reservation, with
92  `xl sched-rtds -d vm -v all -p 100000 -b 50000`,
93* run a CPU-burning process inside the VM (e.g., `yes`),
94* check with `xentop` (in Domain0) that the VM is getting no more than
95  50% pCPU time.
96
97# Areas for improvement
98
99* performance assessment, especially focusing on what level of real-time
100  behavior the scheduler enables.
101
102# Known issues
103
104* OSSTest reports occasional failures on ARM.
105
106# References
107
108* "RT-Xen: Real-Time Virtualization" [XPDS14 Presentation](http://events.linuxfoundation.org/sites/events/files/slides/2014_Xen_Developer_Summit_0.pdf)
109* "Scheduling in Xen" [XPDS15 Presentation](http://events.linuxfoundation.org/sites/events/files/slides/Faggioli_XenSummit.pdf)
110* [RT-Xen Project](https://sites.google.com/site/realtimexen/)
111* [RTDS-Based-Scheduler](https://wiki.xenproject.org/wiki/RTDS-Based-Scheduler)
112* "The RTDS Scheduler" [on the Xen-Project wiki](https://wiki.xenproject.org/wiki/RTDS-Based-Scheduler)
113* "Xen Project Schedulers" [on the Xen-Project wiki](https://wiki.xenproject.org/wiki/Xen_Project_Schedulers)
114
115# History
116
117------------------------------------------------------------------------
118Date       Revision Version  Notes
119---------- -------- -------- -------------------------------------------
1202016-10-14 1        Xen 4.8  Document written
1212017-08-31 2        Xen 4.10 Revise for work conserving feature
122---------- -------- -------- -------------------------------------------
123