1
2=======================
3Kexec and Kdump for Xen
4=======================
5
6This is a brief guide to using Kexec and Kdump in conjunction with Xen.
7This functionally works at the level of the hypervisor and dom0 kernel.
8And will thus affect all guests running on a machine.
9
10At this stage it does not work in conjunction with domU kernels.
11
12This document should be read in conjunction with
13Documentation/kdump/kdump.txt from the Linux kernel source.
14Some of the information in this document has been
15sourced from that document.
16
17
18Kexec
19=====
20
21It is possible to kexec from Xen or Linux to either Xen or Linux.
22
23Pattern        | Before Kexec       | After Kexec
24---------------+--------------------+--------------------
25Xen -> Xen     | first hypervisor & | second hypervisor &
26               | dom0 kernel        | dom0 kernel
27---------------+--------------------+--------------------
28Xen   -> Linux | first hypervisor & | second kernel
29               | dom0 kernel        |
30---------------+--------------------+--------------------
31Linux -> Xen   | first kernel       | second hypervisor &
32               |                    | dom0 kernel
33---------------+--------------------+--------------------
34Linux -> Linux | first kernel       | second kernel
35
36If you are kexecing to Xen then you will also need to prepare the second
37hypervisor and dom0 kernel that will run after kexec. These may be the same
38as the first hypervisor and dom0 kernel that are used before kexec if you
39are kexecing from Xen to Xen.
40
41If you are kexecing to Linux then you will need to prepare the second Linux
42kernel that will run after kexec. In the case that you are kexecing from
43Linux, it may be the same as the first kernel image that that runs before
44kexec.
45
46Regardless of which kexec pattern you wish to run, you will
47need to have kexec-tools installed. This provides the kexec command.
48
491. Load
50-------
51
52Before kexecing the second kernel or hypervisor & dom0 kernel
53need to be loaded into the running hypervisor or kernel using
54the kexec command.
55
56  a. To kexec to Xen (Xen->Xen or Linux->Xen)
57
58  kexec -l --append="XEN_ARGS -- DOM0_ARGS" \
59	--vmm="XEN_IMAGE" "DOM0_IMAGE" KEXEC_ARGS
60
61  where:
62    XEN_ARGS: command line arguments to the xen hypervisor
63              On x86 the no-real-mode argument should be included
64    DOM0_ARGS: command line arguments to the dom0 kernel
65    XEN_IMAGE: xen hypervisor image
66    DOM0_IMAGE: dom0 kernel image
67    KEXEC_ARGS: additional kexec-tools command line arguments
68
69  e.g. kexec -l --append "no-real-mode" --vmm="/boot/xen.gz" /boot/vmlinuz.gz
70
71  OR
72
73  b. To kexec to Linux (Xen->Linux or Linux->Linux)
74
75  kexec -l LINUX_IMAGE --append "$LINUX_ARGS" KEXEC_ARGS
76
77  where:
78    LINUX_IMAGE: the second linux kernel image
79    LINUX_ARGS: command line arguments to the second linux kernel
80    KEXEC_ARGS: additional kexec-tools command line arguments
81
82  e.g. kexec -l /boot/second-vmlinuz.gz
83
842. Execute
85----------
86
87Once the second kernel is loaded, it can be executed at any time.
88If you don't see the second kernel booting within a second or so,
89you are in trouble :(
90
91   kexec -e
92
93Kdump
94=====
95
96It is possible to kdump from Xen or Linux to a Linux crash kernel.
97It is not possible to use xen as a crash kernel.
98
99Pattern        | Before Kexec       | After Kexec
100---------------+--------------------+--------------------
101Xen -> Linux   | first hypervisor & | crash kernel
102               | dom0 kernel        |
103---------------+--------------------+--------------------
104Linux -> Linux | first kernel       | crash kernel
105
106Regardless of if you are kdumping from Xen or Linux you will need to
107prepare a linux crash kernel.  You will also need to have kexec-tools
108installed. This provides the kexec command.
109
1100. Set-Up The Crash Kernel Region
111---------------------------------
112
113In order to use kdump an area of memory has to be reserved at boot time.
114This is the area of memory that the crash kernel will use, thus allowing it
115to run without disrupting the memory used by the first kernel. This area is
116called the crash kernel region and is reserved using the crashkernel
117command line parameter to the Xen hypervisor. It has two forms:
118
119  i) crashkernel=size
120
121     This is the simplest and recommended way to reserve the crash kernel
122     region. Just specify how large the region should be and the hypervisor
123     will find a good location for it. A good size to start with is 128Mb
124
125     e.g.
126
127     crashkernel=128M
128
129  ii) crashkernel=size@base
130
131      In this form the base address is provided in addition to
132      the size. Use this if auto-placement doesn't work for some reason.
133      It is strongly recommended that the base address be aligned
134      to 64Mb, else memory below the alignment point will not
135      be usable.
136
137      e.g. crashkernel=128M@256M
138
139  iii) crashkernel=size,below=offset
140
141      This allows us to place the crash kernel within the usuable address
142      space without having to worry about a specific phyiscal address.
143      The '<' and 'below' options are  synonymous
144
145   Regardless of which of the forms of the crashkernel command line you
146   use, the crash kernel region should appear in /proc/iomem on x86. If it
147   doesn't then either the crashkernel parameter is missing, or for some
148   reason the region couldn't be placed - for instance because it is too large.
149
150   # cat /proc/iomem
151   ...
152   00100000-07feffff : System RAM
153     00100000-00bfffff : Hypervisor code and data
154     0533f000-0733efff : Crash kernel
155   ...
156
157
1581. Load
159-------
160
161Once you are running in a kexec-enabled hypervisor and dom0,
162you can prepare to kdump by loading the crash kernel into the
163running kernel.
164
165  kexec -p CRASH_KERNEL_IMAGE --append "$CRASH_KERNEL_ARGS" KEXEC_ARGS
166
167  where:
168    CRASH_KERNEL_IMAGE: the crash kernel image
169    CRASH_KERNEL_ARGS: command line arguments to the crash kernel
170		       init 1 is strongly recommended
171		       irqpoll is strongly recommended
172		       maxcpus=1 is required if the crash kernel is SMP
173		       reset_devices is strongly recommended
174    KEXEC_ARGS: additional kexec-tools command line arguments
175                On x86 --args-linux should be supplied if an uncompressed
176		vmlinux image is used as the crash kernel
177
178  e.g. kexec -p /boot/crash-vmlinuz \
179        --append "init 1 irqpoll maxcpus=1 reset_devices" --args-linux
180
181On x86 systems the crash kernel may be either
182- A uncompressed vmlinux image if the kernel is not relocatable
183- A compressed bzImage or vmlinuz image if the kernel is relocatable
184- Relocatability is controlled by the CONFIG_RELOCATABLE kernel
185  compile configuration parameter. This option may not be available
186  depending on the kernel version
187
188
1892. Execute
190----------
191
192Once the second kernel is loaded, the crash kernel will be executed if the
193hypervisor panics. It will also be executed if dom0 panics or if dom0
194oopses and /proc/sys/kernel/panic_on_oops is set to a non-zero value
195
196echo 1 > /proc/sys/kernel/panic_on_oops
197
198Kdump may also be triggered (for testing)
199
200  a. From Domain 0
201
202  echo c > /proc/sysrq-trigger
203
204  b. From Xen
205
206     Enter the xen console
207
208     ctrl^a ctrl^a  (may be bound to a different key, this is the default)
209
210     Select C for "trigger a crashdump"
211
212     C
213
214If you don't see the crash kernel booting within a second or so,
215you are in trouble :(
216
217