1==============================
2Memory Layout on AArch64 Linux
3==============================
4
5Author: Catalin Marinas <catalin.marinas@arm.com>
6
7This document describes the virtual memory layout used by the AArch64
8Linux kernel. The architecture allows up to 4 levels of translation
9tables with a 4KB page size and up to 3 levels with a 64KB page size.
10
11AArch64 Linux uses either 3 levels or 4 levels of translation tables
12with the 4KB page configuration, allowing 39-bit (512GB) or 48-bit
13(256TB) virtual addresses, respectively, for both user and kernel. With
1464KB pages, only 2 levels of translation tables, allowing 42-bit (4TB)
15virtual address, are used but the memory layout is the same.
16
17ARMv8.2 adds optional support for Large Virtual Address space. This is
18only available when running with a 64KB page size and expands the
19number of descriptors in the first level of translation.
20
21User addresses have bits 63:48 set to 0 while the kernel addresses have
22the same bits set to 1. TTBRx selection is given by bit 63 of the
23virtual address. The swapper_pg_dir contains only kernel (global)
24mappings while the user pgd contains only user (non-global) mappings.
25The swapper_pg_dir address is written to TTBR1 and never written to
26TTBR0.
27
28
29AArch64 Linux memory layout with 4KB pages + 4 levels (48-bit)::
30
31  Start			End			Size		Use
32  -----------------------------------------------------------------------
33  0000000000000000	0000ffffffffffff	 256TB		user
34  ffff000000000000	ffff7fffffffffff	 128TB		kernel logical memory map
35 [ffff600000000000	ffff7fffffffffff]	  32TB		[kasan shadow region]
36  ffff800000000000	ffff800007ffffff	 128MB		bpf jit region
37  ffff800008000000	ffff80000fffffff	 128MB		modules
38  ffff800010000000	fffffbffefffffff	 124TB		vmalloc
39  fffffbfff0000000	fffffbfffdffffff	 224MB		fixed mappings (top down)
40  fffffbfffe000000	fffffbfffe7fffff	   8MB		[guard region]
41  fffffbfffe800000	fffffbffff7fffff	  16MB		PCI I/O space
42  fffffbffff800000	fffffbffffffffff	   8MB		[guard region]
43  fffffc0000000000	fffffdffffffffff	   2TB		vmemmap
44  fffffe0000000000	ffffffffffffffff	   2TB		[guard region]
45
46
47AArch64 Linux memory layout with 64KB pages + 3 levels (52-bit with HW support)::
48
49  Start			End			Size		Use
50  -----------------------------------------------------------------------
51  0000000000000000	000fffffffffffff	   4PB		user
52  fff0000000000000	ffff7fffffffffff	  ~4PB		kernel logical memory map
53 [fffd800000000000	ffff7fffffffffff]	 512TB		[kasan shadow region]
54  ffff800000000000	ffff800007ffffff	 128MB		bpf jit region
55  ffff800008000000	ffff80000fffffff	 128MB		modules
56  ffff800010000000	fffffbffefffffff	 124TB		vmalloc
57  fffffbfff0000000	fffffbfffdffffff	 224MB		fixed mappings (top down)
58  fffffbfffe000000	fffffbfffe7fffff	   8MB		[guard region]
59  fffffbfffe800000	fffffbffff7fffff	  16MB		PCI I/O space
60  fffffbffff800000	fffffbffffffffff	   8MB		[guard region]
61  fffffc0000000000	ffffffdfffffffff	  ~4TB		vmemmap
62  ffffffe000000000	ffffffffffffffff	 128GB		[guard region]
63
64
65Translation table lookup with 4KB pages::
66
67  +--------+--------+--------+--------+--------+--------+--------+--------+
68  |63    56|55    48|47    40|39    32|31    24|23    16|15     8|7      0|
69  +--------+--------+--------+--------+--------+--------+--------+--------+
70   |                 |         |         |         |         |
71   |                 |         |         |         |         v
72   |                 |         |         |         |   [11:0]  in-page offset
73   |                 |         |         |         +-> [20:12] L3 index
74   |                 |         |         +-----------> [29:21] L2 index
75   |                 |         +---------------------> [38:30] L1 index
76   |                 +-------------------------------> [47:39] L0 index
77   +-------------------------------------------------> [63] TTBR0/1
78
79
80Translation table lookup with 64KB pages::
81
82  +--------+--------+--------+--------+--------+--------+--------+--------+
83  |63    56|55    48|47    40|39    32|31    24|23    16|15     8|7      0|
84  +--------+--------+--------+--------+--------+--------+--------+--------+
85   |                 |    |               |              |
86   |                 |    |               |              v
87   |                 |    |               |            [15:0]  in-page offset
88   |                 |    |               +----------> [28:16] L3 index
89   |                 |    +--------------------------> [41:29] L2 index
90   |                 +-------------------------------> [47:42] L1 index (48-bit)
91   |                                                   [51:42] L1 index (52-bit)
92   +-------------------------------------------------> [63] TTBR0/1
93
94
95When using KVM without the Virtualization Host Extensions, the
96hypervisor maps kernel pages in EL2 at a fixed (and potentially
97random) offset from the linear mapping. See the kern_hyp_va macro and
98kvm_update_va_mask function for more details. MMIO devices such as
99GICv2 gets mapped next to the HYP idmap page, as do vectors when
100ARM64_SPECTRE_V3A is enabled for particular CPUs.
101
102When using KVM with the Virtualization Host Extensions, no additional
103mappings are created, since the host kernel runs directly in EL2.
104
10552-bit VA support in the kernel
106-------------------------------
107If the ARMv8.2-LVA optional feature is present, and we are running
108with a 64KB page size; then it is possible to use 52-bits of address
109space for both userspace and kernel addresses. However, any kernel
110binary that supports 52-bit must also be able to fall back to 48-bit
111at early boot time if the hardware feature is not present.
112
113This fallback mechanism necessitates the kernel .text to be in the
114higher addresses such that they are invariant to 48/52-bit VAs. Due
115to the kasan shadow being a fraction of the entire kernel VA space,
116the end of the kasan shadow must also be in the higher half of the
117kernel VA space for both 48/52-bit. (Switching from 48-bit to 52-bit,
118the end of the kasan shadow is invariant and dependent on ~0UL,
119whilst the start address will "grow" towards the lower addresses).
120
121In order to optimise phys_to_virt and virt_to_phys, the PAGE_OFFSET
122is kept constant at 0xFFF0000000000000 (corresponding to 52-bit),
123this obviates the need for an extra variable read. The physvirt
124offset and vmemmap offsets are computed at early boot to enable
125this logic.
126
127As a single binary will need to support both 48-bit and 52-bit VA
128spaces, the VMEMMAP must be sized large enough for 52-bit VAs and
129also must be sized large enough to accommodate a fixed PAGE_OFFSET.
130
131Most code in the kernel should not need to consider the VA_BITS, for
132code that does need to know the VA size the variables are
133defined as follows:
134
135VA_BITS		constant	the *maximum* VA space size
136
137VA_BITS_MIN	constant	the *minimum* VA space size
138
139vabits_actual	variable	the *actual* VA space size
140
141
142Maximum and minimum sizes can be useful to ensure that buffers are
143sized large enough or that addresses are positioned close enough for
144the "worst" case.
145
14652-bit userspace VAs
147--------------------
148To maintain compatibility with software that relies on the ARMv8.0
149VA space maximum size of 48-bits, the kernel will, by default,
150return virtual addresses to userspace from a 48-bit range.
151
152Software can "opt-in" to receiving VAs from a 52-bit space by
153specifying an mmap hint parameter that is larger than 48-bit.
154
155For example:
156
157.. code-block:: c
158
159   maybe_high_address = mmap(~0UL, size, prot, flags,...);
160
161It is also possible to build a debug kernel that returns addresses
162from a 52-bit space by enabling the following kernel config options:
163
164.. code-block:: sh
165
166   CONFIG_EXPERT=y && CONFIG_ARM64_FORCE_52BIT=y
167
168Note that this option is only intended for debugging applications
169and should not be used in production.
170