1# SPDX-License-Identifier: GPL-2.0+
2# Copyright (c) 2013 The Chromium OS Authors.
3
4(Please read 'How to change from MAKEALL' if you are used to that tool)
5
6Quick-start
7===========
8
9If you just want to quickly set up buildman so you can build something (for
10example Raspberry Pi 2):
11
12 cd /path/to/u-boot
13 PATH=$PATH:`pwd`/tools/buildman
14 buildman --fetch-arch arm
15 buildman -k rpi_2
16 ls ../current/rpi_2
17 # u-boot.bin is the output image
18
19
20What is this?
21=============
22
23This tool handles building U-Boot to check that you have not broken it
24with your patch series. It can build each individual commit and report
25which boards fail on which commits, and which errors come up. It aims
26to make full use of multi-processor machines.
27
28A key feature of buildman is its output summary, which allows warnings,
29errors or image size increases in a particular commit or board to be
30quickly identified and the offending commit pinpointed. This can be a big
31help for anyone working with >10 patches at a time.
32
33
34Caveats
35=======
36
37Buildman can be stopped and restarted, in which case it will continue
38where it left off. This should happen cleanly and without side-effects.
39If not, it is a bug, for which a patch would be welcome.
40
41Buildman gets so tied up in its work that it can ignore the outside world.
42You may need to press Ctrl-C several times to quit it. Also it will print
43out various exceptions when stopped. You may have to kill it since the
44Ctrl-C handling is somewhat broken.
45
46
47Theory of Operation
48===================
49
50(please read this section in full twice or you will be perpetually confused)
51
52Buildman is a builder. It is not make, although it runs make. It does not
53produce any useful output on the terminal while building, except for
54progress information (but see -v below). All the output (errors, warnings and
55binaries if you ask for them) is stored in output directories, which you can
56look at from a separate 'buildman -s' instance while the build is progressing,
57or when it is finished.
58
59Buildman is designed to build entire git branches, i.e. muliple commits. It
60can be run repeatedly on the same branch after making changes to commits on
61that branch. In this case it will automatically rebuild commits which have
62changed (and remove its old results for that commit). It is possible to build
63a branch for one board, then later build it for another board. This adds to
64the output, so now you have results for two boards. If you want buildman to
65re-build a commit it has already built (e.g. because of a toolchain update),
66use the -f flag.
67
68Buildman produces a concise summary of which boards succeeded and failed.
69It shows which commit introduced which board failure using a simple
70red/green colour coding (with yellow/cyan for warnings). Full error
71information can be requested, in which case it is de-duped and displayed
72against the commit that introduced the error. An example workflow is below.
73
74Buildman stores image size information and can report changes in image size
75from commit to commit. An example of this is below.
76
77Buildman starts multiple threads, and each thread builds for one board at
78a time. A thread starts at the first commit, configures the source for your
79board and builds it. Then it checks out the next commit and does an
80incremental build (i.e. not using 'make xxx_defconfig' unless you use -C).
81Eventually the thread reaches the last commit and stops. If a commit causes
82an error or warning, buildman will try it again after reconfiguring (but see
83-Q). Thus some commits may be built twice, with the first result silently
84discarded. Lots of errors and warnings will causes lots of reconfigures and your
85build will be very slow. This is because a file that produces just a warning
86would not normally be rebuilt in an incremental build. Once a thread finishes
87building all the commits for a board, it starts on the commits for another
88board.
89
90Buildman works in an entirely separate place from your U-Boot repository.
91It creates a separate working directory for each thread, and puts the
92output files in the working directory, organised by commit name and board
93name, in a two-level hierarchy (but see -P).
94
95Buildman is invoked in your U-Boot directory, the one with the .git
96directory. It clones this repository into a copy for each thread, and the
97threads do not affect the state of your git repository. Any checkouts done
98by the thread affect only the working directory for that thread.
99
100Buildman automatically selects the correct tool chain for each board. You
101must supply suitable tool chains (see --fetch-arch), but buildman takes care
102of selecting the right one.
103
104Buildman generally builds a branch (with the -b flag), and in this case
105builds the upstream commit as well, for comparison. So even if you have one
106commit in your branch, two commits will be built. Put all your commits in a
107branch, set the branch's upstream to a valid value, and all will be well.
108Otherwise buildman will perform random actions. Use -n to check what the
109random actions might be.
110
111Buildman effectively has two modes: without -s it builds, with -s it
112summarises the results of previous (or active) builds.
113
114If you just want to build the current source tree, leave off the -b flag.
115This will display results and errors as they happen. You can still look at
116them later using -se. Note that buildman will assume that the source has
117changed, and will build all specified boards in this case.
118
119Buildman is optimised for building many commits at once, for many boards.
120On multi-core machines, Buildman is fast because it uses most of the
121available CPU power. When it gets to the end, or if you are building just
122a few commits or boards, it will be pretty slow. As a tip, if you don't
123plan to use your machine for anything else, you can use -T to increase the
124number of threads beyond the default.
125
126
127Selecting which boards to build
128===============================
129
130Buildman lets you build all boards, or a subset. Specify the subset by passing
131command-line arguments that list the desired board name, architecture name,
132SOC name, or anything else in the boards.cfg file. Multiple arguments are
133allowed. Each argument will be interpreted as a regular expression, so
134behaviour is a superset of exact or substring matching. Examples are:
135
136* 'tegra20' All boards with a Tegra20 SoC
137* 'tegra' All boards with any Tegra Soc (Tegra20, Tegra30, Tegra114...)
138* '^tegra[23]0$' All boards with either Tegra20 or Tegra30 SoC
139* 'powerpc' All PowerPC boards
140
141While the default is to OR the terms together, you can also make use of
142the '&' operator to limit the selection:
143
144* 'freescale & arm sandbox' All Freescale boards with ARM architecture,
145 plus sandbox
146
147You can also use -x to specifically exclude some boards. For example:
148
149 buildman arm -x nvidia,freescale,.*ball$
150
151means to build all arm boards except nvidia, freescale and anything ending
152with 'ball'.
153
154For building specific boards you can use the --boards (or --bo) option, which
155takes a comma-separated list of board target names and be used multiple times
156on the command line:
157
158 buildman --boards sandbox,snow --boards
159
160It is convenient to use the -n option to see what will be built based on
161the subset given. Use -v as well to get an actual list of boards.
162
163Buildman does not store intermediate object files. It optionally copies
164the binary output into a directory when a build is successful (-k). Size
165information is always recorded. It needs a fair bit of disk space to work,
166typically 250MB per thread.
167
168
169Setting up
170==========
171
1721. Get the U-Boot source. You probably already have it, but if not these
173steps should get you started with a repo and some commits for testing.
174
175$ cd /path/to/u-boot
176$ git clone git://git.denx.de/u-boot.git .
177$ git checkout -b my-branch origin/master
178$ # Add some commits to the branch, reading for testing
179
1802. Create ~/.buildman to tell buildman where to find tool chains (see 'The
181.buildman file' later for details). As an example:
182
183# Buildman settings file
184
185[toolchain]
186root: /
187rest: /toolchains/*
188eldk: /opt/eldk-4.2
189arm: /opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux
190aarch64: /opt/linaro/gcc-linaro-aarch64-none-elf-4.8-2013.10_linux
191
192[toolchain-alias]
193x86: i386
194blackfin: bfin
195nds32: nds32le
196openrisc: or1k
197
198
199This selects the available toolchain paths. Add the base directory for
200each of your toolchains here. Buildman will search inside these directories
201and also in any '/usr' and '/usr/bin' subdirectories.
202
203Make sure the tags (here root: rest: and eldk:) are unique.
204
205The toolchain-alias section indicates that the i386 toolchain should be used
206to build x86 commits.
207
208Note that you can also specific exactly toolchain prefixes if you like:
209
210[toolchain-prefix]
211arm: /opt/arm-eabi-4.6/bin/arm-eabi-
212
213or even:
214
215[toolchain-prefix]
216arm: /opt/arm-eabi-4.6/bin/arm-eabi-gcc
217
218This tells buildman that you want to use this exact toolchain for the arm
219architecture. This will override any toolchains found by searching using the
220[toolchain] settings.
221
222Since the toolchain prefix is an explicit request, buildman will report an
223error if a toolchain is not found with that prefix. The current PATH will be
224searched, so it is possible to use:
225
226[toolchain-prefix]
227arm: arm-none-eabi-
228
229and buildman will find arm-none-eabi-gcc in /usr/bin if you have it installed.
230
231[toolchain-wrapper]
232wrapper: ccache
233
234This tells buildman to use a compiler wrapper in front of CROSS_COMPILE. In
235this example, ccache. It doesn't affect the toolchain scan. The wrapper is
236added when CROSS_COMPILE environtal variable is set. The name in this
237section is ignored. If more than one line is provided, only the last one
238is taken.
239
2403. Make sure you have the require Python pre-requisites
241
242Buildman uses multiprocessing, Queue, shutil, StringIO, ConfigParser and
243urllib2. These should normally be available, but if you get an error like
244this then you will need to obtain those modules:
245
246 ImportError: No module named multiprocessing
247
248
2494. Check the available toolchains
250
251Run this check to make sure that you have a toolchain for every architecture.
252
253$ ./tools/buildman/buildman --list-tool-chains
254Scanning for tool chains
255 - scanning prefix '/opt/gcc-4.6.3-nolibc/x86_64-linux/bin/x86_64-linux-'
256Tool chain test: OK, arch='x86', priority 1
257 - scanning prefix '/opt/arm-eabi-4.6/bin/arm-eabi-'
258Tool chain test: OK, arch='arm', priority 1
259 - scanning path '/toolchains/gcc-4.9.0-nolibc/i386-linux'
260 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/.'
261 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/bin'
262 - found '/toolchains/gcc-4.9.0-nolibc/i386-linux/bin/i386-linux-gcc'
263 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/usr/bin'
264Tool chain test: OK, arch='i386', priority 4
265 - scanning path '/toolchains/gcc-4.9.0-nolibc/aarch64-linux'
266 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/.'
267 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin'
268 - found '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc'
269 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/usr/bin'
270Tool chain test: OK, arch='aarch64', priority 4
271 - scanning path '/toolchains/gcc-4.9.0-nolibc/microblaze-linux'
272 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/.'
273 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin'
274 - found '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc'
275 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/usr/bin'
276Tool chain test: OK, arch='microblaze', priority 4
277 - scanning path '/toolchains/gcc-4.9.0-nolibc/mips64-linux'
278 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/.'
279 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/bin'
280 - found '/toolchains/gcc-4.9.0-nolibc/mips64-linux/bin/mips64-linux-gcc'
281 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/usr/bin'
282Tool chain test: OK, arch='mips64', priority 4
283 - scanning path '/toolchains/gcc-4.9.0-nolibc/sparc64-linux'
284 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/.'
285 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin'
286 - found '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc'
287 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/usr/bin'
288Tool chain test: OK, arch='sparc64', priority 4
289 - scanning path '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi'
290 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/.'
291 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin'
292 - found '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc'
293 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/usr/bin'
294Tool chain test: OK, arch='arm', priority 3
295Toolchain '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc' at priority 3 will be ignored because another toolchain for arch 'arm' has priority 1
296 - scanning path '/toolchains/gcc-4.9.0-nolibc/sparc-linux'
297 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/.'
298 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/bin'
299 - found '/toolchains/gcc-4.9.0-nolibc/sparc-linux/bin/sparc-linux-gcc'
300 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/usr/bin'
301Tool chain test: OK, arch='sparc', priority 4
302 - scanning path '/toolchains/gcc-4.9.0-nolibc/mips-linux'
303 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/.'
304 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/bin'
305 - found '/toolchains/gcc-4.9.0-nolibc/mips-linux/bin/mips-linux-gcc'
306 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/usr/bin'
307Tool chain test: OK, arch='mips', priority 4
308 - scanning path '/toolchains/gcc-4.9.0-nolibc/x86_64-linux'
309 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/.'
310 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin'
311 - found '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc'
312 - found '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-x86_64-linux-gcc'
313 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/usr/bin'
314Tool chain test: OK, arch='x86_64', priority 4
315Tool chain test: OK, arch='x86_64', priority 4
316Toolchain '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-x86_64-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'x86_64' has priority 4
317 - scanning path '/toolchains/gcc-4.9.0-nolibc/m68k-linux'
318 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/.'
319 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/bin'
320 - found '/toolchains/gcc-4.9.0-nolibc/m68k-linux/bin/m68k-linux-gcc'
321 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/usr/bin'
322Tool chain test: OK, arch='m68k', priority 4
323 - scanning path '/toolchains/gcc-4.9.0-nolibc/powerpc-linux'
324 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/.'
325 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin'
326 - found '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc'
327 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/usr/bin'
328Tool chain test: OK, arch='powerpc', priority 4
329 - scanning path '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux'
330 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/.'
331 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin'
332 - found '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin/bfin-uclinux-gcc'
333 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/usr/bin'
334Tool chain test: OK, arch='bfin', priority 6
335 - scanning path '/toolchains/gcc-4.6.3-nolibc/sparc-linux'
336 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/.'
337 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin'
338 - found '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin/sparc-linux-gcc'
339 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/usr/bin'
340Tool chain test: OK, arch='sparc', priority 4
341Toolchain '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin/sparc-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'sparc' has priority 4
342 - scanning path '/toolchains/gcc-4.6.3-nolibc/mips-linux'
343 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/.'
344 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin'
345 - found '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin/mips-linux-gcc'
346 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/usr/bin'
347Tool chain test: OK, arch='mips', priority 4
348Toolchain '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin/mips-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'mips' has priority 4
349 - scanning path '/toolchains/gcc-4.6.3-nolibc/m68k-linux'
350 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/.'
351 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin'
352 - found '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux-gcc'
353 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/usr/bin'
354Tool chain test: OK, arch='m68k', priority 4
355Toolchain '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'm68k' has priority 4
356 - scanning path '/toolchains/gcc-4.6.3-nolibc/powerpc-linux'
357 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/.'
358 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/bin'
359 - found '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/bin/powerpc-linux-gcc'
360 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/usr/bin'
361Tool chain test: OK, arch='powerpc', priority 4
362Tool chain test: OK, arch='or32', priority 4
363 - scanning path '/'
364 - looking in '/.'
365 - looking in '/bin'
366 - looking in '/usr/bin'
367 - found '/usr/bin/i586-mingw32msvc-gcc'
368 - found '/usr/bin/c89-gcc'
369 - found '/usr/bin/x86_64-linux-gnu-gcc'
370 - found '/usr/bin/gcc'
371 - found '/usr/bin/c99-gcc'
372 - found '/usr/bin/arm-linux-gnueabi-gcc'
373 - found '/usr/bin/aarch64-linux-gnu-gcc'
374 - found '/usr/bin/winegcc'
375 - found '/usr/bin/arm-linux-gnueabihf-gcc'
376Tool chain test: OK, arch='i586', priority 11
377Tool chain test: OK, arch='c89', priority 11
378Tool chain test: OK, arch='x86_64', priority 4
379Toolchain '/usr/bin/x86_64-linux-gnu-gcc' at priority 4 will be ignored because another toolchain for arch 'x86_64' has priority 4
380Tool chain test: OK, arch='sandbox', priority 11
381Tool chain test: OK, arch='c99', priority 11
382Tool chain test: OK, arch='arm', priority 4
383Toolchain '/usr/bin/arm-linux-gnueabi-gcc' at priority 4 will be ignored because another toolchain for arch 'arm' has priority 1
384Tool chain test: OK, arch='aarch64', priority 4
385Toolchain '/usr/bin/aarch64-linux-gnu-gcc' at priority 4 will be ignored because another toolchain for arch 'aarch64' has priority 4
386Tool chain test: OK, arch='sandbox', priority 11
387Toolchain '/usr/bin/winegcc' at priority 11 will be ignored because another toolchain for arch 'sandbox' has priority 11
388Tool chain test: OK, arch='arm', priority 4
389Toolchain '/usr/bin/arm-linux-gnueabihf-gcc' at priority 4 will be ignored because another toolchain for arch 'arm' has priority 1
390List of available toolchains (34):
391aarch64 : /toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc
392alpha : /toolchains/gcc-4.9.0-nolibc/alpha-linux/bin/alpha-linux-gcc
393am33_2.0 : /toolchains/gcc-4.9.0-nolibc/am33_2.0-linux/bin/am33_2.0-linux-gcc
394arm : /opt/arm-eabi-4.6/bin/arm-eabi-gcc
395bfin : /toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin/bfin-uclinux-gcc
396c89 : /usr/bin/c89-gcc
397c99 : /usr/bin/c99-gcc
398frv : /toolchains/gcc-4.9.0-nolibc/frv-linux/bin/frv-linux-gcc
399h8300 : /toolchains/gcc-4.9.0-nolibc/h8300-elf/bin/h8300-elf-gcc
400hppa : /toolchains/gcc-4.9.0-nolibc/hppa-linux/bin/hppa-linux-gcc
401hppa64 : /toolchains/gcc-4.9.0-nolibc/hppa64-linux/bin/hppa64-linux-gcc
402i386 : /toolchains/gcc-4.9.0-nolibc/i386-linux/bin/i386-linux-gcc
403i586 : /usr/bin/i586-mingw32msvc-gcc
404ia64 : /toolchains/gcc-4.9.0-nolibc/ia64-linux/bin/ia64-linux-gcc
405m32r : /toolchains/gcc-4.9.0-nolibc/m32r-linux/bin/m32r-linux-gcc
406m68k : /toolchains/gcc-4.9.0-nolibc/m68k-linux/bin/m68k-linux-gcc
407microblaze: /toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc
408mips : /toolchains/gcc-4.9.0-nolibc/mips-linux/bin/mips-linux-gcc
409mips64 : /toolchains/gcc-4.9.0-nolibc/mips64-linux/bin/mips64-linux-gcc
410or32 : /toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc
411powerpc : /toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc
412powerpc64 : /toolchains/gcc-4.9.0-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc
413ppc64le : /toolchains/gcc-4.9.0-nolibc/ppc64le-linux/bin/ppc64le-linux-gcc
414s390x : /toolchains/gcc-4.9.0-nolibc/s390x-linux/bin/s390x-linux-gcc
415sandbox : /usr/bin/gcc
416sh4 : /toolchains/gcc-4.6.3-nolibc/sh4-linux/bin/sh4-linux-gcc
417sparc : /toolchains/gcc-4.9.0-nolibc/sparc-linux/bin/sparc-linux-gcc
418sparc64 : /toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc
419tilegx : /toolchains/gcc-4.6.2-nolibc/tilegx-linux/bin/tilegx-linux-gcc
420x86 : /opt/gcc-4.6.3-nolibc/x86_64-linux/bin/x86_64-linux-gcc
421x86_64 : /toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc
422
423
424You can see that everything is covered, even some strange ones that won't
425be used (c88 and c99). This is a feature.
426
427
4285. Install new toolchains if needed
429
430You can download toolchains and update the [toolchain] section of the
431settings file to find them.
432
433To make this easier, buildman can automatically download and install
434toolchains from kernel.org. First list the available architectures:
435
436$ ./tools/buildman/buildman --fetch-arch list
437Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.3/
438Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.2/
439Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1/
440Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.2.4/
441Available architectures: alpha am33_2.0 arm bfin cris crisv32 frv h8300
442hppa hppa64 i386 ia64 m32r m68k mips mips64 or32 powerpc powerpc64 s390x sh4
443sparc sparc64 tilegx x86_64 xtensa
444
445Then pick one and download it:
446
447$ ./tools/buildman/buildman --fetch-arch or32
448Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.3/
449Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.2/
450Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1/
451Downloading: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1//x86_64-gcc-4.5.1-nolibc_or32-linux.tar.xz
452Unpacking to: /home/sjg/.buildman-toolchains
453Testing
454 - looking in '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/.'
455 - looking in '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin'
456 - found '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc'
457Tool chain test: OK
458
459Or download them all from kernel.org and move them to /toolchains directory,
460
461$ ./tools/buildman/buildman --fetch-arch all
462$ sudo mkdir -p /toolchains
463$ sudo mv ~/.buildman-toolchains/*/* /toolchains/
464
465For those not available from kernel.org, download from the following links.
466
467arc: https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/releases/
468 download/arc-2016.09-release/arc_gnu_2016.09_prebuilt_uclibc_le_archs_linux_install.tar.gz
469blackfin: http://sourceforge.net/projects/adi-toolchain/files/
470 blackfin-toolchain-elf-gcc-4.5-2014R1_45-RC2.x86_64.tar.bz2
471nds32: http://osdk.andestech.com/packages/
472 nds32le-linux-glibc-v1.tgz
473nios2: http://sourcery.mentor.com/public/gnu_toolchain/nios2-linux-gnu/
474 sourceryg++-2015.11-27-nios2-linux-gnu-i686-pc-linux-gnu.tar.bz2
475sh: http://sourcery.mentor.com/public/gnu_toolchain/sh-linux-gnu/
476 renesas-4.4-200-sh-linux-gnu-i686-pc-linux-gnu.tar.bz2
477
478Note openrisc kernel.org toolchain is out of date. Download the latest one from
479http://opencores.org/or1k/OpenRISC_GNU_tool_chain#Prebuilt_versions - eg:
480ftp://ocuser:ocuser@openrisc.opencores.org/toolchain/gcc-or1k-elf-4.8.1-x86.tar.bz2.
481
482Buildman should now be set up to use your new toolchain.
483
484At the time of writing, U-Boot has these architectures:
485
486 arc, arm, blackfin, m68k, microblaze, mips, nds32, nios2, openrisc
487 powerpc, sandbox, sh, sparc, x86
488
489Of these, only arc and nds32 are not available at kernel.org..
490
491
492How to run it
493=============
494
495First do a dry run using the -n flag: (replace <branch> with a real, local
496branch with a valid upstream)
497
498$ ./tools/buildman/buildman -b <branch> -n
499
500If it can't detect the upstream branch, try checking out the branch, and
501doing something like 'git branch --set-upstream-to upstream/master'
502or something similar. Buildman will try to guess a suitable upstream branch
503if it can't find one (you will see a message like" Guessing upstream as ...).
504You can also use the -c option to manually specify the number of commits to
505build.
506
507As an example:
508
509Dry run, so not doing much. But I would do this:
510
511Building 18 commits for 1059 boards (4 threads, 1 job per thread)
512Build directory: ../lcd9b
513 5bb3505 Merge branch 'master' of git://git.denx.de/u-boot-arm
514 c18f1b4 tegra: Use const for pinmux_config_pingroup/table()
515 2f043ae tegra: Add display support to funcmux
516 e349900 tegra: fdt: Add pwm binding and node
517 424a5f0 tegra: fdt: Add LCD definitions for Tegra
518 0636ccf tegra: Add support for PWM
519 a994fe7 tegra: Add SOC support for display/lcd
520 fcd7350 tegra: Add LCD driver
521 4d46e9d tegra: Add LCD support to Nvidia boards
522 991bd48 arm: Add control over cachability of memory regions
523 54e8019 lcd: Add CONFIG_LCD_ALIGNMENT to select frame buffer alignment
524 d92aff7 lcd: Add support for flushing LCD fb from dcache after update
525 dbd0677 tegra: Align LCD frame buffer to section boundary
526 0cff9b8 tegra: Support control of cache settings for LCD
527 9c56900 tegra: fdt: Add LCD definitions for Seaboard
528 5cc29db lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console
529 cac5a23 tegra: Enable display/lcd support on Seaboard
530 49ff541 wip
531
532Total boards to build for each commit: 1059
533
534This shows that it will build all 1059 boards, using 4 threads (because
535we have a 4-core CPU). Each thread will run with -j1, meaning that each
536make job will use a single CPU. The list of commits to be built helps you
537confirm that things look about right. Notice that buildman has chosen a
538'base' directory for you, immediately above your source tree.
539
540Buildman works entirely inside the base directory, here ../lcd9b,
541creating a working directory for each thread, and creating output
542directories for each commit and board.
543
544
545Suggested Workflow
546==================
547
548To run the build for real, take off the -n:
549
550$ ./tools/buildman/buildman -b <branch>
551
552Buildman will set up some working directories, and get started. After a
553minute or so it will settle down to a steady pace, with a display like this:
554
555Building 18 commits for 1059 boards (4 threads, 1 job per thread)
556 528 36 124 /19062 -18374 1:13:30 : SIMPC8313_SP
557
558This means that it is building 19062 board/commit combinations. So far it
559has managed to successfully build 528. Another 36 have built with warnings,
560and 124 more didn't build at all. It has 18374 builds left to complete.
561Buildman expects to complete the process in around an hour and a quarter.
562Use this time to buy a faster computer.
563
564
565To find out how the build went, ask for a summary with -s. You can do this
566either before the build completes (presumably in another terminal) or
567afterwards. Let's work through an example of how this is used:
568
569$ ./tools/buildman/buildman -b lcd9b -s
570...
57101: Merge branch 'master' of git://git.denx.de/u-boot-arm
572 powerpc: + galaxy5200_LOWBOOT
57302: tegra: Use const for pinmux_config_pingroup/table()
57403: tegra: Add display support to funcmux
57504: tegra: fdt: Add pwm binding and node
57605: tegra: fdt: Add LCD definitions for Tegra
57706: tegra: Add support for PWM
57807: tegra: Add SOC support for display/lcd
57908: tegra: Add LCD driver
58009: tegra: Add LCD support to Nvidia boards
58110: arm: Add control over cachability of memory regions
58211: lcd: Add CONFIG_LCD_ALIGNMENT to select frame buffer alignment
58312: lcd: Add support for flushing LCD fb from dcache after update
584 arm: + lubbock
58513: tegra: Align LCD frame buffer to section boundary
58614: tegra: Support control of cache settings for LCD
58715: tegra: fdt: Add LCD definitions for Seaboard
58816: lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console
58917: tegra: Enable display/lcd support on Seaboard
59018: wip
591
592This shows which commits have succeeded and which have failed. In this case
593the build is still in progress so many boards are not built yet (use -u to
594see which ones). But already we can see a few failures. The galaxy5200_LOWBOOT
595never builds correctly. This could be a problem with our toolchain, or it
596could be a bug in the upstream. The good news is that we probably don't need
597to blame our commits. The bad news is that our commits are not tested on that
598board.
599
600Commit 12 broke lubbock. That's what the '+ lubbock', in red, means. The
601failure is never fixed by a later commit, or you would see lubbock again, in
602green, without the +.
603
604To see the actual error:
605
606$ ./tools/buildman/buildman -b <branch> -se
607...
60812: lcd: Add support for flushing LCD fb from dcache after update
609 arm: + lubbock
610+common/libcommon.o: In function `lcd_sync':
611+common/lcd.c:120: undefined reference to `flush_dcache_range'
612+arm-none-linux-gnueabi-ld: BFD (Sourcery G++ Lite 2010q1-202) 2.19.51.20090709 assertion fail /scratch/julian/2010q1-release-linux-lite/obj/binutils-src-2010q1-202-arm-none-linux-gnueabi-i686-pc-linux-gnu/bfd/elf32-arm.c:12572
613+make: *** [build/u-boot] Error 139
61413: tegra: Align LCD frame buffer to section boundary
61514: tegra: Support control of cache settings for LCD
61615: tegra: fdt: Add LCD definitions for Seaboard
61716: lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console
618-common/lcd.c:120: undefined reference to `flush_dcache_range'
619+common/lcd.c:125: undefined reference to `flush_dcache_range'
62017: tegra: Enable display/lcd support on Seaboard
62118: wip
622
623So the problem is in lcd.c, due to missing cache operations. This information
624should be enough to work out what that commit is doing to break these
625boards. (In this case pxa did not have cache operations defined).
626
627Note that if there were other boards with errors, the above command would
628show their errors also. Each line is shown only once. So if lubbock and snow
629produce the same error, we just see:
630
63112: lcd: Add support for flushing LCD fb from dcache after update
632 arm: + lubbock snow
633+common/libcommon.o: In function `lcd_sync':
634+common/lcd.c:120: undefined reference to `flush_dcache_range'
635+arm-none-linux-gnueabi-ld: BFD (Sourcery G++ Lite 2010q1-202) 2.19.51.20090709 assertion fail /scratch/julian/2010q1-release-linux-lite/obj/binutils-src-2010q1-202-arm-none-linux-gnueabi-i686-pc-linux-gnu/bfd/elf32-arm.c:12572
636+make: *** [build/u-boot] Error 139
637
638But if you did want to see just the errors for lubbock, use:
639
640$ ./tools/buildman/buildman -b <branch> -se lubbock
641
642If you see error lines marked with '-', that means that the errors were fixed
643by that commit. Sometimes commits can be in the wrong order, so that a
644breakage is introduced for a few commits and fixed by later commits. This
645shows up clearly with buildman. You can then reorder the commits and try
646again.
647
648At commit 16, the error moves: you can see that the old error at line 120
649is fixed, but there is a new one at line 126. This is probably only because
650we added some code and moved the broken line further down the file.
651
652As mentioned, if many boards have the same error, then -e will display the
653error only once. This makes the output as concise as possible. To see which
654boards have each error, use -l. So it is safe to omit the board name - you
655will not get lots of repeated output for every board.
656
657Buildman tries to distinguish warnings from errors, and shows warning lines
658separately with a 'w' prefix. Warnings introduced show as yellow. Warnings
659fixed show as cyan.
660
661The full build output in this case is available in:
662
663../lcd9b/12_of_18_gd92aff7_lcd--Add-support-for/lubbock/
664
665 done: Indicates the build was done, and holds the return code from make.
666 This is 0 for a good build, typically 2 for a failure.
667
668 err: Output from stderr, if any. Errors and warnings appear here.
669
670 log: Output from stdout. Normally there isn't any since buildman runs
671 in silent mode. Use -V to force a verbose build (this passes V=1
672 to 'make')
673
674 toolchain: Shows information about the toolchain used for the build.
675
676 sizes: Shows image size information.
677
678It is possible to get the build binary output there also. Use the -k option
679for this. In that case you will also see some output files, like:
680
681 System.map toolchain u-boot u-boot.bin u-boot.map autoconf.mk
682 (also SPL versions u-boot-spl and u-boot-spl.bin if available)
683
684
685Checking Image Sizes
686====================
687
688A key requirement for U-Boot is that you keep code/data size to a minimum.
689Where a new feature increases this noticeably it should normally be put
690behind a CONFIG flag so that boards can leave it disabled and keep the image
691size more or less the same with each new release.
692
693To check the impact of your commits on image size, use -S. For example:
694
695$ ./tools/buildman/buildman -b us-x86 -sS
696Summary of 10 commits for 1066 boards (4 threads, 1 job per thread)
69701: MAKEALL: add support for per architecture toolchains
69802: x86: Add function to get top of usable ram
699 x86: (for 1/3 boards) text -272.0 rodata +41.0
70003: x86: Add basic cache operations
70104: x86: Permit bootstage and timer data to be used prior to relocation
702 x86: (for 1/3 boards) data +16.0
70305: x86: Add an __end symbol to signal the end of the U-Boot binary
704 x86: (for 1/3 boards) text +76.0
70506: x86: Rearrange the output input to remove BSS
706 x86: (for 1/3 boards) bss -2140.0
70707: x86: Support relocation of FDT on start-up
708 x86: + coreboot-x86
70908: x86: Add error checking to x86 relocation code
71009: x86: Adjust link device tree include file
71110: x86: Enable CONFIG_OF_CONTROL on coreboot
712
713
714You can see that image size only changed on x86, which is good because this
715series is not supposed to change any other board. From commit 7 onwards the
716build fails so we don't get code size numbers. The numbers are fractional
717because they are an average of all boards for that architecture. The
718intention is to allow you to quickly find image size problems introduced by
719your commits.
720
721Note that the 'text' region and 'rodata' are split out. You should add the
722two together to get the total read-only size (reported as the first column
723in the output from binutil's 'size' utility).
724
725A useful option is --step which lets you skip some commits. For example
726--step 2 will show the image sizes for only every 2nd commit (so it will
727compare the image sizes of the 1st, 3rd, 5th... commits). You can also use
728--step 0 which will compare only the first and last commits. This is useful
729for an overview of how your entire series affects code size. It will build
730only the upstream commit and your final branch commit.
731
732You can also use -d to see a detailed size breakdown for each board. This
733list is sorted in order from largest growth to largest reduction.
734
735It is even possible to go a little further with the -B option (--bloat). This
736shows where U-Boot has bloated, breaking the size change down to the function
737level. Example output is below:
738
739$ ./tools/buildman/buildman -b us-mem4 -sSdB
740...
74119: Roll crc32 into hash infrastructure
742 arm: (for 10/10 boards) all -143.4 bss +1.2 data -4.8 rodata -48.2 text -91.6
743 paz00 : all +23 bss -4 rodata -29 text +56
744 u-boot: add: 1/0, grow: 3/-2 bytes: 168/-104 (64)
745 function old new delta
746 hash_command 80 160 +80
747 crc32_wd_buf - 56 +56
748 ext4fs_read_file 540 568 +28
749 insert_var_value_sub 688 692 +4
750 run_list_real 1996 1992 -4
751 do_mem_crc 168 68 -100
752 trimslice : all -9 bss +16 rodata -29 text +4
753 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12)
754 function old new delta
755 hash_command 80 160 +80
756 crc32_wd_buf - 56 +56
757 ext4fs_iterate_dir 672 668 -4
758 ext4fs_read_file 568 548 -20
759 do_mem_crc 168 68 -100
760 whistler : all -9 bss +16 rodata -29 text +4
761 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12)
762 function old new delta
763 hash_command 80 160 +80
764 crc32_wd_buf - 56 +56
765 ext4fs_iterate_dir 672 668 -4
766 ext4fs_read_file 568 548 -20
767 do_mem_crc 168 68 -100
768 seaboard : all -9 bss -28 rodata -29 text +48
769 u-boot: add: 1/0, grow: 3/-2 bytes: 160/-104 (56)
770 function old new delta
771 hash_command 80 160 +80
772 crc32_wd_buf - 56 +56
773 ext4fs_read_file 548 568 +20
774 run_list_real 1996 2000 +4
775 do_nandboot 760 756 -4
776 do_mem_crc 168 68 -100
777 colibri_t20 : all -9 rodata -29 text +20
778 u-boot: add: 1/0, grow: 2/-3 bytes: 140/-112 (28)
779 function old new delta
780 hash_command 80 160 +80
781 crc32_wd_buf - 56 +56
782 read_abs_bbt 204 208 +4
783 do_nandboot 760 756 -4
784 ext4fs_read_file 576 568 -8
785 do_mem_crc 168 68 -100
786 ventana : all -37 bss -12 rodata -29 text +4
787 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12)
788 function old new delta
789 hash_command 80 160 +80
790 crc32_wd_buf - 56 +56
791 ext4fs_iterate_dir 672 668 -4
792 ext4fs_read_file 568 548 -20
793 do_mem_crc 168 68 -100
794 harmony : all -37 bss -16 rodata -29 text +8
795 u-boot: add: 1/0, grow: 2/-3 bytes: 140/-124 (16)
796 function old new delta
797 hash_command 80 160 +80
798 crc32_wd_buf - 56 +56
799 nand_write_oob_syndrome 428 432 +4
800 ext4fs_iterate_dir 672 668 -4
801 ext4fs_read_file 568 548 -20
802 do_mem_crc 168 68 -100
803 medcom-wide : all -417 bss +28 data -16 rodata -93 text -336
804 u-boot: add: 1/-1, grow: 1/-2 bytes: 88/-376 (-288)
805 function old new delta
806 crc32_wd_buf - 56 +56
807 do_fat_read_at 2872 2904 +32
808 hash_algo 16 - -16
809 do_mem_crc 168 68 -100
810 hash_command 420 160 -260
811 tec : all -449 bss -4 data -16 rodata -93 text -336
812 u-boot: add: 1/-1, grow: 1/-2 bytes: 88/-376 (-288)
813 function old new delta
814 crc32_wd_buf - 56 +56
815 do_fat_read_at 2872 2904 +32
816 hash_algo 16 - -16
817 do_mem_crc 168 68 -100
818 hash_command 420 160 -260
819 plutux : all -481 bss +16 data -16 rodata -93 text -388
820 u-boot: add: 1/-1, grow: 1/-3 bytes: 68/-408 (-340)
821 function old new delta
822 crc32_wd_buf - 56 +56
823 do_load_serial_bin 1688 1700 +12
824 hash_algo 16 - -16
825 do_fat_read_at 2904 2872 -32
826 do_mem_crc 168 68 -100
827 hash_command 420 160 -260
828 powerpc: (for 5/5 boards) all +37.4 data -3.2 rodata -41.8 text +82.4
829 MPC8610HPCD : all +55 rodata -29 text +84
830 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
831 function old new delta
832 hash_command - 176 +176
833 do_mem_crc 184 88 -96
834 MPC8641HPCN : all +55 rodata -29 text +84
835 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
836 function old new delta
837 hash_command - 176 +176
838 do_mem_crc 184 88 -96
839 MPC8641HPCN_36BIT: all +55 rodata -29 text +84
840 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
841 function old new delta
842 hash_command - 176 +176
843 do_mem_crc 184 88 -96
844 sbc8641d : all +55 rodata -29 text +84
845 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
846 function old new delta
847 hash_command - 176 +176
848 do_mem_crc 184 88 -96
849 xpedite517x : all -33 data -16 rodata -93 text +76
850 u-boot: add: 1/-1, grow: 0/-1 bytes: 176/-112 (64)
851 function old new delta
852 hash_command - 176 +176
853 hash_algo 16 - -16
854 do_mem_crc 184 88 -96
855...
856
857
858This shows that commit 19 has reduced codesize for arm slightly and increased
859it for powerpc. This increase was offset in by reductions in rodata and
860data/bss.
861
862Shown below the summary lines are the sizes for each board. Below each board
863are the sizes for each function. This information starts with:
864
865 add - number of functions added / removed
866 grow - number of functions which grew / shrunk
867 bytes - number of bytes of code added to / removed from all functions,
868 plus the total byte change in brackets
869
870The change seems to be that hash_command() has increased by more than the
871do_mem_crc() function has decreased. The function sizes typically add up to
872roughly the text area size, but note that every read-only section except
873rodata is included in 'text', so the function total does not exactly
874correspond.
875
876It is common when refactoring code for the rodata to decrease as the text size
877increases, and vice versa.
878
879
880The .buildman file
881==================
882
883The .buildman file provides information about the available toolchains and
884also allows build flags to be passed to 'make'. It consists of several
885sections, with the section name in square brackets. Within each section are
886a set of (tag, value) pairs.
887
888'[toolchain]' section
889
890 This lists the available toolchains. The tag here doesn't matter, but
891 make sure it is unique. The value is the path to the toolchain. Buildman
892 will look in that path for a file ending in 'gcc'. It will then execute
893 it to check that it is a C compiler, passing only the --version flag to
894 it. If the return code is 0, buildman assumes that it is a valid C
895 compiler. It uses the first part of the name as the architecture and
896 strips off the last part when setting the CROSS_COMPILE environment
897 variable (parts are delimited with a hyphen).
898
899 For example powerpc-linux-gcc will be noted as a toolchain for 'powerpc'
900 and CROSS_COMPILE will be set to powerpc-linux- when using it.
901
902'[toolchain-alias]' section
903
904 This converts toolchain architecture names to U-Boot names. For example,
905 if an x86 toolchains is called i386-linux-gcc it will not normally be
906 used for architecture 'x86'. Adding 'x86: i386 x86_64' to this section
907 will tell buildman that the i386 and x86_64 toolchains can be used for
908 the x86 architecture.
909
910'[make-flags]' section
911
912 U-Boot's build system supports a few flags (such as BUILD_TAG) which
913 affect the build product. These flags can be specified in the buildman
914 settings file. They can also be useful when building U-Boot against other
915 open source software.
916
917 [make-flags]
918 at91-boards=ENABLE_AT91_TEST=1
919 snapper9260=${at91-boards} BUILD_TAG=442
920 snapper9g45=${at91-boards} BUILD_TAG=443
921
922 This will use 'make ENABLE_AT91_TEST=1 BUILD_TAG=442' for snapper9260
923 and 'make ENABLE_AT91_TEST=1 BUILD_TAG=443' for snapper9g45. A special
924 variable ${target} is available to access the target name (snapper9260
925 and snapper9g20 in this case). Variables are resolved recursively. Note
926 that variables can only contain the characters A-Z, a-z, 0-9, hyphen (-)
927 and underscore (_).
928
929 It is expected that any variables added are dealt with in U-Boot's
930 config.mk file and documented in the README.
931
932 Note that you can pass ad-hoc options to the build using environment
933 variables, for example:
934
935 SOME_OPTION=1234 ./tools/buildman/buildman my_board
936
937
938Quick Sanity Check
939==================
940
941If you have made changes and want to do a quick sanity check of the
942currently checked-out source, run buildman without the -b flag. This will
943build the selected boards and display build status as it runs (i.e. -v is
944enabled automatically). Use -e to see errors/warnings as well.
945
946
947Building Ranges
948===============
949
950You can build a range of commits by specifying a range instead of a branch
951when using the -b flag. For example:
952
953 upstream/master..us-buildman
954
955will build commits in us-buildman that are not in upstream/master.
956
957
958Building Faster
959===============
960
961By default, buildman doesn't execute 'make mrproper' prior to building the
962first commit for each board. This reduces the amount of work 'make' does, and
963hence speeds up the build. To force use of 'make mrproper', use -the -m flag.
964This flag will slow down any buildman invocation, since it increases the amount
965of work done on any build.
966
967One possible application of buildman is as part of a continual edit, build,
968edit, build, ... cycle; repeatedly applying buildman to the same change or
969series of changes while making small incremental modifications to the source
970each time. This provides quick feedback regarding the correctness of recent
971modifications. In this scenario, buildman's default choice of build directory
972causes more build work to be performed than strictly necessary.
973
974By default, each buildman thread uses a single directory for all builds. When a
975thread builds multiple boards, the configuration built in this directory will
976cycle through various different configurations, one per board built by the
977thread. Variations in the configuration will force a rebuild of affected source
978files when a thread switches between boards. Ideally, such buildman-induced
979rebuilds would not happen, thus allowing the build to operate as efficiently as
980the build system and source changes allow. buildman's -P flag may be used to
981enable this; -P causes each board to be built in a separate (board-specific)
982directory, thus avoiding any buildman-induced configuration changes in any
983build directory.
984
985U-Boot's build system embeds information such as a build timestamp into the
986final binary. This information varies each time U-Boot is built. This causes
987various files to be rebuilt even if no source changes are made, which in turn
988requires that the final U-Boot binary be re-linked. This unnecessary work can
989be avoided by turning off the timestamp feature. This can be achieved by
990setting the SOURCE_DATE_EPOCH environment variable to 0.
991
992Combining all of these options together yields the command-line shown below.
993This will provide the quickest possible feedback regarding the current content
994of the source tree, thus allowing rapid tested evolution of the code.
995
996 SOURCE_DATE_EPOCH=0 ./tools/buildman/buildman -P tegra
997
998
999Checking configuration
1000======================
1001
1002A common requirement when converting CONFIG options to Kconfig is to check
1003that the effective configuration has not changed due to the conversion.
1004Buildman supports this with the -K option, used after a build. This shows
1005differences in effective configuration between one commit and the next.
1006
1007For example:
1008
1009 $ buildman -b kc4 -sK
1010 ...
1011 43: Convert CONFIG_SPL_USBETH_SUPPORT to Kconfig
1012 arm:
1013 + u-boot.cfg: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_NET_SUPPORT=1
1014 + u-boot-spl.cfg: CONFIG_SPL_MMC_SUPPORT=1 CONFIG_SPL_NAND_SUPPORT=1
1015 + all: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_MMC_SUPPORT=1 CONFIG_SPL_NAND_SUPPORT=1 CONFIG_SPL_NET_SUPPORT=1
1016 am335x_evm_usbspl :
1017 + u-boot.cfg: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_NET_SUPPORT=1
1018 + u-boot-spl.cfg: CONFIG_SPL_MMC_SUPPORT=1 CONFIG_SPL_NAND_SUPPORT=1
1019 + all: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_MMC_SUPPORT=1 CONFIG_SPL_NAND_SUPPORT=1 CONFIG_SPL_NET_SUPPORT=1
1020 44: Convert CONFIG_SPL_USB_HOST_SUPPORT to Kconfig
1021 ...
1022
1023This shows that commit 44 enabled three new options for the board
1024am335x_evm_usbspl which were not enabled in commit 43. There is also a
1025summary for 'arm' showing all the changes detected for that architecture.
1026In this case there is only one board with changes, so 'arm' output is the
1027same as 'am335x_evm_usbspl'/
1028
1029The -K option uses the u-boot.cfg, spl/u-boot-spl.cfg and tpl/u-boot-tpl.cfg
1030files which are produced by a build. If all you want is to check the
1031configuration you can in fact avoid doing a full build, using -D. This tells
1032buildman to configuration U-Boot and create the .cfg files, but not actually
1033build the source. This is 5-10 times faster than doing a full build.
1034
1035By default buildman considers the follow two configuration methods
1036equivalent:
1037
1038 #define CONFIG_SOME_OPTION
1039
1040 CONFIG_SOME_OPTION=y
1041
1042The former would appear in a header filer and the latter in a defconfig
1043file. The achieve this, buildman considers 'y' to be '1' in configuration
1044variables. This avoids lots of useless output when converting a CONFIG
1045option to Kconfig. To disable this behaviour, use --squash-config-y.
1046
1047
1048Checking the environment
1049========================
1050
1051When converting CONFIG options which manipulate the default environment,
1052a common requirement is to check that the default environment has not
1053changed due to the conversion. Buildman supports this with the -U option,
1054used after a build. This shows differences in the default environment
1055between one commit and the next.
1056
1057For example:
1058
1059$ buildman -b squash brppt1 -sU
1060boards.cfg is up to date. Nothing to do.
1061Summary of 2 commits for 3 boards (3 threads, 3 jobs per thread)
106201: Migrate bootlimit to Kconfig
106302: Squashed commit of the following:
1064 c brppt1_mmc: altbootcmd=mmc dev 1; run mmcboot0; -> mmc dev 1; run mmcboot0
1065 c brppt1_spi: altbootcmd=mmc dev 1; run mmcboot0; -> mmc dev 1; run mmcboot0
1066 + brppt1_nand: altbootcmd=run usbscript
1067 - brppt1_nand: altbootcmd=run usbscript
1068(no errors to report)
1069
1070This shows that commit 2 modified the value of 'altbootcmd' for 'brppt1_mmc'
1071and 'brppt1_spi', removing a trailing semicolon. 'brppt1_nand' gained an a
1072value for 'altbootcmd', but lost one for ' altbootcmd'.
1073
1074The -U option uses the u-boot.env files which are produced by a build.
1075
1076
1077Building with clang
1078===================
1079
1080To build with clang (sandbox only), use the -O option to override the
1081toolchain. For example:
1082
1083 buildman -O clang-7 --board sandbox
1084
1085
1086Doing a simple build
1087====================
1088
1089In some cases you just want to build a single board and get the full output, use
1090the -w option, for example:
1091
1092 buildman -o /tmp/build --board sandbox -w
1093
1094This will write the full build into /tmp/build including object files. You must
1095specify the output directory with -o when using -w.
1096
1097
1098Other options
1099=============
1100
1101Buildman has various other command-line options. Try --help to see them.
1102
1103To find out what toolchain prefix buildman will use for a build, use the -A
1104option.
1105
1106To request that compiler warnings be promoted to errors, use -E. This passes the
1107-Werror flag to the compiler. Note that the build can still produce warnings
1108with -E, e.g. the migration warnings:
1109
1110 ===================== WARNING ======================
1111 This board does not use CONFIG_DM_MMC. Please update
1112 ...
1113 ====================================================
1114
1115When doing builds, Buildman's return code will reflect the overall result:
1116
1117 0 (success) No errors or warnings found
1118 100 Errors found
1119 101 Warnings found (only if no -W)
1120
1121You can use -W to tell Buildman to return 0 (success) instead of 101 when
1122warnings are found. Note that it can be useful to combine -E and -W. This means
1123that all compiler warnings will produce failures (code 100) and all other
1124warnings will produce success (since 101 is changed to 0).
1125
1126If there are both warnings and errors, errors win, so buildman returns 100.
1127
1128The -y option is provided (for use with -s) to ignore the bountiful device-tree
1129warnings. Similarly, -Y tells buildman to ignore the migration warnings.
1130
1131Sometimes you might get an error in a thread that is not handled by buildman,
1132perhaps due to a failure of a tool that it calls. You might see the output, but
1133then buildman hangs. Failing to handle any eventuality is a bug in buildman and
1134should be reported. But you can use -T0 to disable threading and hopefully
1135figure out the root cause of the build failure.
1136
1137Build summary
1138=============
1139
1140When buildman finishes it shows a summary, something like this:
1141
1142 Completed: 5 total built, duration 0:00:21, rate 0.24
1143
1144This shows that a total of 5 builds were done across all selected boards, it
1145took 21 seconds and the builds happened at the rate of 0.24 per second. The
1146latter number depends on the speed of your machine and the efficiency of the
1147U-Boot build.
1148
1149
1150How to change from MAKEALL
1151==========================
1152
1153Buildman includes most of the features of MAKEALL and is generally faster
1154and easier to use. In particular it builds entire branches: if a particular
1155commit introduces an error in a particular board, buildman can easily show
1156you this, even if a later commit fixes that error.
1157
1158The reasons to deprecate MAKEALL are:
1159- We don't want to maintain two build systems
1160- Buildman is typically faster
1161- Buildman has a lot more features
1162
1163But still, many people will be sad to lose MAKEALL. If you are used to
1164MAKEALL, here are a few pointers.
1165
1166First you need to set up your tool chains - see the 'Setting up' section
1167for details. Once you have your required toolchain(s) detected then you are
1168ready to go.
1169
1170To build the current source tree, run buildman without a -b flag:
1171
1172 ./tools/buildman/buildman <list of things to build>
1173
1174This will build the current source tree for the given boards and display
1175the results and errors.
1176
1177However buildman usually works on entire branches, and for that you must
1178specify a board flag:
1179
1180 ./tools/buildman/buildman -b <branch_name> <list of things to build>
1181
1182followed by (afterwards, or perhaps concurrently in another terminal):
1183
1184 ./tools/buildman/buildman -b <branch_name> -s <list of things to build>
1185
1186to see the results of the build. Rather than showing you all the output,
1187buildman just shows a summary, with red indicating that a commit introduced
1188an error and green indicating that a commit fixed an error. Use the -e
1189flag to see the full errors and -l to see which boards caused which errors.
1190
1191If you really want to see build results as they happen, use -v when doing a
1192build (and -e to see the errors/warnings too).
1193
1194You don't need to stick around on that branch while buildman is running. It
1195checks out its own copy of the source code, so you can change branches,
1196add commits, etc. without affecting the build in progress.
1197
1198The <list of things to build> can include board names, architectures or the
1199like. There are no flags to disambiguate since ambiguities are rare. Using
1200the examples from MAKEALL:
1201
1202Examples:
1203 - build all Power Architecture boards:
1204 MAKEALL -a powerpc
1205 MAKEALL --arch powerpc
1206 MAKEALL powerpc
1207 ** buildman -b <branch> powerpc
1208 - build all PowerPC boards manufactured by vendor "esd":
1209 MAKEALL -a powerpc -v esd
1210 ** buildman -b <branch> esd
1211 - build all PowerPC boards manufactured either by "keymile" or "siemens":
1212 MAKEALL -a powerpc -v keymile -v siemens
1213 ** buildman -b <branch> keymile siemens
1214 - build all Freescale boards with MPC83xx CPUs, plus all 4xx boards:
1215 MAKEALL -c mpc83xx -v freescale 4xx
1216 ** buildman -b <branch> mpc83xx freescale 4xx
1217
1218Buildman automatically tries to use all the CPUs in your machine. If you
1219are building a lot of boards it will use one thread for every CPU core
1220it detects in your machine. This is like MAKEALL's BUILD_NBUILDS option.
1221You can use the -T flag to change the number of threads. If you are only
1222building a few boards, buildman will automatically run make with the -j
1223flag to increase the number of concurrent make tasks. It isn't normally
1224that helpful to fiddle with this option, but if you use the BUILD_NCPUS
1225option in MAKEALL then -j is the equivalent in buildman.
1226
1227Buildman puts its output in ../<branch_name> by default but you can change
1228this with the -o option. Buildman normally does out-of-tree builds: use -i
1229to disable that if you really want to. But be careful that once you have
1230used -i you pollute buildman's copies of the source tree, and you will need
1231to remove the build directory (normally ../<branch_name>) to run buildman
1232in normal mode (without -i).
1233
1234Buildman doesn't keep the output result normally, but use the -k option to
1235do this.
1236
1237Please read 'Theory of Operation' a few times as it will make a lot of
1238things clearer.
1239
1240Some options you might like are:
1241
1242 -B shows which functions are growing/shrinking in which commit - great
1243 for finding code bloat.
1244 -S shows image sizes for each commit (just an overall summary)
1245 -u shows boards that you haven't built yet
1246 --step 0 will build just the upstream commit and the last commit of your
1247 branch. This is often a quick sanity check that your branch doesn't
1248 break anything. But note this does not check bisectability!
1249
1250
1251TODO
1252====
1253
1254Many improvements have been made over the years. There is still quite a bit of
1255scope for more though, e.g.:
1256
1257- easier access to log files
1258- 'hunting' for problems, perhaps by building a few boards for each arch, or
1259 checking commits for changed files and building only boards which use those
1260 files
1261- using the same git repo for all threads instead of cloning it. Currently
1262 it uses about 500MB per thread, so on a 64-thread machine this is 32GB for
1263 the build.
1264
1265
1266Credits
1267=======
1268
1269Thanks to Grant Grundler <grundler@chromium.org> for his ideas for improving
1270the build speed by building all commits for a board instead of the other
1271way around.
1272
1273
1274Simon Glass
1275sjg@chromium.org
1276Halloween 2012
1277Updated 12-12-12
1278Updated 23-02-13
1279Updated 09-04-20
1280