[build2] LTO and parallelization with build2

Matthew Krupcale mkrupcale at matthewkrupcale.com
Sat Aug 1 16:45:09 UTC 2020


Hello,

Fedora is looking to enable LTO by default in F33[1], and recently
they decided to use -flto=auto for the default GCC LTO build flags[2].
This means[3] that GCC will attempt to use the GNU make jobserver, if
available, but otherwise fall back to using the number of CPU threads
on the system. Since build2 has its own scheduler, though, this means
there might be e.g. up to n^2 threads on an n CPU thread system during
linking, which could negatively impact performance / waste CPU time.

So when building with -flto={auto,n} using GCC, build2 may either need
to instead invoke the compiler/linker with -flto=m, where m is the
number of available threads from the scheduler (I'm not sure the
scheduler currently supports something like this), or just disable
parallel linking with -flto[=1]. Alternatively, build2 could attempt
to act as a GNU make jobserver itself somehow. This also might become
relevant during compilation as GCC looks to parallelize
compilation[4,5].

I wonder if you've given any thought on how best to handle this. As
far as I can tell, ninja hasn't really addressed this either, but it
looks like they're considering making ninja work as a GNU make
jobserver client[6,7], and there's a recent PR to make ninja work with
multiple simultaneous ninja processes, all sharing a job limit[8].

Best,
Matthew

[1] https://fedoraproject.org/wiki/LTOByDefault
[2] https://src.fedoraproject.org/rpms/redhat-rpm-config/c/4637e1bd5512b869bd07c58b7545d2528a9bc4c8?branch=master
[3] https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#Optimize-Options
[4] https://gcc.gnu.org/wiki/ParallelGcc
[5] https://www.phoronix.com/scan.php?page=news_item&px=GCC-2020-More-Parallel-Large-S
[6] https://github.com/ninja-build/ninja/issues/1139#issuecomment-520546470
[7] https://github.com/ninja-build/ninja/pull/1140
[8] https://github.com/ninja-build/ninja/pull/1815



More information about the users mailing list