[build2] LTO and parallelization with build2
boris at codesynthesis.com
Fri Aug 7 08:45:51 UTC 2020
Matthew Krupcale <mkrupcale at matthewkrupcale.com> writes:
> [...] (by default, m=32) [...]
> Based on my reading of how the partitioning works, as long as
> lto-partitons=m > -flto=n, I think it should be possible for the
> linker to fully utilize all given threads. This might require some
> experimentation and verification, though.
CPUs with more than 32 hardware threads are not that uncommon these
> With the current understanding of the partitioning, I was also
> wondering if -flto=auto might be as likely to OOM as we previously
> thought (i.e. with current build2 without changes).
> This of course has some assumptions about the cumulative memory of the
> m partitioned threads being the same as 1 thread doing the full link,
> but maybe it's not far off.
Could be. There is at least some fixed memory cost for each process
In any case, I've added the scheduler API to allocate extra active
threads. What's left is to translate -flto=auto. Would you like to take
a stab at it? I could give you some pointers if you are interested.
(BTW, there is now etc/bootstrap script for setting up the build2
development environment. It might still have some rough edges but we
find it pretty usable.)
More information about the users