[build2] LTO and parallelization with build2

[Matthew Krupcale]

On Wed, Aug 12, 2020 at 8:14 AM Boris Kolpackov <boris at codesynthesis.com> wrote:
>
> Merged to master. Your first contribution to build2, congrats & thanks!

Awesome--fortunately, it was fairly simple with your advice :). Did
you get a chance to test on a larger machine?

> I wonder what's next... ;-)

Good question lol. I do have a few ideas:

1. It might make sense to implement the find_option{,_prefix}
functions taking {c,}strings in terms of the new iterator variants and
the compare_option{,_prefix} functions you wrote.
2. Investigate the use of BLAKE3 hash for file checksums. BLAKE3[1] is
significantly faster than SHA-1 and SHA-2 (5-10 times) and is highly
parallelizable since it uses Merkle trees internally. This could
utilize the new scheduler thread allocator, but even without
parallelization, it's much faster. For small inputs, this may not
matter much, but for many, large TUs or object file checksums, this
might be noticeable, especially if solution 1 of [2] were implemented.
3. Write a Fortran language build system module. This would likely
need a lot of similar machinery in the cc module as well as some of
the cxx module dependency scanning logic to handle Fortran modules.
Fortran compilers though don't have a protocol for communication
between buildsystem and compiler like C++ modules for module name-file
mapping. Instead, compiled Fortran module interface files are named
according to the (lowercase) module name and searched for in the -I,
-J, and current directories (at least that's what gfortran seems to
do). So we just need to find the module source file and compile that
before any module "use"s it, and gfortran should find it. gfortran can
use the C preprocessor (in traditional mode), but it's not invoked by
default unless the file extension is .fpp or is like ".F*", and files
can be textually included using either "include" statements or
"#include" directives (when cpp is invoked).

[1] https://github.com/BLAKE3-team/BLAKE3
[2] https://github.com/build2/build2/issues/87