[build2] Build2 configuration for component based projects

Boris Kolpackov boris at codesynthesis.com
Tue Nov 14 13:45:39 UTC 2017

asatsuyu at tutanota.com <asatsuyu at tutanota.com> writes:

> My main concern is to keep the config files for out-of-tree builds at
> hand to have them under version control. In particular, launching a
> build for a particular config should remain a straight forward
> operation (one simple command line ideally) after checking out the
> project from a git repo.

This is definitely not how we roll since we assume different users
may want slightly/significantly different configurations (say I am
on Debian using g++-7 while you are on Fedora using g++-6). But I
see how this can work in a more tightly controlled environment where
you use RedHat Linux with g++-4.2 or you are fired ;-).

So I think you will need to play with this and see what feels right.
For example, you can create an amalgamation (see the create command
in the b(1) man page) and place it in a repository. Then you just
clone that repository and configure your project in it which will
inherit the configuration from the amalgamation. The nice thing
about this approach is that you can still override some settings
during the configuration step and you can use the same amalgamation
to build multiple projects (we do that a lot in our setup).

> When using the version of build2 that you indicated, things are
> working fine as long as the content of a given implementation file
> (e.g. cde/cde_XXX.cpp) remains unused outside of its test driver
> (e.g. cde/cde_XXX-t.cpp).

Well, to be fair, you never mentioned this requirement in your original

> To illustrate a situation where things do not seem to work, imagine
> that the test driver of the component efg_ZZZ (file efg/efg_ZZZ-t.cpp)
> calls a function "void foo()" declared in file "cde/cde_XXX.hpp" and
> implemented in file "cde/cde_XXX.cpp". In that case, a link time error
> that looks like the following is generated:
> g++ -o bld/cfg_A/efg/efg_ZZZ bld/cfg_A/efg/efg_ZZZ.o bld/cfg_A/efg/efg_ZZZ-t.o
> g++: error: undefined symbol: foo()
> I am not sure to see a satisfying workaround at this point. In particular,
> the following ideas look either inefficient or impractical:
> - Packaging all the object files into one big library. This would mean
>   relinking every test driver of the project at build time after a
>   change in any of the header or implementation file. The amount of
>   superfluous work would be significant.
>   I am not sure that it makes this idea much better but at least in the
>   case of a shared library, there should be no need to relink every test
>   driver of the project if the change is only affecting an implementation
>   file (e.g. cde_XXX.cpp).

With the current state of tooling it is not easy to detect if the shared
library changes involve only the "interface" or also the "implementation".
Even these two terms are not well defined for shared libraries.

> - Packaging all the object files within a given directory (e.g. "src/cde/")
>   into a library (e.g. libcde). This would slightly reduce (but not
>   suppress) the amount of superfluous work at build time at the expense
>   of manually specifying and maintaining the dependencies between those
>   libraries.
> Do you see anything that could help solving this issue?

Yes, these are pretty much the alternatives. A build system could
theoretically have a mode where it would detect that a source file
includes header cde/cde_XXX.hpp, assume that it will also need
cde/cde_XXX.cpp (which, BTW, could very well be an incorrect
assumption -- think of inline functions), and then automatically
link the corresponding object file. While I remember hearing someone
doing it, this approach does feel pretty complex and ad hoc with
quite a few assumptions.

> 4) Minimizing the number of tests to run:
> This item might worth a separate email thread, just let me know
> if you like me to start one. When issuing a command like
> "build2 '{test}(bld/cfg_A)'", every single test available seem
> to be run on every invocation. That's fine from the end user
> perspective who is expected to run the test suite at most once
> before the installation.
> From the developer perspective though it could be helpful to have
> a way to run only the test drivers that were either updated since
> the last "build2 test" command or unsuccessful during the last run.
> I realize that this is not ideal because a test does not only depend
> on the associated binary but also (at least) on the associated
> "testscript" file and the files eventually referenced in this
> "testscript" file.
> That being said, it would be nice to find a simple solution that
> covers a common use case where the test depends only on its
> associated binary and its associated "testscript" file. Would
> this be an acceptable solution to identify such kind of tests
> in the the buildfile using a special option and to have them
> processed accordingly by build2? The default behavior would
> remain the current one if the option is not set for a given
> test.

I agree it would be nice but again all the possible solutions
that come to mind seem very magicky. But let's see:

Tests that are executed via testscript have a working directory
created for them. For successful tests this directory is
automatically cleaned up but that can be suppressed (see
config.test.output). So we could store some state in there.
The biggest issue, as you have identified, is making sure
that we keep track of everything that affects the test. We
can handle the testscript (including everything that it
includes automatically) automatically. As well as the target
being tests (by just saving its timestamp). And for the rest
we can require that you explicitly list all other dependencies
as prerequisites of the target being tested (which you probably
already do anyway if you want the dist operation to work). Yeah,
maybe this can work.

BTW, not sure if you are aware, but you can already run individual
testscript tests manually. See the description of the test module
in build system manual for details.


More information about the users mailing list