[build2] Build system reorganization (heads-up to distribution packagers)

[Boris Kolpackov]

Matthew Krupcale <mkrupcale at matthewkrupcale.com> writes:

> Do you have a list of modules which are considered required by
> libbuild2? Or would you consider all of the bundled modules required
> by libbuild2 and as such I should not make sub-packages for the
> modules?

Modules that are bundled with `libbuild2` (in the `build2` package)
are fairly essential and the build system driver links them rather
than loading them dynamically. So I don't think it make much sense
to split them into separate distribution packages.


> I'm wondering if this may become too complicated, especially with
> platform-specific library versioning also in the mix. How will these
> two filename-based versioning systems work together?

In the future we can use the platform-specific versioning for one
of them, most likely the actual library version rather that the
build system core version. So it will be:

libbuild2-hello-0.12.so.0.1

I don't think there will be any clash with the "platform's way of 
doing things" since the dynamic linker cares about the dynamic
linking rather than loading (via dlopen()). It's also not uncommon
to see libraries like `libfoo.so.1.2` and `libfoo2.so.1.2` -- this
is essentially the same thing.

BTW, the reason why we are going through all this trouble is GCC's
early experience with their plugins: initially the plugins weren't
versioned with the GCC version and this ended up in a big mess. We
want to avoid this from the outset.


> I wonder if it might be better to use symbol versioning[1,2] for this
> (at least on Linux--I don't know how Mac/Windows deal with this).
> 
> Consider a scenario in which the module developer links against a
> different version of libbuild2 than that against which the user's
> build2 is linked.
> 
> With filename versioning:
>   1. Someone writes a (possibly proprietary) build system module "a"
> linked against libbuild2-0.12.so, producing
> libbuild2-a-0.12{,-0.1}.so.
>   2. Module "a" is distributed along with libbuild2-0.12.so.
>   3. User loads module "a" in some build2 project with build2 linked
> against libbuild2-0.13.so, dynamically loading both libbuild2-0.12.so
> and libbuild2-0.13.so in the process. Depending on the particular
> changes between these two library versions, this may fail
> catastrophically.

This scenario can never happen since the build system core will only
load (BTW, this is who actually loads modules) a module with a matching
build core interface version. So in the above example, if the build
system driver is linked to libbuild2-0.13.so, then it will only try
to load libbuild2-a-0.13.so.


> With symbol versioning:
>   1. Someone writes a (possibly proprietary) build system module "a"
> linked against libbuild2.so.1 with symbol versions 0.12, producing
> libbuild2-a.so.1.
>   2. Module "a" is distributed along with libbuild2.so.1 with symbol
> versions 0.12.
>   3. User loads module "a" in some build2 project with build2 linked
> against libbuild2.so.1 with symbol versions 0.12 and 0.13, dynamically
> loading only libbuild2.so.1 with symbol versions 0.12 and 0.13.
> Provided the symbol versions were used in a compatible manner (e.g. a
> symbol previously versioned 0.12 must be kept), this should work just
> fine.

I don't have much experience with symbol versioning, but it sounds like
we will have to carefully track changes to our ABI and adjust each
symbol's version accordingly. I don't think it will practical in our
case, at least at this stage.


> Even if multiple versions of libbuild2-0.{x,y,z}.so are distributed in
> `$prefix/lib`, there's no guarantee that they can be simultaneously
> loaded by a single executable [...]

We have no plans to provide such a guarantee, probably ever ;-).

What we have now is essentially how GCC handles plugins, which works
reasonably well. The main difference is they version their plugins
by installing them into the compiler version-specific directory while
we embed the build system core interface version into the library
name.