[build2] build2 0.15.0 released
Boris Kolpackov
boris at codesynthesis.com
Thu Jul 28 14:24:05 UTC 2022
We have released build2 toolchain version 0.15.0.
The overall theme of this release is more advanced functionality that
is becoming necessary as we start to use build2 to handle more complex
projects. Specifically, the package manager now supports a number of
more advanced dependency declaration features, including conditional
dependencies, dependency alternatives, and dependency configuration.
On the build system side we now have rule hints, support for dynamic
dependencies in ad hoc recipes, and the ability to save user metadata
in C/C++ libraries.
Another new development is the creation of the HOWTO repository with
practical advice on using build2 to achieve common tasks. At the time
of the release the repository contained 10 articles:
https://github.com/build2/HOWTO/#readme
A note on backwards compatibility: this release cannot be upgraded to from
0.14.0 and has to be installed from scratch.
The corresponding NEWS file entries are included below with a more detailed
discussion of the major new features available in the release notes:
https://build2.org/release/0.15.0.xhtml
This release adds one new standard pre-installed build system module,
autoconf (GNU Autoconf emulation for processing config.h.in files):
https://github.com/build2/libbuild2-autoconf/
For background on standard pre-installed modules see:
https://lists.build2.org/archives/users/2021-January/000862.html
All the information-querying commands in the package and project managers
as well as the structured result output and the info meta-operation in the
build system can now produce JSON output. This should make it easier to
integrate build2 with other tools, IDEs, etc (see the NEWS files below
for details).
This release also includes a large number of performance optimizations in
the build system which should especially help projects that make use of
heavily inter-dependent libraries (such as Boost). For example, on some
tests the up-to-date check times went down from 1.2s to 0.3s.
This release has been tested on Windows, Linux, Mac OS (including M1), and
FreeBSD with GCC, Clang (both vanilla and Apple's), MinGW GCC, and MSVC (14,
15, 16, and 17). There is also preliminary support for OpenBSD. A big thank
you to everyone who helped test the development snapshots and the release
candidate.
The following new build configurations have been added to the CI service:
freebsd_13-clang_13.0
linux_debian_11-gcc_12.1
linux_debian_11-clang_14.0[_libc++]
macos_12-clang_13.1 (Xcode 13.4 Clang 13.1.6)
macos_12-gcc_12.1_homebrew
windows_10-msvc_17.2
windows_10-clang_14.0_llvm_msvc_17.2[_lld]
windows_10-gcc_11.2_mingw_w64
linux_debian_11-emcc_3.1.6 (Emscripten)
In addition, all the Linux configurations in the default class now include
system packages necessary for GUI development and run a mock X server via
Xvfb. All in all, there are now 66 build configurations. For the complete
list, see:
https://ci.cppget.org/?build-configs
Installation instructions:
https://build2.org/install.xhtml
Download directory:
https://download.build2.org/0.15.0/
SHA256 checksums:
814c4f475b42749dab49c52549ef85899749c20e5c32276b281fd58dad54f47b *build2-install-0.15.0.sh
2556da043d11d56a58bb5b4a8279e6eddf87d81b70e72c0e808008d807195efb *build2-install-msvc-0.15.0.bat
40d266125e4c7fd35801ea5e8375a34d50b14cf8aaf36e21abf6dfafa081344d *build2-install-clang-0.15.0.bat
487b8aa29fc350413de0652ad7a04e605f8c0ca067d8a9765b4c213683af6e36 *build2-install-mingw-0.15.0.bat
b7df173dc1b7b48ab6e09d201112c0fcec7ba62b2b33789c5e17516e9850342c *build2-toolchain-0.15.0.tar.gz
5152f679daeb9627f9710c60ef88de1591c02097146268be2f5aea929d2837c4 *build2-toolchain-0.15.0.tar.xz
8aaaeea6b99c4c123be87f0c3906321c512418523530c3427d6802e81e62a150 *build2-baseutils-0.15.0-x86_64-windows.zip
a1adb7da12db9d88b0fbf528b29dc39b5568693fae0558d9866689263871063c *build2-mingw-0.15.0-x86_64-windows.tar.xz
Repository certificate fingerprint for cppget.org:
70:64:FE:E4:E0:F3:60:F1:B4:51:E1:FA:12:5C:E0:B3:DB:DF:96:33:39:B9:2E:E5:C2:68:63:4C:A6:47:39:43
NEWS file entries:
build2 version 0.15.0
---------------------
* Generated C/C++ headers and ad hoc sources are now updated during match.
Specifically, all headers as well as ad hoc headers and sources are now
treated by the cc::link_rule as if they had update=match unless explicit
update=execute is specified (see below on the update operation-specific
variable).
This change should be transparent to most projects. For background and
discussion of rare cases where you may wish to disable this, see:
https://github.com/build2/HOWTO/blob/master/entries/handle-auto-generated-headers.md
* Support for rule hints.
A rule hint is a target attribute, for example:
[rule_hint=cxx] exe{hello}: c{hello}
Rule hints can be used to resolve ambiguity when multiple rules match the
same target as well as to override an unambiguous match.
In cc::link_rule we now support "linking" libraries without any sources or
headers with a hint. This can be useful for creating "metadata libraries"
whose only purpose is to convey metadata (options to use and/or libraries
to link).
* UTF-8 is now the default input/source character set for C/C++ compilation.
Specifically, the cc module now passes the appropriate compiler option
(/utf-8 for MSVC and -finput-charset=UTF-8 for GCC and Clang) unless a
custom value is already specified (with /{source,execution}-charset for
MSVC and -finput-charset for GCC and Clang).
This change may trigger new compilation errors in your source code if
it's not valid UTF-8 (such errors most commonly point into comments).
For various ways to fix this, see:
https://github.com/build2/HOWTO/blob/master/entries/convert-source-files-to-utf8.md
* Project configuration variables are now non-nullable by default.
A project configuration variable with the NULL default value is naturally
assumed nullable, for example:
config [string] config.libhello.fallback_name ?= [null]
Otherwise, to make a project configuration nullable use the `null`
variable attribute, for example:
config [string, null] config.libhello.fallback_name ?= "World"
* New $relative(<path>, <dir-path>) function.
* New $root_directory(<path>) function.
* New $size() function to get the size of string, path, dir_path.
* New $size() function to get the size of a sequence (strings, paths, etc).
* New $sort() function to sort a sequence (strings, paths, etc).
The function has the following signature:
$sort(<sequence> [, <flags>])
The following flag is supported by all the overloads:
dedup - in addition to sorting also remove duplicates
Additionally, the strings overload also support the following flag:
icase - sort ignoring case
Note that on case-insensitive filesystems the paths and dir_paths
overloads' order is case-insensitive.
* New $config.origin() function for querying configuration value origin.
Give a config.* variable name, this function returns one of `undefined`,
`default`, `buildfile`, or `override`.
* Recognition of -pthread as a special -l option in *.libs.
For background, see:
https://github.com/build2/HOWTO/blob/master/entries/link-pthread.md
* The bin.whole (whole archive) value is now saved in generated pkg-config
files.
* Ability to customize header and library search paths in generated
pkg-config files.
Specifically, {cc,c,cxx}.pkgconfig.{include,lib} variables specify header
(-I) and library (-L) search paths to use in the generated pkg-config
files instead of the default install.{include,lib}. Relative paths are
resolved as installation paths. For example:
lib{Qt6Core}: cxx.pkgconfig.include = include/qt6/
* Ability to save user metadata in C/C++ libraries, including in generated
pkg-config files.
For background and details, see:
https://github.com/build2/HOWTO/blob/master/entries/convey-additional-information-with-exe-lib.md
* Support for rule-specific search in immediate import.
We can now nominate a rule to perform the rule-specific search (if
required) using the rule_hint attribute. For example:
import! [metadata, rule_hint=cxx.link] lib = libhello%lib{hello}
* Support for dynamic dependencies in ad hoc recipes.
Specifically, the `depdb` builtin now has the new `dyndep` command that
can be used to extract dynamic dependencies from program output or a
file. For example, from program output:
obje{hello.o}: cxx{hello}
{{
s = $path($<[0])
o = $path($>)
poptions = $cxx.poptions $cc.poptions
coptions = $cc.coptions $cxx.coptions
depdb dyndep $poptions --what=header --default-type=h -- \
$cxx.path $poptions $coptions $cxx.mode -M -MG $s
diag c++ ($<[0])
$cxx.path $poptions $coptions $cxx.mode -o $o -c $s
}}
Or, alternatively, from a file:
t = $(o).t
depdb dyndep $poptions --what=header --default-type=h --file $t -- \
$cxx.path $poptions $coptions $cxx.mode -M -MG $s >$t
The above depdb-dyndep commands will run the C++ compiler with the -M -MG
options to extract the header dependency information, parse the resulting
make dependency declaration (either from stdout or from file) and enter
each header as a prerequisite of the obje{hello.o} target, as if they were
listed explicitly. It will also save this list of headers in the auxiliary
dependency database (hello.o.d file) in order to detect changes to these
headers on subsequent updates. The --what option specifies what to call
the dependencies being extracted in diagnostics. The --default-type option
specifies the default target type to use for a dependency if its file name
cannot be mapped to a target type.
The above depdb-dyndep variant extracts the dependencies ahead of the
compilation proper and will handle auto-generated headers (see the -MG
option for details) provided we pass the header search paths where they
could be generated with the -I options (passed as $poptions in the above
example).
If there can be no auto-generated dependencies or if they can all be
listed explicitly as static prerequisites, then we can use a variant of
the depdb-dyndep command that extracts the dependencies as a by-product of
compilation. In this mode only the --file input is supported. For example
(assuming hxx{config} is auto-generated):
obje{hello.o}: cxx{hello} hxx{config}
{{
s = $path($<[0])
o = $path($>)
t = $(o).t
poptions = $cxx.poptions $cc.poptions
coptions = $cc.coptions $cxx.coptions
depdb dyndep --byproduct --what=header --default-type=h --file $t
diag c++ ($<[0])
$cxx.path $poptions $coptions $cxx.mode -MD -MF $t -o $o -c $s
}}
Other options supported by the depdb-dyndep command:
--format <name>
Dependency format. Currently only the `make` dependency format is
supported and is the default.
--cwd <dir>
Working directory used to complete relative dependency paths. This
option is currently only valid in the --byproduct mode (in the normal
mode relative paths indicate non-existent files).
--adhoc
Treat dynamically discovered prerequisites as ad hoc (so they don't end
up in $<; only in the normal mode).
--drop-cycles
Drop prerequisites that are also targets. Only use this option if you
are sure such cycles are harmless, that is, the output is not affected
by such prerequisites' content.
--update-{include,exclude} <tgt>|<pat>
Prerequisite targets/patterns to include/exclude (from the static
prerequisite set) for update during match (those excluded will be
updated during execute). The order in which these options are specified
is significant with the first target/pattern that matches determining
the result. If only the --update-include options are specified, then
only the explicitly included prerequisites will be updated. Otherwise,
all prerequisites that are not explicitly excluded will be updated. If
none of these options is specified, then all the static prerequisites
are updated during match. Note also that these options do not apply to
ad hoc prerequisites which are always updated during match.
The common use-case for the --update-exclude option is to omit updating
a library which is only needed to extract exported preprocessor options.
Here is a typical pattern:
import libs = libhello%lib{hello}
libue{hello-meta}: $libs
obje{hello.o}: cxx{hello} libue{hello-meta}
{{
s = $path($<[0])
o = $path($>)
lib_poptions = $cxx.lib_poptions(libue{hello-meta}, obje)
depdb hash $lib_poptions
poptions = $cxx.poptions $cc.poptions $lib_poptions
coptions = $cc.coptions $cxx.coptions
depdb dyndep $poptions --what=header --default-type=h \
--update-exclude libue{hello-meta} -- \
$cxx.path $poptions $coptions $cxx.mode -M -MG $s
diag c++ ($<[0])
$cxx.path $poptions $coptions $cxx.mode -o $o -c $s
}}
As another example, sometimes we need to extract the "common interface"
preprocessor options that are independent of the the library type (static
or shared). For example, the Qt moc compiler needs to "see" the C/C++
preprocessor options from imported libraries if they could affect its
input. Here is how we can implement this:
import libs = libhello%lib{hello}
libul{hello-meta}: $libs
cxx{hello-moc}: hxx{hello} libul{hello-meta} $moc
{{
s = $path($<[0])
o = $path($>[0])
t = $(o).t
lib_poptions = $cxx.lib_poptions(libul{hello-meta})
depdb hash $lib_poptions
depdb dyndep --byproduct --drop-cycles --what=header --default-type=h \
--update-exclude libul{hello-meta} --file $t
diag moc ($<[0])
$moc $cc.poptions $cxx.poptions $lib_poptions \
-f $leaf($s) --output-dep-file --dep-file-path $t -o $o $s
}}
Planned future improvements include support for the `lines` (list of
files, one per line) input format in addition to `make` and support for
dynamic targets in addition to prerequisites.
* Support for specifying custom ad hoc pattern rule names.
Besides improving diagnostics, this allows us to use such a name in the
rule hints, for example:
[rule_name=hello.link] exe{~'/(.*)/'}: obje{~'/\1/'}
{{
$cxx.path -o $path($>) $path($<[0])
}}
[rule_hint=hello] exe{hello}: obje{hello}
obje{hello}: c{hello-c}
* Ability to disfigure specific configuration variables.
The new config.config.disfigure variable can be used to specify the list
of variables to ignore when loading config.build (and any files specified
in config.config.load), letting them to take on the default values. For
example:
$ b configure config.config.disfigure=config.hello.fancy
Besides names, variables can also be specified as patterns in the
config.<prefix>.(*|**)[<suffix>] form where `*` matches single
component names (i.e., `foo` but not `foo.bar`), and `**` matches
single and multi-component names. Currently only single wildcard (`*` or
`**`) is supported. Additionally, a pattern in the config.<prefix>(*|**)
form (i.e., without `.` after <prefix>) matches config.<prefix>.(*|**)
plus config.<prefix> itself (but not config.<prefix>foo).
For example, to disfigure all the project configuration variables (while
preserving all the module configuration variables; note quoting to prevent
pattern expansion):
$ b config.config.disfigure="'config.hello**'"
* Ability to omit loading config.build.
If the new config.config.unload variable is set to true, then omit loading
the project's configuration from the config.build file. Note that the
configuration is still loaded from config.config.load if specified. Note
also that similar to config.config.load, only overrides specified on this
project's root scope and global scope are considered.
* Ability to match libul{} targets.
The bin.libul rule picks, matches, and unmatches (if possible) a member
for the purpose of making its metadata (for example, library's poptions,
if it's one of the cc libraries) available.
* Ability to get common interface options via ${c,cxx}.lib_poptions().
Specifically, the output target type may now be omitted for utility
libraries (libul{} and libu[eas]{}). In this case, only "common interface"
options will be returned for lib{} dependencies. This is primarily useful
for obtaining poptions to be passed to tools other than C/C++ compilers
(for example, Qt moc).
* Ability to control -I translation to -isystem or /external:I in
${c,cxx}.lib_poptions().
See the function documentation for details.
* New `update` operation-specific variable.
This variable is similar to the already existing `clean` and `test`
variables but besides the standard `true` and `false` values, it can also
be set to `unmatch` (match but do not update) and `match` (update during
match) and `execute` (update during execute, as is normally; this value is
primarily useful if the rule has the `match` semantics by default).
Note that the unmatch (match but do not update) and match (update during
match) values are only supported by certain rules (and potentially only
for certain prerequisite types).
Additionally:
- All the operation-specific variables are now checked for `false` as an
override for the prerequisite-specific `include` variable. This can now
be used to disable a prerequisite for update, for example:
./: exe{test}: update = false
- Ad hoc Buildscript recipes now support update=unmatch|match.
- The cc::link_rule now supports the `match` value for headers and ad hoc
prerequisites. This can be used to make sure all the library headers are
updated before matching any of its (or dependent's) object files.
* New build.mode global scope variable.
This variable signals the mode the build system may be running in. The two
core modes are `no-external-modules` (bootstrapping of external modules is
disabled, see --no-external-modules for details) and `normal` (normal
execution). Other build system drivers may invent additional modes (for
example, the bpkg `skeleton` mode; see "Package Build System Skeleton" in
the package manager manual for details).
* New cmdline value type for canned command lines.
The Testscript and Buildscript languages now use the special cmdline value
type for canned command lines. Specifically, the re-lexing after expansion
now only happens if the expended value is of the cmdline type. See
"Lexical Structure" in the Testscript manual for details.
* The bash build system module now installs bash modules into
bin/<project>.bash/ instead of bin/<project>/ to avoid clashes.
* New --trace-{match,execute} options.
These options can be used to understand which dependency chain causes
matching or execution of a particular target. See b(1) for details.
* JSON format support for the --structured-result option and the info meta
operation.
See b(1) for details.
* Switch to using libpkg-config instead of libpkfconf for loading pkg-config
files.
bpkg version 0.15.0
-------------------
* New dependency declaration features:
- Dependency groups, for example:
depends: { libboost-any libboost-log libboost-uuid } ~1.77.0
- Conditional dependencies, for example:
depends: libposix-getopt ^1.0.0 ? ($cxx.target.class == 'windows')
- Dependency alternatives, for example:
depends: libmysqlclient >= 5.0.3 | libmariadb ^10.2.2
- Reflected configuration variables, for example:
depends: libposix-getopt ^1.0.0 \
? ($cxx.target.class == 'windows') \
config.hello.external_regex=true
Or:
depends: libmysqlclient >= 5.0.3 config.hello.db='mysql' | \
libmariadb ^10.2.2 config.hello.db='mariadb'
- Dependency configuration, for example:
depends:
\
libmariadb ^10.2.2
{
require
{
config.libmariadb.cache = true
if ($cxx.target.class != 'windows')
config.libmariadb.tls = true
}
}
\
Or:
depends:
\
libmariadb ^10.2.2
{
prefer
{
config.libmariadb.cache = true
config.libmariadb.buffer = ($config.libmariadb.buffer < 4096 \
? 4096 \
: $config.libmariadb.buffer)
}
accept ($config.libmariadb.buffer >= 4096)
}
\
See the `depends` package manifest value in the manual for details.
The implementation of these features led to bpkg becoming a special build
system driver with repository metadata now containing the minimal subset
of build system files for each package (called the package build system
skeleton). See "Package Build System Skeleton" in the manual for details.
* Support for JSON output in the bpkg-pkg-status command.
See the --stdout-format option in bpkg-pkg-status(1) for details.
* New --all, --all-pattern bpkg-pkg-drop options.
These options can be used to drop all the held packages (in case of
--all-pattern, limited to those that match a wildcard pattern).
* New --keep-tmp common option.
This option instructs bpkg not to remove its temporary directory at the
end of the command execution and print its path if the verbosity level is
2 or higher. This option is primarily useful for troubleshooting.
bdep version 0.15.0
-------------------
* Support for JSON output in the bdep-status, bdep-config-list commands.
See the --stdout-format option in bdep-status(1) and bdep-config(1)
for details.
* Support for submitting to a CI server packages with a non-standard
version.
See bdep-ci(1) for details.
* Support for publishing packages with a non-standard version.
See bdep-publish(1) for details.
More information about the users
mailing list