From boris at codesynthesis.com Tue Jun 18 13:14:36 2024 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue, 18 Jun 2024 15:14:36 +0200 Subject: [build2-announce] build2 0.17.0 released Message-ID: We have released build2 toolchain version 0.17.0. The main area of focus in this release was again more advanced functionality for more complex projects. There are also notable improvements in C++20 modules support, including for `import std;` in Clang and MSVC. On the documentation front, we now have a comprehensive, step-by-step guide for packaging third-party projects for build2: https://build2.org/build2-toolchain/doc/build2-toolchain-packaging.xhtml There are also quite a few smaller features in this release, such as new buildfile value types and functions, including support for JSON. Additionally, a large amount of maintenance work has been done, mostly in the form of handling various corner cases and fixing bugs. For example, there are about a hundred commit messages in this release that start with "Fix". A note on backwards compatibility: this release cannot be upgraded to from 0.16.0 and has to be installed from scratch. Also note that build2 0.16.0 has a bug which will prevent it from working with package repositories containing any packages with the build2 version constraint greater than 0.16.0. As a result, we recommend that you upgrade to 0.17.0 as soon as possible. To ease the impact of this bug we will embargo publishing packages with such constraints to cppget.org for one month from the date of this release (so until 18 July 2024). The corresponding NEWS file entries are included below with a more detailed discussion of the major new features available in the release notes: https://build2.org/release/0.17.0.xhtml Additional notes for distribution package maintainers can be found in the following pre-release announcement: https://lists.build2.org/archives/users/2024-June/001117.html This release has been tested on Windows, Linux (x86_64 and aarch64), Mac OS (x86_64 and aarch64), and FreeBSD with GCC, Clang (both vanilla and Apple's), MinGW GCC, and MSVC (14, 15, 16, and 17). A big thank you to everyone who helped test the development snapshots and the release candidate. The following new build configurations have been added to the CI service: freebsd_13.3-clang_17 freebsd_14.1-clang_18 linux_debian_12-gcc_14 (x86_64 and aarch64) linux_debian_12-clang_17[_libc++] (x86_64 and aarch64) linux_debian_12-clang_18[_libc++] (x86_64 and aarch64) macos_14-clang_15.0 (Xcode 15.3, Apple Clang 1500.3.9.4) macos_14-gcc_14_homebrew windows_10-msvc_17.8 (LTSC) windows_10-msvc_17.10 windows_10-clang_17_msvc_msvc_17.10 windows_10-clang_18_llvm_msvc_17.10 windows_10-gcc_13.2_mingw_w64 Note also that we have removed the Emscripten configuration (linux_debian_11-emcc_3.1.6) from the `all` and `experimental` classes. So if you want your packages to continue building in this configuration, you will need to add it explicitly, for example, using the `wasm` class or, alternatively, with build-include manifest value. Additionally, the following new configurations have been added to the `bindist` class: linux_ubuntu_22.04-gcc_11-bindist linux_ubuntu_24.04-gcc_13-bindist linux_fedora_39-gcc_13-bindist linux_fedora_40-gcc_14-bindist linux_rhel_9.2-gcc_11-bindist macos_13-clang_15.0-bindist windows_10-msvc_17.8-bindist All in all, there are now 100 build configurations. For the complete list, see: https://ci.cppget.org/?build-configs Installation instructions: https://build2.org/install.xhtml Download directory: https://download.build2.org/0.17.0/ SHA256 checksums: b84e4114c61aa94c3f6278f010a0dc0536dda65ac39d3863152ec9b64510b86e *build2-install-0.17.0.sh 9af34586cbe32360fa783883446cd90033152a1225c73acd92b45a95beafa695 *build2-install-msvc-0.17.0.bat a612a7761774c4602a7633eb9b1acc791f8c51771fdb59c7b43a878ffec5d7fc *build2-install-clang-0.17.0.bat ddd3cbfbc5927cc1736cadf3bdb6395c8261a3bebfedb2bb83e72f3964ad1bb8 *build2-install-mingw-0.17.0.bat 5c7d4a797c252e4516d1e913e50d6cf4d461d340d1e4fa295c878667e6b8bab0 *build2-toolchain-0.17.0.tar.gz 3722a89ea86df742539d0f91bb4429fd46bbf668553a350780a63411b648bf5d *build2-toolchain-0.17.0.tar.xz 2e64660c38cea2a9ef81c367ec80af4aa9ff606c6bb6ad0f9647b80909648bea *build2-baseutils-0.17.0-x86_64-windows.zip f95c894573c271f1bc51e059dc1b3287f6111d06ed099b0e596053af6f0b4675 *build2-mingw-0.17.0-x86_64-windows.tar.xz Repository certificate fingerprint for cppget.org: 70:64:FE:E4:E0:F3:60:F1:B4:51:E1:FA:12:5C:E0:B3:DB:DF:96:33:39:B9:2E:E5:C2:68:63:4C:A6:47:39:43 NEWS file entries: build2 version 0.17.0 --------------------- * C++20 modules support improvements: - Named modules support in Clang, including automatic building of the `std` and `std.compat` standard library modules from libc++ with Clang 18 or later. - Named modules support in MSVC, including automatic building of the `std` and `std.compat` standard library modules. Note: if combining the standard library modules importation with the standard library headers inclusion, MSVC 17.10 or later is recommended. - Modules support can now be enabled for any std.cxx value greater or equal 20 (including `latest`). - The "C++ Modules Support" section in the manual has been updated to match the current state of the implementation. * The `latest` and `experimental` cxx.std values are now mapped to C++26 from GCC 14 and Clang 18. * The 23 and 2x c.std values are now mapped to /std:clatest starting from MSVC 17.9. In particular, this option enables C23 typeof support. * /Zc:preprocessor is now added for the `experimental` cxx.std value from MSVC 17.9. * New string_set buildfile value type. This exposes the std::set type to buildfiles. New functions: $size() Subscript returns true if the value is present and false otherwise (so it is mapped to std::set::contains()). For example: set = [string_set] a b c if ($set[b]) ... Note that append (+=) and prepend (=+) have the same semantics (std::set::insert()). For example: set = [string_set] a b set += c b # a b c set =+ d b # a b c d Example of iteration: set = [string_set] a b c for k: $set ... * New string_map buildfile value type. This exposes the std::map type to buildfiles. New functions: $size() $keys() Subscript can be used to look up a value by key. The result is [null] if there is no value associated with the specified key. For example: map = [string_map] a at 1 b at 2 c at 3 b = ($map[b]) # 2 if ($map[z] == [null]) ... Note that append (+=) is overriding (like std::map::insert_or_assign()) while prepend (=+) is not (like std::map::insert()). In a sense, whatever appears last (from left to right) is kept, which is consistent with what we expect to happen when specifying the same key repeatedly in a literal representation. For example: map = [string_map] a at 0 b at 2 a at 1 # a at 1 b at 2 map += b at 0 c at 3 # a at 1 b at 0 c at 3 map =+ b at 1 d at 4 # a at 1 b at 0 c at 3 d at 4 Example of iteration: map = [string_map] a at 1 b at 2 c at 3 for p: $map { k = $first($p) v = $second($p) } While the subscript is mapped to key lookup only, index-based access can be implemented (with a bit of overhead) using the $keys() function: map = [string_map] a at 1 b at 2 c at 3 keys = $keys($m) for i: $integer_sequence(0, $size($keys)) { k = ($keys[$i]) v = ($map[$k]) } * New JSON buildfile value types. New types: json json_array json_object New functions: $json.value_type() $json.value_size() $json.member_{name,value}() $json.object_names() $json.array_size() $json.array_find(, ) $json.array_find_index(, ) $json.load() $json.parse() $json.serialize([, ]) See "JSON Functions" in the manual for details. For example, to load a JSON value from a file: j = $json.load($src_base/board.json) Or to construct it in a buildfile: j = [json] one at 1 two@([json] 2 3 4) three@([json] x at 1 y at -1) This can also be done incrementally with append/prepend: j = [json_object] j += one at 1 j += two@([json] 2 3 4) j += three@([json] x at 1 y at -1) Instead of using this JSON-like syntax, one can also specify valid JSON input text: j = [json] '{"one":1, "two":[2, 3, 4], "three":{"x":1, "y":-1}' Besides the above set of functions, other handy ways to access components in a JSON value are iteration and subscript. For example: for m: $j print $member_name($m) $member_value($m) print ($j[three]) A subscript can be nested: print ($j[two][1]) print ($j[three][x]) While a JSON value can be printed directly like any other value, the representation will not be pretty-printed. As a result, for complex JSON values, printing a serialized representation might be a more readable option: info $serialize($j) * New json_map and json_set buildfile value types. These expose the std::map and std::set types to buildfiles. New functions: $size() $size() $keys() Note that the $keys() function returns the list of map key as a json array. For example: m = [json_map] 2@([json] a at 1 b at 2) 1@([json] 1 2) s = [json_set] ([json] x at 1 y at 2) ([json] a at 1 b at 2) print ($m[2][b]) # 2 print ($s[([json] y at 2 x at 1)]) # true * New $first() and $second() functions which return the first and second halves of a pair, respectively. * New string functions: $string.contains() $string.starts_with() $string.ends_with() $string.replace() See "String Functions" in the manual for details. * New path functions: $path.absolute() $path.simple() $path.sub_path() $path.super_path() $path.complete() $path.try_normalize() $path.try_actualize() See "Path Functions" in the manual for details. * New filesystem functions: $filesystem.file_exists() $filesystem.directory_exists() See "Filesystem Functions" in the manual for details. * Runtime/buildtime distinction when installing libraries. Specifically, now, if a library is installed solely as a prerequisite of an executable (potentially recursively), then only its runtime files are installed omitting everything buildtime-related (static/import libraries, non-versioned symlinks for shared libraries, pkg-config files, headers, etc). If you are familiar with the runtime and -dev/-devel package splits for libraries in Debian/Fedora, this is the analogous semantics. * Support for extracting C and C++ predefined macros (predefs). Specifically, the c and cxx modules now provide the c.predefs and cxx.predefs submodules which can be loaded in order to register a rule that generates a C or C++ header with the predefined compiler macros, respectively. For details, refer to "C Compiler Predefined Macro Extraction" and "C++ Compiler Predefined Macro Extraction" in the manual. * Ability to serialize compilation/linking in C/C++ rules. Specifically, both the C/C++ compile and link rules now recognize the cc.serialize boolean variable which instructs them to compile/link serially with regards to any other recipe. This is primarily useful when compiling large translation units or linking large binaries that require so much memory that doing that in parallel with other compilation/linking jobs is likely to summon the OOM killer. For example: obj{memory-hog}: cc.serialize = true * Ability to specify compiler mode options in a buildfile. Now the configured mode options are appended to buildfile-specified (which must be specified before loading the guess module). In particular, this ability to specify the compiler mode in a buildfile is useful in embedded development where the project may need to hardcode options like -target, -nostdinc, etc. For example: cxx.std = 20 cxx.mode = -target riscv32-unknown-unknown -nostdinc using cxx * New ~host-no-warnings and ~build2-no-warnings special configurations. These are parallel to ~host and ~build2 but with suppressed C/C++ compiler warnings. Note also that the C++ ad hoc recipes are now by default built in ~build2-no-warnings instead of ~build2 unless the project is configured for development with config..develop=true. * New {bin,c,cxx}.types submodules that only register target types. This is primarily useful when providing custom C/C++ compilation/linking rules. * New -s|--timeout-success option in the `env` script builtin. The semantics is equivalent to the --success option in the `timeout` builtin. * Ability to specify alternative sysroot for pkg-config files. Specifically, the new config.cc.pkgconfig.sysroot variable provides roughly equivalent functionality to PKG_CONFIG_SYSROOT_DIR in pkg-config. For details and limitations, see "Rewriting Installed Libraries System Root (sysroot)" in the manual for details. * Ability to alias a target type from another project. The syntax is: define = / For example: sdk_proj = ... # Root directory of the SDK. define ldscript = $sdk_proj/ldscript Additionally, unknown target types of imported targets are now aliased automatically. * New no_default_target attribute for source, buildfile import directives. This attribute can be used to disable the default target semantics for the sources/imported buildfile. * Allow imported buildfiles to use config.* variables from own projects (that is, the project from which the buildfile is imported). * The fsdir{} targets are now usable in ad hoc recipes. In particular, they can now be used to represent directory symlinks. For example: exe{hello}: ... fsdir{assets} fsdir{assets}: % update {{ ln -s $src_base/assets $out_base/assets }} % clean {{ rm $out_base/assets }} Likewise, file{} targets can now be used to represent file symlinks created in ad hoc recipes. * Ability to specify ad hoc recipes in separate files. This can now be achieved with the new `recipe` directive: recipe Note that similar to the use of if-else and switch directives with recipes, this directive requires explicit % recipe header. For example, instead of: file{foo.output}: {{ echo 'hello' >$path($>) }} We can now write: file{foo.output}: % recipe buildscript hello.buildscript With hello.buildscript containing: echo 'hello' >$path($>) Similarly, for C++ recipes (this time for a pattern rule), instead of: [rule_name=hello] file{~'/(.+)\.output/'}: % update clean {{ c++ 1 -- -- ... }} We can now write: [rule_name=hello] file{~'/(.+)\.output/'}: % update clean recipe c++ hello.cxx With hello.cxx containing: // c++ 1 -- -- ... Relative paths are resolved using the buildfile directory that contains the `recipe` directive as a base. Note also that this mechanism can be used in exported buildfiles with recipe files placed into build/export/ together with buildfiles. bpkg version 0.17.0 ------------------- * Support for preserving existing package configuration on up/downgrade and reconfiguration. * Support for the enable clause in the `tests` package manifest value. * New per package configuration *-build-*email manifest values. * New build-bot package manifest value allows specifying package-custom build bots. * Support for specifying package archives and directories as dependencies for pkg-build. * New `none` value for the --recursive option in pkg-bindist. * Empty --archive-lang* option values in pkg-bindist are now treated as a request to clear previous entries. bdep version 0.17.0 ------------------- * New `third-party` library and executable sub-option in bdep-new. This sub-option is meant for converting an existing third-party project to build2. It automatically enables a number of other sub-options (such as no-version, no-readme, and no-symexport). It also adds a number of values to manifest that makes sense to specify in a package of a third- party project and, unless no-package-readme is specified, generates the PACKAGE-README.md template. * New buildfile-in-prefix library and executable sub-option in bdep-new. * New export-stub executable sub-option in bdep-new. * New no-symexport and auto-symexport library sub-options in bdep-new. * New no-subdir-include library sub-option in bdep-new. From boris at codesynthesis.com Wed Aug 14 10:27:13 2024 From: boris at codesynthesis.com (Boris Kolpackov) Date: Wed, 14 Aug 2024 12:27:13 +0200 Subject: [build2-announce] Changes to package review/transition policies on cppget.org Message-ID: We are inviting the build2 community to help with reviewing package submissions. We are also adjusting the package transition policies between queue.cppget.org and cppget.org as well as between sections within cppget.org. Specifically, migrating a package from queue.cppget.org to cppget.org no longer requires a review, only that the package builds in at least one configuration. As a result, the alpha, beta, and testing sections of cppget.org may now contain unreviewed packages. However, a package can only be migrated from testing to stable if it passes a review. See the full announcement for details, including the review procedure: https://build2.org/blog/package-review-transition.xhtml Let me know if you have any question or concerns.