From finjulhich at gmail.com Sat Jan 11 15:03:25 2020 From: finjulhich at gmail.com (MM) Date: Sat, 11 Jan 2020 15:03:25 +0000 Subject: [build2] build2 on fedora: bpkg's own rpms Message-ID: Hi, >From Fedora 31's main repos, I see a list of rpms: build2-...bootstrap build2-rpm-macros...bootstrap libbuild2...bootstrap In those I do find b, but not bpkg. There was a 3rd path repo https://masterspline.gitlab.io/build2-linux-packages/rpm_packages/ that hosted packages like build2-toolchain-0.11.0-1.fc30.x86_64 which contained both b and bpkg. I guess the boostrap rpms only help me build build2. I suppose at some point a prebuild build2 will start appear in Fedora's own repos? MM -------------- next part -------------- An HTML attachment was scrubbed... URL: From mkrupcale at matthewkrupcale.com Sat Jan 11 15:58:12 2020 From: mkrupcale at matthewkrupcale.com (Matthew Krupcale) Date: Sat, 11 Jan 2020 10:58:12 -0500 Subject: [build2] build2 on fedora: bpkg's own rpms In-Reply-To: References: Message-ID: Hello, I am the maintainer for the official build2 packages on Fedora. I'm not sure exactly where you are looking at that list of RPMs, but the main source package is build2, and this has several subpackages[1], including build2, build2-rpm-macros, libbuild2, libbutl, libbpkg, bpkg, and bdep. If you wish to install bpkg, you should install the bpkg subpackage. Let me know if you have any other questions. P.S.: The ~bootstrap appearing in the package names just means that these packages were bootstrapped, not that you need to bootstrap anything yourself. These should be fully functional build2 packages. Best, Matthew [1] https://koji.fedoraproject.org/koji/buildinfo?buildID=1415879 On Sat, Jan 11, 2020 at 10:03 AM MM wrote: > Hi, > > From Fedora 31's main repos, I see a list of rpms: > build2-...bootstrap > build2-rpm-macros...bootstrap > libbuild2...bootstrap > > In those I do find b, but not bpkg. > > There was a 3rd path repo > https://masterspline.gitlab.io/build2-linux-packages/rpm_packages/ > that hosted packages like > > build2-toolchain-0.11.0-1.fc30.x86_64 > > which contained both b and bpkg. > > I guess the boostrap rpms only help me build build2. > I suppose at some point a prebuild build2 will start appear in Fedora's > own repos? > > MM > -------------- next part -------------- An HTML attachment was scrubbed... URL: From finjulhich at gmail.com Sat Jan 11 17:41:50 2020 From: finjulhich at gmail.com (MM) Date: Sat, 11 Jan 2020 17:41:50 +0000 Subject: [build2] build2 on fedora: bpkg's own rpms Message-ID: On Sat, 11 Jan 2020 at 15:58, Matthew Krupcale < mkrupcale at matthewkrupcale.com> wrote: > Hello, > > I am the maintainer for the official build2 packages on Fedora. I'm not > sure exactly where you are looking at that list of RPMs, but the main > source package is build2, and this has several subpackages[1], including > build2, build2-rpm-macros, libbuild2, libbutl, libbpkg, bpkg, and bdep. If > you wish to install bpkg, you should install the bpkg subpackage. > > Let me know if you have any other questions. > > P.S.: The ~bootstrap appearing in the package names just means that these > packages were bootstrapped, not that you need to bootstrap anything > yourself. These should be fully functional build2 packages. > > Best, > Matthew > Thanks. I simply missed that bpkg and bdep rpms were already there too, I had just listed dnf list *build2* I have ran the from source script: build2-install-0.12.0.sh and it installed everything in /usr/local and then i built odb & co. So now I can simply uninstall via build2-install-0.12.0.sh and install bpkg rpm for future changes. Thanks MM -------------- next part -------------- An HTML attachment was scrubbed... URL: From boris at codesynthesis.com Tue Jan 21 09:30:55 2020 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue, 21 Jan 2020 11:30:55 +0200 Subject: [build2] Rust In-Reply-To: References: Message-ID: [CC'ing users at build2.org to a reply from private conversation] Matthew Krupcale writes: > My interest in Rust is primarily due to the fact that it is marketed > as a compiled, "systems" programming language focused on achieving > high performance with memory safety. This has garnered it a lot of > popularity lately, although I myself still prefer C++ for several > reasons, including: > > 1. it still hasn't really achieved performance parity with C++ in many cases > > 2. its compile times are even (far?) worse than C++ > > 3. it is focused on building statically linked artifacts which do not > integrate well with the traditional Linux FOSS ecosystem While that's the default, there is support for producing shared libraries. I agree this can't be ignored forever but making shared library builds usable across platforms is not easy (to put it very mildly and based on personal experience). And so I can't really blame them for ignoring this mess for the time being. Maybe we can help on this front seeing that we made it work for C/C++. > 4. its crates packaging system, while it allows for easy integration > of external projects into ones own (this is a nice feature to have in > general, including for C++), also comes with the downside of creating > large dependency chains typical of most other language-level package > managers (although maybe not quite as bad as e.g. NPM since the bar > for writing something useful is higher for Rust than JS, I would say) > > 5. it lacks toolchain diversity (i.e. it's still a monolithic system > built around LLVM). This also means it doesn't support as many target > architectures (although maybe not ones about which most people care). Yes, that's true, though there is talk about having a Rust frontend in GCC. Whether it will result in anything and if so, how soon, is another question, of course. Also, having multiple backends will probably slow down the language evolution. BTW, it just occurred to me that the same toolchain diversity argument can be applies to languages: it would be nice to have alternatives when it comes to C++. > 6. C++ is still more capable with respect to things like templates I believe > > That said, I have worked on some Rust projects (albeit mostly > peripherally, i.e. mostly insofar as the project uses C/C++) and have > learned a little bit about how the language/tooling works to some > extent, but I'm not sure I know any more than you might. I have also > peripherally watched what Meson has done with Rust (Meson has become > pretty popular in GNOME projects, some of which also use Rust), and I > believe they are mostly limited to projects which are fully > self-contained (i.e. no external crate dependencies[1]) because Cargo > is such a complex system that the Rust devs so far[2,3] have not > designed to allow any other build system to replicate the complicated > (and ever changing) incantations necessary to invoke rustc for a given > "real-world" project (Meson specifically wanted to deal with Rust by > invoking rustc itself, like all other languages[4]). Thus, they > haven't really bothered so far trying to create some sort of API which > alternative build-systems could use, and much like the Rust language > itself, the single, monolithic implementation of Cargo and the Rust > toolchain is the de facto standard as opposed to some more general > "Rust 2018" standard. Yes, we also would want to invoke rustc ourselves, at least for the project's (as opposed to dependencies) source code. I've been thinking about this (and this idea is also echoed in the issues you have linked): it feels like we need some sort of "foreign package" support in build2 that can be used to consume dependencies from other package managers. > > I've been looking into writing a build2 build system module for > > Rust compilation. > > I saw you had created a new repo for it on GH, but given everything I > mentioned above, I'm not sure it's currently feasible to make a proper > build2 module for Rust unless you: > > 1. limit it to fully self-contained crates as Meson currently does > 2. fully utilize and depend on both Cargo (along with its heavy > reliance on crates.io) for invoking rustc and building Rust artifacts For starters I am focusing on (1) but we will want to figure out (2) at some point as well. There is only so far you can get in Rust without external crates. I also want to explore the mixed (e.g., C/C++ and Rust) applications more. If anyone is interested in working on this, let me know. > Even these though don't really address the linkage issues associated > with Rust (see point 3 above). Yes, and I would like to look into this. Specifically, we will probably want to customize the linking step to control what linker/options are used so that we can make sure they are the same as what our cc module is using and take advantage of things like rpath, etc. Interestingly, I looked into how Bazel (currently) does it, and it's the same idea (i.e., invoke rustc but instruct it to use our linker/options). > Unless I'm mistaken, the current approach Fedora has taken to packaging > Rust is that the crates (which I believe are basically just tar archives > of the Rust source code) are equivalent to -devel packages only used > during build time to produce the various static artifacts which are > the actually distributed binaries. Yes, and I believe this is true for Debian as well. > All of this is to say that I more or less have avoided Rust because > despite its claimed advantages, they cannot (or have not prioritized) > seem to make a system which integrates well with the traditional Linux > ecosystem in an implementation-agnostic way. As a Linux packager, this > is important to me. > > > [1] https://github.com/mesonbuild/meson/issues/2173 > [2] https://github.com/rust-lang/cargo/issues/5579 > [3] https://github.com/rust-lang/cargo/issues/7614 > [4] https://github.com/mesonbuild/meson/pull/2617 Thanks for the links. A couple of interesting/relevant ones from my side: Crate dependency discovery: https://internals.rust-lang.org/t/crate-dependency-discovery/11418 Safe interop between Rust and C++: https://github.com/dtolnay/cxx Cross-language (C/C++ and Rust) LTO: http://blog.llvm.org/2019/09/closing-gap-cross-language-lto-between.html From mkrupcale at matthewkrupcale.com Tue Jan 21 14:28:04 2020 From: mkrupcale at matthewkrupcale.com (Matthew Krupcale) Date: Tue, 21 Jan 2020 09:28:04 -0500 Subject: [build2] Rust In-Reply-To: References: Message-ID: On Tue, Jan 21, 2020 at 4:31 AM Boris Kolpackov wrote: > > While that's the default, there is support for producing shared > libraries. Yeah, I know there is some sort of support for producing them, but it seems to me like they're not used much in practice (except the standard library?). > Maybe we can help on this front seeing that we made it work for C/C++. Perhaps so, but Rust also would have to make this an explicit goal (i.e. to prefer shared linking over static linking), and I'm not sure that's the case or if they even think it's desirable. Funnily enough, it looks like Rust might use the same name-mangling as for C++: $ readelf -Ws /usr/lib64/libstd-5420060374c44e8e.so | c++filt | less > Yes, that's true, though there is talk about having a Rust frontend in > GCC. Whether it will result in anything and if so, how soon, is another > question, of course. Yeah, I saw discussions around that as well, but I came away from them thinking if and when they do make a "GCC frontend", they're still going to be relying on the rustc frontend[1,2]. This has the obvious advantage of re-using the fast-moving rustc FE, but then it's not really an independent implementation. Also, having multiple backends will probably slow down the language evolution. I'm not sure the backends are the bottleneck for Rust language evolution, though, are they? From the discussions around the GCC FE, it seemed like MIR output of rustc was relatively stable, whereas the language FE itself is evolving more rapidly. > BTW, it just occurred to me that the same toolchain diversity argument > can be applies to languages: it would be nice to have alternatives when > it comes to C++. I'm not sure this scales in the same way that multiple implementations of a given standard/language does. LLVM and GCC are more or less substitutable for one another for a given language, but the same does not hold for developing and (more importantly) learning a new language. New languages fracture development and end users, whereas new implementations of a standard create competition while re-using existing skills/knowledge and tooling/infrastructure. There are many reasons that D, Rust, and Circle[3] have not or are unlikely to replace C++ any time soon, but the high cost relative to the small gains of learning a new language is probably foremost among them. At the end of the day, it seems to me like developers are the bottleneck, whereas there's a pretty low cost of substituting a compiler implementation for a given language. Of course, C++ evolves as well, but it evolves in a way which is incremental (in the sense that it is mostly additive and consistent with the existing foundations) and controlled. One could argue that this is a disadvantage compared to e.g. the rapid development of Rust, but I think C++ is in the right place in terms of speed of development of new features--too fast, and you risk a chaotic bombardment of ideas/features that do not work well (together); too slow, and you risk stagnation and obsolescence. But I digress. > Yes, we also would want to invoke rustc ourselves, at least for the > project's (as opposed to dependencies) source code. Yeah, this is probably preferable. > I've been thinking about this (and this idea is also echoed in the > issues you have linked): it feels like we need some sort of "foreign > package" support in build2 that can be used to consume dependencies > from other package managers. In principle, build2 has one already: pkg-config. The problem is I guess this is mostly only useful for C/C++(/fortran?) packages, and not all C/C++ projects provide pkg-config files. So yeah, you're probably right that there needs to be a way to consume packages from other package managers. > For starters I am focusing on (1) but we will want to figure out (2) > at some point as well. There is only so far you can get in Rust without > external crates. Sounds reasonable. > I also want to explore the mixed (e.g., C/C++ and Rust) applications more. Yeah, that would be useful, especially for projects which are transitioning from C/C++ to Rust (e.g. librsvg). > Interestingly, I looked into how Bazel (currently) does it, and it's the same idea > (i.e., invoke rustc but instruct it to use our linker/options). Sounds reasonable. > Yes, and I believe this is true for Debian as well. Good to know. > Thanks for the links. No problem. > Crate dependency discovery: https://internals.rust-lang.org/t/crate-dependency-discovery/11418 Interesting read. Seems like this is still unresolved, though, and it's unclear if there's interest in improving the dependency discovery. > Cross-language (C/C++ and Rust) LTO: http://blog.llvm.org/2019/09/closing-gap-cross-language-lto-between.html I've seen this, and it's good to see someone working on it, but unless I'm mistaken, the GCC LTO is more comprehensive (i.e. whole program) than LLVM LTO[4], so it'd be nice to have the same capability in GCC. [1] https://github.com/sapir/gcc-rust/tree/rust#how-does-it-work [2] https://www.bountysource.com/issues/86138921-rfe-add-a-frontend-for-the-rust-programming-language [3] https://www.circle-lang.org/ [4] http://hubicka.blogspot.com/2018/12/even-more-fun-with-building-and.html From boris at codesynthesis.com Thu Jan 23 11:01:53 2020 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu, 23 Jan 2020 13:01:53 +0200 Subject: [build2] Rust In-Reply-To: References: Message-ID: Matthew Krupcale writes: > > I've been thinking about this (and this idea is also echoed in the > > issues you have linked): it feels like we need some sort of "foreign > > package" support in build2 that can be used to consume dependencies > > from other package managers. > > In principle, build2 has one already: pkg-config. The problem is I > guess this is mostly only useful for C/C++(/fortran?) packages, and > not all C/C++ projects provide pkg-config files. So yeah, you're > probably right that there needs to be a way to consume packages from > other package managers. pkg-config is not really a package manager, it's just a way to query library metadata that has been built and installed in some unspecified way. What I am thinking about is the ability to specify (say, in the manifest) that libfoo comes from a "foreign" package manager/repository and have bpkg automatically call that manager to build it. But I guess this wouldn't help distribution maintainers much since they replace the bpkg's functionality with the distribution's package manager. And that means they will have to deal with the "foreign" manager as well. > > Crate dependency discovery: https://internals.rust-lang.org/t/crate-dependency-discovery/11418 > > Interesting read. Seems like this is still unresolved, though, and > it's unclear if there's interest in improving the dependency > discovery. Yes. One nice thing I noticed about Rust (and you can see this quite a bit in this thread), is that the community is not afraid to admit limitations or deficiencies. From mkrupcale at matthewkrupcale.com Thu Jan 23 13:46:22 2020 From: mkrupcale at matthewkrupcale.com (Matthew Krupcale) Date: Thu, 23 Jan 2020 08:46:22 -0500 Subject: [build2] Rust In-Reply-To: References: Message-ID: On Thu, Jan 23, 2020 at 6:02 AM Boris Kolpackov wrote: > > pkg-config is not really a package manager, it's just a way to query > library metadata that has been built and installed in some unspecified > way. I didn't really mean that it's a package manager in and of itself, but it does allow one to consume packages produced by other build systems once installed. > What I am thinking about is the ability to specify (say, in the > manifest) that libfoo comes from a "foreign" package manager/repository > and have bpkg automatically call that manager to build it. I see. To clarify, you're talking about a level above the build system, so the foreign package manager might itself use some alternative build system (in the case of non-binary, source packages), right? That is, this is separate from bpkg invoking other build systems directly? > But I guess this wouldn't help distribution maintainers much since they > replace the bpkg's functionality with the distribution's package manager. > And that means they will have to deal with the "foreign" manager as well. Yeah, distros will likely use the package manager / build system of each project directly. That is unless perhaps bpkg becomes so comprehensive/useful that they could replace all of their macros for each system with a single set of macros for bpkg. > Yes. One nice thing I noticed about Rust (and you can see this quite > a bit in this thread), is that the community is not afraid to admit > limitations or deficiencies. Sure, acknowledging the limitations is good, but it would also be better to address them and try to resolve them. Like I said, it was unclear if they see this as enough of an issue to be worth resolving since they think the parallelism can happen on the crate/package level versus the source file level. Even then, as you said, this does not deal with inter-crate dependencies.