My last blog post got some interest on the permacomputing mailing list, especially as it relates to designing a personal or permacomputing build system. I thought I would give a quick overview of the ../lua/civ build system as well as explore those talked about by the participants, as well as ideas that have been brewing in my brain about a civ package manager.
For reference, here are the build systems presented. For the most part I am not going to give targeted criticism but rather lay out what I believe a good build system needs to accomplish. Most of my criticism will be leveled at bazel and Make:
I once had a fresh-from-college engineer ask me about some problem related to CAD files he was dealing with. He spent 5-10 minutes trying to explain all the minutae of his particular CAD tooling, how certain pieces of representation are difficult to script and how he got around it, etc etc. I know almost nothing of CAD, so was very little help with the technical details but I did ask him one question:
What are your inputs and what are your outputs?
With this one question he completely reframed the problem. He quickly began to untangle the web of complexity he had formed and started to discuss technical details of how things were cleaner, simpler, easier than he had realized. Knowing nothing of CAD, I could only smile – but that was enough.
So what is a build system? Civ's answer is that it is a program which takes as input source code (files), binaries (compilers), and configuration (flags, config files) and outputs files, binaries and configuration according to a specific folder structure, which can then be executed on a target platform.
So the first principle of a good build system is this: all inputs and outputs of each phase of the build must somewhere be explicit.
This is where a program like Make fails miserably – Make is a effectively a collection of shell scripts in an obtuse, platform-specific syntax. It defines each "rule" as a collection of inputs and outputs while failing to offer an overarching vision on how to stitch them together.
This is the next principle of a good build system: It must be posible to programatically configure the inputs and outputs to adapt to different target systems.
In essense, a good build system gives the correct layer of abstraction between the source code and the program actually built – as simple as possible but no simpler.
Civ's answer is this: relegate the decisions of where files go and how to build them to something a user can configure centrally – effectively a build library – which has a minimal and well-defined API but who's internals can be swapped out for different platforms. In civ, this is called the sys hub. For instance cc.luk (luk is lua configuration files, a sandboxed lua process with slightly different globals) contains the logic of converting the user's cc{hrd='foo.h'} into the object sent to sys/cc/build.lua the script actually executed to build the program.
This accomplishes two things simultaniously:
This is effectively what Bazel (Google's build system) does so well, but while Bazel is a huge and complex monolith of software which decided to fork python (of all things) for it's config language, civ is written in a 1000 line cmd utility libraries which spawns a hermetic lua to interpret configs using a ~100 line lua library.
Then someone else can depend on your mylib by using the dep of myhub:path/to:mylib. What is this "hub" you might ask? When you run civ init it creates a ~/.config/civ.luk file which contains a map of hubs, here is mine:
These keys currently point to local directories but they could point to anything, including repositories. This is the top layer, the "override" layer of hubs – but we can add one abstraction and we will have created a package manager: the hubrepo.
The hubrepo is simply a VCS (e.g. git) repo containing the following directories:
When a user uses a hub: for a dependency, it first looks in the .config/civ (or manually specified config), then in the hubrepo which is also specified in that config. If the hub is remote it then downloads it. The user's project can lock a specific version (commit tag) of each hub in a .civ.lockfile to avoid sudden dependency drift and allow them to control when updates are done.
Conversly, when a user wants to submit code to the hub they have to create a CL on github which gets auto merged by a bot. This must:
That's really it – there is nothing more needed.
Thanks for reading, feel free to discuss on the civboot mailing list or wherever you found this link.
Source: Hacker News — This article was automatically imported from the source. Read full article at original source →
