summary: "bzlmod makes bazel extremely appealing and isn't hard to grasp for anyone already familiar with go modules. My frustration with make for complex builds led me to bazel."
I write programs to solve problems. Most of the time these are pretty personal and only get used once or twice, never see the light of day again, and that's fine.
Lately, though, I've been working on tooling for my Obsidian notes and I want to make my tools as accessible as possible. This involves a couple steps that are particularly important, tedious, and error prone when done manually:
- trying to cross compile my binaries for a relatively long list of cpu/arch combinations
- build a docker image
- push that docker image to OCI image repositories
- run tests
- run benchmarks
- cache builds effectively
I've started with a Makefile I stole from some gist I didn't save a link to. This [makefile](https://code.ndumas.com/ndumas/wikilinks-parser/src/tag/v0.0.5/Makefile) is kinda hefty so I'm gonna focus on compiling go binaries and preparing OCI images that contain those binaries.
This makefile's extremely opinionated, hyper-targeted at Go builds. It assumes that your binaries live in `cmd/binaryName/`.
```Makefile
# Parameters
PKG = code.ndumas.com/ndumas/obsidian-markdown
NAME = parse-wikilinks
DOC = README.md LICENSE
DISTDIR ?= $(WD)/dist
CMDS := $(shell find "$(CMDDIR)/" -mindepth 1 -maxdepth 1 -type d | sed 's/ /\\ /g' | xargs -n1 basename)
INSTALL_TARGETS := $(addprefix install-,$(CMDS))
VERSION ?= $(shell git -C "$(MD)" describe --tags --dirty=-dev)
COMMIT_ID := $(shell git -C "$(MD)" rev-parse HEAD | head -c8)
$(GOBUILD) "$(CMDPKG)/$@" | tee "$(RPTDIR)/build-$@.out"
install: $(INSTALL_TARGETS)
$(INSTALL_TARGETS):
$(GOINSTALL) "$(CMDPKG)/$(subst install-,,$@)"
dist: clean build
for docfile in $(DOC); do \
for dir in "$(DISTDIR)"/*; do \
cp "$(PKGDIR)/$$docfile" "$$dir/"; \
done; \
done
cd "$(DISTDIR)"; for dir in ./*linux*; do $(GZCMD) "$(basename "$$dir").tar.gz" "$$dir"; done
cd "$(DISTDIR)"; for dir in ./*windows*; do $(ZIPCMD) "$(basename "$$dir").zip" "$$dir"; done
cd "$(DISTDIR)"; for dir in ./*darwin*; do $(GZCMD) "$(basename "$$dir").tar.gz" "$$dir"; done
cd "$(DISTDIR)"; find . -maxdepth 1 -type f -printf "$(SHACMD) %P | tee \"./%P.sha\"\n" | sh
$(info "Built v$(VERSION), build $(COMMIT_ID)")
```
Because this isn't a makefile tutorial, I'm going to just hit the high notes and explain why this isn't working. Given the parameters at the top, it looks in `cmd/` for directories and passes them to `go build` with `-ldflags` thrown in.
Here we have the machinery behind `make bump`, github link below. `bump` is a tool that'll automatically create semantic versioning tags in a git repo based on existing tags. You can `bump {patch,minor,major}` and it'll create the next tag in the versioning sequence for you.
```Makefile
setup-bump:
go install github.com/guilhem/bump@latest
bump-major: setup-bump
bump major
bump-minor: setup-bump
bump minor
bump-patch: setup-bump
bump patch
```
## Why does it work?
Automation is a great thing. This makefile inspired me to start actually using semantic versioning diligently. It didn't hurt that I was working on a lot Drone pipelines at the time and was starting to get incredibly frustrated debugging `:latest` images and never being certain what code was running.
Working with bash is never...pleasant, but it definitely gets the job done. I'm no stranger to shell scripts and the minor modifications needed to get `bump` integrated and other miscellany I've legitimately fully forgotten by now ( document your code for your own sake ) posed no real burden.
This makefile helped me rapidly iterate on code and release it in a way that was easily consumable, including docker images pushed to my self-hosted registry on Gitea. The pipeline that handles this blog post is using a docker image tagged by the makefile components described above, in fact.
## Why doesn't it work?
The real kink in the hose ended up being gox. Gox worked great until I tried to generate alpine builds. It was possible, but I'd have to start changing the makefile pretty significantly, write bash helper functions, and more. I decided that wasn't worth the maintenance overhead pretty quickly and started looking in
it's not "smart". The solutions for cross-compilation ended up being clunky to compose with Docker builds
## What are the options?
The only real solution is a smarter build system. I had to choose between hand-rolling something with a bunch of switch statements in bash, or I could look into more modern toolkits. I looked into three:
- meson
- bazel
- scons
## The contenders
Bazel looked like it had the most to offer:
- hermetic builds
- reproducible builds
- aggressive, fine-grained caching
- extensible
All of these fit the bill for what I needed. In particular, it has pretty decent go support through [rules_go](https://github.com/bazelbuild/rules_go) and [gazelle](https://github.com/bazelbuild/bazel-gazelle), which we'll look at in more depth later.
There's not a lot to say here, I knew nothing about any of the three candidates and when I started I wasn't certain I'd stick with bazel all the way. Sometimes you just have to try stuff and see how it feels.
### Caution
bazel seems to be going through an ecosystem shift from the WORKSPACE paradigm to bzlmod. Documentation does exist, but it might not be in the README yet. I've tested the code here and it works in this narrow case. Caveat emptor.
## Getting Going with Gazelle
With that, here is how a modern bzlmod enabled go repo is born.
### Building Go code
The first step is, in no particular order, init your git repository and init your go module. The former is helpful for keeping track of when you broke something and the latter is required for gazelle to do its job.
-`go mod init`
-`git init`
Write your go code. The simplest hello world will work for demonstration purposes.
Create your `MODULE.bazel` file.
``` {title="MODULE.bazel"}
module(
name = "obsidian-markdown", # set this manually
repo_name = "code.ndumas.com_ndumas_obsidian-markdown", # this is the name of your go module, with /'s replaces with _'s
`module()` is how you declare a top-level bazel project. Everything is namedspaced under this module.
`bazel_dep` tells bazel to retrieve modules from the [bazel registry](https://registry.bazel.build/).
`use_extension` imports functions from bazel modules; here we're importing `go_deps` because it'll read out `go.mod` file and help bazel automatically calculate direct and transitive dependencies.
and `BUILD.bazel`
``` {title="BUILD.bazel"}
load("@gazelle//:def.bzl", "gazelle")
gazelle(name = "gazelle")
gazelle(
name = "gazelle-update-repos",
args = [
"-from_file=go.mod",
"-to_macro=deps.bzl%go_dependencies",
"-prune",
],
command = "update-repos",
)
```
This is straight from the gazelle README. You `load()` the gazelle module and declare two build targets: `gazelle` and `gazelle-update-repos`. After the rest of the setup, these targets are what will do the work of actually generating build/test targets for all your code.
# Artifacts are typically placed in a directory called "dist"
# Be aware that this setup will still create a bazel-out symlink in
# your project directory, which you must exclude from version control and your
# editor's search path.
build --symlink_prefix=dist/
###############################
# Output #
###############################
# A more useful default output mode for bazel query, which
# prints "ng_module rule //foo:bar" instead of just "//foo:bar".
query --output=label_kind
# By default, failing tests don't print any output, it's logged to a
# file instead.
test --test_output=errors
```
Only the first line is required; the rest are just conveniences. I do **strongly** recommend the `query` setting though, extremely nice for debugging.
Finally, a `.gitignore` to mask out generated artifacts.
``` {title=".gitignore"}
dist/*
reports/*
bazel-*
*.bazel.lock
```
Run `bazel build //:gazelle`. This will auto-generate a lot of scaffolding, and probably emit a `buildozer` command that will modify something. This is the build system (specifically gazelle ) automatically detecting dependencies that are declared in `go.mod` but not in your bazel code.
```bash
$ bazel run //:gazelle
WARNING: /home/ndumas/work/gomud/MODULE.bazel:8:24: The module extension go_deps defined in @gazelle//:extensions.bzl reported incorrect imports of repositorie
s via use_repo():
Not imported, but reported as direct dependencies by the extension (may cause the build to fail):
com_github_therealfakemoot_go_telnet
** You can use the following buildozer command(s) to fix these issues:
This diff shows how bazel references external dependencies. gazelle's `go_deps` tool acts as a provider for these lookups and offers information bazel needs to verify its build graphs. Yours may look different depending on what you've imported, if anything.
Examining the produced `BUILD.bazel` file should yield something like this for a `main` package.
These are examples of `rules_go` build targets. These do a bunch of magic to invoke Go toolchains and in theory let bazel cache builds at a pretty granular level. I'm hoping this is true, I've got a few pipelines that are starting to run way longer than I like.
### OCI Images
For ease of use, I like to build docker images containing my packages. This is particularly important for Drone pipelines.
We're gonna amend our `MODULE.bazel` to add some new tools.
``` {title="MODULE.bazel"}
bazel_dep(name = "rules_oci", version = "1.3.1") # gives us ways to interact with OCI images and repositories
bazel_dep(name = "rules_pkg", version = "0.9.1") # exposes a way to tar our app, which is necessary for packing with rules_oci
tag = "latest", # This is temporary. For reproducible builds, you'll want to use digest hashes.
)
use_repo(oci, "distroless_base")
```
`pull()` does more or less what it says: it creates a target that represents an OCI image pulled from a registry, and another `use_repo()` call tells bazel that we're *using* our image.
And add this to the `BUILD.bazel` file for the binary you want built into an OCI image
``` {title="cmd/echo/BUILD.bazel"}
load("@rules_pkg//:pkg.bzl", "pkg_tar")
pkg_tar(
name = "tar",
srcs = [":echo"],
)
load("@rules_oci//oci:defs.bzl", "oci_image")
oci_image(
name = "image",
base = "@distroless_base",
entrypoint = ["/echo"],
tars = [":tar"],
)
```
`oci_image` requires that whatever you package into the image it creates be contained in a tar file, which seems pretty reasonable. `rules_pkg` handles that for us.
Run `bazel build //cmd/echo:image` and you'll see another `buildozer` command and a lot of errors. This is to be expected, bazel wants builds to be reproducible and because we haven't specified a version or a hash it can't do that. It helpfully emits the `buildozer` command that'll set the proper digest hash and platforms bazel needs to resolve its builds.
```
bazel build //cmd/echo:image
WARNING: fetching from https://gcr.io/v2/distroless/base/manifests/latest without an integrity hash. The result will not be cached.
WARNING: for reproducible builds, a digest is recommended.
Either set 'reproducible = False' to silence this warning,
or run the following command to change oci.pull to use a digest:
(make sure you use a recent buildozer release with MODULE.bazel support)
oci~distroless_base//:distroless_base depends on @rules_oci~1.3.1~oci~distroless_base_single//:distroless_base_single in repository @rules_oci~1.3.1~oci~distro
less_base_single which failed to fetch. no such package '@rules_oci~1.3.1~oci~distroless_base_single//': gcr.io/distroless/base is a multi-architecture image,
so attribute 'platforms' is required.
ERROR: Analysis of target '//cmd/echo:image' failed; build aborted:
INFO: Elapsed time: 2.434s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (27 packages loaded, 341 targets configured)
```
A diff after running should show something like this:
INFO: Build completed successfully, 17 total actions
```
Pushing our image to a repository ends up being relatively simple after all the legwork. The diff below shows the full changes but in summary
- change the `load()` call for `rules_oci`. It's variadic and takes an arbitrary number of arguments indicating names to import. Add `oci_push` to the list.
- Use the imported `oci_push` rule to set tags and the destination registry
Running `bazel run //cmd/echo:registry` will push your image, as long as you'd otherwise be able to use `docker push` or similar. You will need to inject authentication details into your build pipelines, etc.
And with that, you've got an OCI image pushed to your repository of choice. Note that bazel relies on the environment to provide an OCI toolchain and the authorization. I've got my drone credentials in environment variables, but your setup may vary.
## Success Story???
The next step forward is to take a step backwards: integrate bazel into a Makefile. `make` is actually pretty nice as a task-runner; now that bazel can handle the top-to-bottom process of builds, the makefile doesn't need much, if any, logic in it. All it'll have to do is serve as fancy aliases for bazel invocations.
I also haven't actually set up cross-compilation. Work for another day.
### Useful Tips
#### Supported go compilation targets
I haven't used this one yet, but it's handy for manually cross-compiling.
