Requesting Build Outputs

Now we'll finally build some code.

By the end of this section, you'll be able to get desired outputs with bazel build and execute some application with bazel run for at least the language you picked.

Exercise: let's build something

Instructor chooses some code in the examples repo to demonstrate.

This may be where you discover errors in the BUILD files that weren't evident earlier when you only analyzed, as this is the first time we're running the tools.

Make some minor change to the sources and build again. Can you get some cache hits in this second build?

Bazel prints the paths to the resulting output files.

Aspect CLI only

You can predict where the outputs will be written, if a script wants to do something with them:

bazel help outputs

Actions

When Bazel needs to transform inputs to outputs, it does it by spawning an Action, which is just a subprocess invoking some tool.

The Action Graph

In the Analysis phase, the dependency graph is "lowered" to an action graph. In the action graph, each node is a subprocess to spawn (invoking some tool) and the edges are files and Providers which are output by one action and needed as inputs to another.

The graphs are NOT one-to-one! For example, a ts_project rule with a custom transpiler produces several actions.

Exercise: bazel analyze

Bazel doesn't actually have a command called analyze. It's spelled "build --nobuild" instead. This command is rarely useful. You might use it if you're making a big, breaking refactoring, so that you can resolve all the analysis failures first before attempting to build anything. You could also use it to reason about what is the slow step in your CI pipeline.

bazel build --nobuild //...

Querying the action graph

This is a valuable skill when debugging a failure of some rule, especially when required inputs aren't declared.

You can run arbitrary starlark programs on the action graph with --output=starlark which is a powerful tool.

Exercise: bazel aquery

• What are the declared input files to the compile action for a library target you've created?
• What providers are produced by the library target? (You'll need a tiny Starlark program)

The Action Cache

Earlier, we introduced the Repository Cache (actually a downloader cache). This is entirely separate from the Action Cache, because only actions are hermetic and know all their inputs, and therefore a correct cache key can be computed.

When a file in bazel-out is replaced, then Bazel normally needs to run the action again to restore it. You can have an "L1" cache locally on the machine to avoid this - --disk_cache is simple, but grows unbounded since there is no eviction so beware!

The cache key is typically dependent on the execution platform. This makes it very hard to share intermediate results between a MacOS development machine and a Linux CI system. Non-determinism will cause cache misses, because the cache key changes anytime a dependency produces a different output. Hash ordering and timestamps are typical sources of non-determinism.

Spawn Strategies

Bazel can take several approaches to execute an action. See https://bazel.build/docs/user-manual#execution-strategy

• sandboxed causes commands to be executed inside a sandbox on the local machine.
  • This requires that all input files, data dependencies and tools are listed as direct dependencies in the srcs, data and tools attributes.
  • Bazel enables local sandboxing by default, on systems that support sandboxed execution.
• local causes commands to be executed as local sub-processes.
• worker causes commands to be executed using a persistent worker, if available.
• docker causes commands to be executed inside a docker sandbox on the local machine.
  • This requires that docker is installed.
• remote causes commands to be executed remotely; this is only available if a remote executor has been configured separately.
• dynamic is a special value to try both remote and local execution, take the first to complete.
  • See https://bazel.build/remote/dynamic

Inspecting the sandbox

How to find it: run Bazel with the --sandbox_debug flag. It will print lots of extra information, and buried in there you can find paths beneath the temporary sandbox folder. This flag also causes Bazel to skip the cleanup and removal of the sandbox folder.

Checking in generated files

Bazel output files are always written beneath the bazel-out tree. But other tools allow, or even expect, that files are present in the source tree. We can un-break the situation if we relax the Bazel dogma around writing to the source tree.

For example, in the logger/schema folder we should have a ts_proto_library target which produces a logger_pb.d.ts interface definition file. The editor expects to read that file in order to provide valuable completion (intellisense) to a developer navigating the API. ts_proto_library is a macro which automatically writes this file to the source tree, which we can observe by deleting it and then running the tests in that package.

There are options to "teach the editor" instead. In Go for example, it's possible to use the GoPackagesDriver to teach the editor to look in bazel-out for logger.pb.go. But, this is a difficult, brittle configuration to setup on developer machines. And then you'll just have the next tool or language where the bazel-out location breaks things.

Of course, bazel build itself cannot result in files in the source tree, but we can get a close approximation with a rule called write_source_files.

It uses a simple pattern:

• the build target writes to a file like bazel-out/pkg/foo.pb.go
• a generated test target asserts that pkg/foo.pb.go in the source folder has the same content
• a generated executable copies files from bazel-out back to the source tree
• when the test fails, it prints an instruction how the developer can run the executable

This is the same pattern we use for "golden" or "snapshot" files which live in the source tree.

Read more about this pattern:

https://blog.aspect.build/bazel-can-write-to-the-source-folder

Exercise: Running programs

Running programs in Bazel is just a syntax sugar.

It really means "build this program, then spawn the resulting executable".

info

Run bazel help run

You should be able to run one of the programs you've built so far, in whichever language you have working. Bazel's naming convention is that any *_binary target should be executable, so you can bazel run it. Some other targets may be executable as well, so refer to the documentation, such as oci_load from rules_oci.

Watch mode

It's really nice to have a "live" development server where code changes are immediately visible. To make this possible, we need Bazel to run in "watch mode" where changes to sources are automatically reflected in the bazel-out tree without us having to manually run bazel after every edit.

Sadly, Bazel doesn't have a watch mode built-in, but bazel-watcher does. That project releases a tool named ibazel, short for "interactive Bazel".

Our example repo has this installed for you in the tools folder. You can read more about the technique used: https://blog.aspect.build/run-tools-installed-by-bazel

It can also be installed in these alternative ways:

• Mac: brew install ibazel
• If you have npm installed: npm install -g @bazel/ibazel
• Download a binary from the releases page and install it as ibazel on your $PATH.

info

We plan to include this feature directly in a future release of Aspect CLI

Exercise: A live devserver

Let's practice using it for the frontend, since JavaScript engineers are accustomed to having watch mode. If you didn't write a BUILD file for the frontend, you can checkout the packages branch.

The target we run has to be aware of the Bazel-watcher protocol:

• The target must be tagged to indicate its awareness: tags = ["ibazel_notify_changes"] This tag prevents the binary being restarted every time the files change.
• The program must read from stdin to find out when a build has finished. Bazel-watcher will write lines like IBAZEL_BUILD_COMPLETED SUCCESS

js_run_devserver is one rule that is aware of the protocol.

$ ./tools/ibazel run logger/frontend

Click the link to open the site in the browser, then make some edits to logger/frontend/index.ts, for example change the Get Server Logs text that appears on the button.