= What does a Build Tool do?
// tag::header[]
:author: Li Haoyi
:revdate: 13 February 2025

_{author}, {revdate}_

The most common question I get asked about the Mill build tool is: what does a build
tool even do? Even software developers may not be familiar with the idea: they may run
`pip install` and `python foo.py`, `javac Foo.java`, or `go build` or some other
language-specific CLI directly. They may have a folder full of custom Bash scripts
they use during development. Or they may develop software on a team where someone
else has set up the tooling that they use. This blog post explores what build tools
are all about, why they are important to most software projects as they scale, and
how they work under the hood.


// end::header[]


== What is a Build Tool?

A build tool is a program that automates selecting which tools or tasks to
run - and how to run them - whenever you want to do something in your codebase.

A hello-world program may get by with `javac Foo.java` to compile your code and
`java Foo` to run it, or even a single command `python foo.py`. But for most real-world
projects a single "user facing" task may require a lot of work to be done first. The
job of the build tool is to help decide what work needs to be done to accomplish what
you want, and how best to perform that work with caching and parallelism to complete it
as quickly and as efficiently as possible.

=== Build Tools as Task Orchestrators

For example, in a real-world project, before _running your code_ you may first need to:

- Use the configured _direct_ third-party dependencies to resolve the _transitive_ dependencies
- Download all transitive third-party library dependencies to files on disk
- Compile _internal_ library dependencies, i.e. upstream modules in the same codebase
- Run code generators on IDL files (e.g. https://protobuf.dev/[protobuf], https://www.openapis.org/[openapi], https://thrift.apache.org/[thrift], etc.) to generate source files
- Run the compiler  (if your language has one) on source files to generate binary files
- Package the binary files into an executable
- Run the static asset pipeline on input files to generate static assets (e.g. assets for video games or websites)

So although the developer may have just asked to `run` their code, it is only after all
these tasks are done does it make sense to begin running it! However, it's not as
simple as running every one of the tasks listed above one after another. You also
need to decide _which ones_ to run and _how_ to run them:

- You want to skip any tasks that have been run earlier whose outputs you can re-use to save time
  (e.g. by not generating static assets if they have already been generated)

- You want to make sure to re-run tasks whose inputs have changed, discarding any out-of-date output (e.g.
  if a static input file is changed, then the static asset pipeline needs to be re-run)

- You want to parallelize tasks which are independent of each other (e.g.
  _running the compiler_ can happen in parallel with _running static asset pipeline_)

- You want to _not_ parallelize tasks which depend on one another,
  and thus cannot happen in parallel (e.g. _running the compiler_ must happen after
  _running code generators_, since we need to compile the generated code)

Working on any real-world codebase will involve a large variety of tasks:
resolving and downloading dependencies, autoformatting, linting, code-generation, compiling,
testing, packaging and publishing, and so on. These tasks depend on each other in various,
ways, and it is the job of the build tool to take what you (as a developer) want
to do, and select _which_ tasks to run and _how_ to run them in order to accomplish
what you want.

=== Build Tools across Languages

Although programming languages are very different, developers in all languages have similar
requirements. Thus most languages have one-or-more build tools, that then integrate with
the equivalent external tools and services specific to that language:

|===
| Language   | Build Tool          | Package Repository    | Compiler | ...

| Java
| https://maven.apache.org[Maven], https://gradle.org[Gradle], https://mill-build.org[Mill]
| https://central.sonatype.com/[Maven Central]
| https://docs.oracle.com/javase/8/docs/technotes/tools/windows/javac.html[javac]
| ...

| Python
| https://python-poetry.org/[Poetry]
| https://pypi.org/[PyPI]
| https://github.com/python/mypy[mypy]
| ...

| Javascript
| https://webpack.js.org/[Webpack], https://vite.dev/[Vite]
| https://www.npmjs.com/[NPM]
| https://www.typescriptlang.org/[tsc]
| ...


| Ruby
| https://github.com/ruby/rake[Rake]
| https://rubygems.org/[RubyGems]
| https://sorbet.org/[Sorbet]
| ...

| Rust
| https://doc.rust-lang.org/cargo/[Cargo]
| https://crates.io/[Crates]
| https://doc.rust-lang.org/rustc/what-is-rustc.html[rustc]
| ...

|===

Generally each language's build tools will come with integrations for
the language's other tooling by default: package repositories, compilers, linters, etc.
There are also language agnostic build tools out there like https://en.wikipedia.org/wiki/Make_(software)[Make],
or multi-language build tools such as https://bazel.build/[Bazel] which is optimized
for large monorepos that often contain multiple languages. The Mill build
tool also supports multiple languages: xref:mill:ROOT:javalib/intro.adoc[Java],
xref:mill:ROOT:kotlinlib/intro.adoc[Kotlin], xref:mill:ROOT:scalalib/intro.adoc[Scala],
xref:mill:ROOT:pythonlib/intro.adoc[Python], and (experimentally)
xref:mill:ROOT:javascriptlib/intro.adoc[Javascript], allowing it to be used
for https://mill-build.org/mill/main-branch/large/multi-language-builds.html[Multi-Language Builds].

=== Build Tools for Small Codebases

Most projects start off small, and while they _can_ use a build tool, do not _need_ one.
A single-file program or script can get by with using its language's built-in CLI
to compile/run/test their code, and for small codebases these workflows are
fast enough that parallelization or caching are not needed.

Nevertheless, even in a small project, a simple task like _"installing third party
dependencies"_ can be surprisingly challenging. Consider
this scenario working with a simple Python script:

- You start off running `pip install something`, run your code with `python foo.py`. Great!

- The project bumped the version of `something` required, and now everyone working on
  the project must `pip install` it again, or else they will get weird errors from
  having the wrong version installed

- If you checkout an old release branch of the project, you need
  to remember to `pip install` to go back to the old version of `something`. And then `pip install`
  the new version when you come back to the `main` branch later!

- What if you end up with two scripts, `foo.py` and `bar.py`, that each requires different versions
  of the same `something`? Now you need to remember to manually `python -m venv` to create
  a virtual environment for each script, and remember to update each virtualenv
  whenever that script's required `something` version changes.

As you can see, even for the single problem of _"installing third-party dependencies"_,
keeping everything in sync and everyone's laptops up to date can be a real headache!


Developer workflows tend to get more complex and slower over time as projects
progress and codebases grow. This can end up being a significant burden on the
project's developers, and slow down the project overall:

* Each project developer ends up spending mental energy to juggle a dozen different tools
  to run linters, code-generation, autoformatting, unit-testing, integration-testing, etc.,
  making sure to run them in the right order for things to work.

* They also end up spending time just _waiting_, as their growing codebase
  takes longer and longer to compile, and their growing test suite takes
  longer and longer to run

Thus, even in small codebases it often makes sense to use a build tool. The manual
work necessary to get a small codebase running can be surprisingly non-trivial,
and as the codebase grows (as they tend to do) the build tool becomes even more
necessary over time.


=== Build Tools vs Build Scripts

One common alternative to build tools is to write scripts to help manage the development
of the project: e.g. you may find a `dev/` folder full of scripts like `run-tests.sh` or
`lint.sh`, and so on. These scripts may be written in Bash, Python, Ruby, Javascript, and so on.
While this works, the approach of scripting your development
workflows directly in Bash runs into trouble as the codebase grows:

- Initially, these scripts serve to just enumerate the necessary steps to do something
  in your code, e.g.

** `integration-tests.sh` may require `download-dependencies.sh`,
   `codegen.sh`, `compile.sh`, `package.sh` to be run in order first

** `unit-tests.sh` on the other hand may need `download-dependencies.sh`, `codegen.sh`,
   `compile.sh`, but not need  `package.sh`

- Next, these scripts inevitably begin performing rudimentary caching: e.g. if you
  ran `unit-tests.sh` earlier and now want to run `integration-tests.sh`, we can skip
  `download-dependencies.sh`, `codegen.sh`, `compile.sh`, but need to now run `package.sh`

- Eventually, these scripts inevitably start parallelizing parts of the workflow:
  e.g. `codegen.sh` and `download-dependencies.sh` may be able to run in parallel, while
  `compile.sh` can only run after both of those are finished. Your `unit-tests` and
  `integration-tests` may also run in parallel to save time.

At this stage, your scripts have their own ad-hoc dependency-management, caching,
and parallelization engine! Because your main focus is on your _actual_ project,
this ad-hoc engine will never be in great shape: the performance won't be optimal, the
error messages and usability won't be great, and bugs and issues will tend to linger.
This will be a drag on your productivity, since even if your focus is on
your main project, you still need to interact with your build scripts constantly
throughout the work day.

At its core, a build tool basically automates these things that you would have
implemented yourself anyway: it provides the ordering of tasks, parallelism,
caching, and probably does so better than you could implement in your own
ad-hoc build scripts. So even though any developer should _be able_ to wrangle
Bash or Python enough to get the ordering/parallelism/caching they need, they probably
shouldn't _actually do it_ and just use an off-the-shelf build tool which
has all these problems already solved.

=== Build Tools as Custom Task Runtimes

Most codebases have some amount of custom tasks and workflows. While many workflows
are standardized - e.g. using the same Java compiler, Python interpreter, etc. -
it is almost inevitable that over time the codebase will pick up workflows unique
to its place in the business and organization. For example:

- *Custom code generation*, to integrate with some internal RPC system no-one else uses
- *Custom linters*, to cover common mistakes that your developers tend to make
- *Custom deployment artifacts*, to deploy to a new cloud platform that hasn't become popular yet

The default way of handling this customization is the aforementioned
folder-full-of-scripts, where you have a `do-custom-thing.sh` script to run
your custom logic. While this does work, it can be problematic for a number of reasons:

1. *Bash scripts are not an easy programming environment to work in*, so
   custom tasks implemented as scripts tend to be buggy and fragile.
   Even implementing logic like "if-else" or "for-loops" in Bash can
   be error-prone and easy to mess up!

2. *Non-Bash scripting languages tend to have portability issues*: e.g. Python
   scripts tend to be difficult to run reliably on different machines which
   may have different Python versions or dependencies installed, and Ruby
   scripts may have issues running on Windows.

3. *You usually want caching and parallelism* in your custom tasks in order to
   make your workflows performant, and correctly and efficiently implementing a
   caching/parallelization engine in Bash (or some other scripting language)
   can be quite a challenge!

Most build tools thus provide some kind of _"plugin system"_ to let you
implement your custom logic in a more comfortable programming environment
than Bash: Maven's https://maven.apache.org/plugin-developers/[MOJO] interface interface
lets you write plugins in Java, Webpack allows you to write https://webpack.js.org/plugins/[Webpack Plugins]
in Javascript, Bazel provides the https://bazel.build/rules/language[Starlark Language]
for writing extensions, and so on. The Mill build tool's custom logic is
xref:mill:ROOT:depth/why-scala.adoc[written in Scala] and runs on the JVM, and
thus comes with typechecking, IDE support, access to the standard JVM libraries
and package repositories and people are already used to.

How custom tasks and workflows are written does not matter for
small projects where customizations are trivial. But in larger projects with
a non-trivial amount of custom logic, it becomes very important. Providing
a safe, easy-to-use way to customize your build is thus a big benefit of using
a build tool, one that gets increasingly important as the project grows.

=== Build Tools for Large Codebases and Monorepos

Twice now we've mentioned that build tools get more important as projects grow,
so it's worth calling out the need for build tools on very-large-codebases:
those with 100 to 1,000 to even 10,000 developers actively working on it.
Perhaps 1,000,000s or 10,000,000s of lines of code. Very-large-codebases have an acute
need for something to help manage the development workflows, which "monorepo"
build tools like Bazel provides. Some examples of "large codebase" requirements are:

- xref:3-selective-testing.adoc[Selective Testing], to avoid running the entire test
  suite (which may take hours) by only running the tests related to a change.
  e.g. Bazel supports this via the third party https://github.com/Tinder/bazel-diff[bazel-diff]
  package, while Mill supports this built-in as xref:mill:ROOT:large/selective-execution.adoc[Selective Test Execution]

- Multi-language support: e.g. a Java server with a Javascript frontend with a Python
  ML workflows. e.g. Bazel has `rules{lang}` for a wide variety of languages (e.g.
  https://github.com/bazelbuild/rules_java[rules_java], https://github.com/bazelbuild/rules_python[rules_python],
  https://github.com/bazel-contrib/rules_go[rules_go]), and Mill
  has support for https://mill-build.org/mill/main-branch/large/multi-language-builds.html[Multi-Language Builds]

- Distributed caching and execution: allowing different CI machines or developer
  laptops to share compiled artifacts so a module compiled on one machine and be
  re-used on another, or submitting a large workflow to a cluster of machines to
  parallelize it more than you could on a single laptop. This has traditionally
  been something only https://bazel.build/remote/caching[Bazel supports], though
  over time more build tools are adopting these techniques

In large codebases, using a build tool is no longer an optional nice-to-have, but becomes
table-stakes. In large codebases you _need_ the parallelism and caching that a
build tool provides, otherwise you may end up waiting hours to compile and test
each small change. You _need_ the ability to customize and extend the build logic,
in some way that doesn't become a rat's nest of shell scripts. You _need_ to build
3-5 different languages in order to generate a single web-service or executable.
This is where tools like https://bazel.build/[Bazel], https://www.pantsbuild.org/[Pants],
or https://buck.build/[Buck] really shine, although other tools like
https://gradle.org/[Gradle] or https://mill-build.org/[Mill] also support some
of the features necessary for working with large codebases.

See the following blog post for a deeper discussion on what features a
_"monorepo build tool"_ provides and why they are necessary:

* xref:2-monorepo-build-tool.adoc[Why Use a Monorepo Build Tool?]

== How Build Tools Work: The Build Graph

After all this talk about what a build tool is and why you would need one,
it is worth exploring how most modern build tools work. At their core,
most modern build tools are some kind of graph evaluation engine.

For example, consider the various tasks we mentioned earlier:

- Use the configured _direct_ third-party dependencies to resolve the _transitive_ dependencies
- Download all transitive third-party library dependencies to files on disk
- Compile upstream _internal_ library dependencies
- Run code generators on IDL files to generate source files
- Run the compiler on source files to generate binary files
- Package the binary files to generate executable
- Run the static asset pipeline on input files to generate static assets

We might even include a few more:

- Run unit tests on binary files to generate a test report
- Run integration tests on executable to generate a test report

At their core, most build steps are of the form

- Run *TOOL* on *INPUT1*, *INPUT2*, ... to generate *OUTPUT*

Which can be visualized as a node in a graph

[graphviz]
....
digraph G {
  rankdir=LR
  node [shape=box width=0 height=0]
  input1 -> tool
  input2 -> tool
  tool -> output
  output[shape=none]
}
....

If we consider the tasks we looked at earlier, it might form a graph as shown below,
where the boxes are the tasks, non-boxed text labels are the input files, and the
arrows are the dataflow between them

[graphviz]
....
digraph G {
  rankdir=LR
  node [shape=box width=0 height=0]
  direct_deps -> resolve_deps -> compile
  code_gen -> compile
  sources -> compile
  static_input_files -> asset_pipeline
  asset_pipeline -> integration_test
  compile -> unit_test
  compile -> package

  package -> integration_test
  direct_deps [shape=none]
  sources [shape=none]
  static_input_files [shape=none]
}
....

=== Ordering on the Build Graph

It is from this graph representation that most build tools are able to work
their magic. For example, if you ask to run `unit_test` (blue), then the build tool
can traverse the graph edges (red) to find it needs to ensure `compile`, `code_gen`, and
`resolve_deps` need to be run (red)

[graphviz]
....
digraph G {
  rankdir=LR
  node [shape=box width=0 height=0]
  direct_deps -> resolve_deps
  resolve_deps -> compile   [color=red penwidth=2]
  code_gen -> compile  [color=red penwidth=2]
  sources -> compile
  static_input_files -> asset_pipeline
  asset_pipeline -> integration_test
  compile -> unit_test  [color=red penwidth=2]
  compile -> package

  package -> integration_test
  direct_deps [shape=none]
  sources [shape=none]
  static_input_files [shape=none]
  resolve_deps [fillcolor=lightpink style=filled]
  code_gen [fillcolor=lightpink style=filled]
  compile [fillcolor=lightpink style=filled]
  unit_test [fillcolor=lightblue style=filled]
}
....

Furthermore, the build tool is able to figure out what tasks need to run in what
order, simply by doing a topological sort of the subset it needs. In this case,
it knows that `resolve_deps` and `code_gen` must both finish running before
`compile` can begin, and `compile` must finish running before `unit_test` can begin.

=== Caching and Invalidation via the Build Graph



If you then subsequently ask to run `integration_test`, the build tool can see that
`compile`, `code_gen`, and `resolve_deps` were run earlier and can be re-used (green)
while `package` and `asset_pipeline` need to be run

[graphviz]
....
digraph G {
  rankdir=LR
  node [shape=box width=0 height=0]
  direct_deps -> resolve_deps
  resolve_deps -> compile [color=red penwidth=2]
  code_gen -> compile  [color=red penwidth=2]
  sources -> compile
  static_input_files -> asset_pipeline
  asset_pipeline -> integration_test  [color=red penwidth=2]
  compile -> unit_test
  compile -> package  [color=red penwidth=2]

  package -> integration_test [color=red penwidth=2]
  direct_deps [shape=none]
  sources [shape=none]
  static_input_files [shape=none]
  resolve_deps [fillcolor=lightgreen style=filled]
  code_gen [fillcolor=lightgreen style=filled]
  compile [fillcolor=lightgreen style=filled]
  package [fillcolor=lightpink style=filled]
  asset_pipeline [fillcolor=lightpink style=filled]
  integration_test [fillcolor=lightblue style=filled]
}
....

If you then change a source file in `sources` and ask to run `integration_test`
again, the build tool can again traverse the graph edges and see that:

- `resolve_deps`, `code_gen`, and `asset_pipeline` are not downstream of `sources` and can be reused
- `compile` and `package` _are_ downstream of `sources` and need to be re-run
- `unit_test` is not needed for `integration_test`, and so can be ignored


[graphviz]
....
digraph G {
  rankdir=LR
  node [shape=box width=0 height=0]
  direct_deps -> resolve_deps -> compile
  code_gen -> compile
  sources -> compile [color=red penwidth=2]
  static_input_files -> asset_pipeline
  asset_pipeline -> integration_test
  compile -> unit_test
  compile -> package [color=red penwidth=2]

  package -> integration_test [color=red penwidth=2]
  direct_deps [shape=none]
  sources [shape=filled color=red]
  static_input_files [shape=none]
  resolve_deps [fillcolor=lightgreen style=filled]
  code_gen [fillcolor=lightgreen style=filled]
  asset_pipeline [fillcolor=lightgreen style=filled]
  compile [fillcolor=lightpink style=filled]
  package [fillcolor=lightpink style=filled]
  integration_test [fillcolor=lightblue style=filled]
}
....

Thus, having a model of the build graph is fundamental to how most build
tools work their magic. When the user asks to run `unit_test` or `integration_test`,
the build tool can simply traversal build graph to automatically determine which
tasks need to run in order to do that, and which tasks have earlier output that can
be re-used.

=== Parallelism on the Build Graph

Apart from caching, the build graph is also useful for automatically parallelizing your build tasks.
For example, consider a case where we want to do a clean build (i.e. no caching)
of `unit_test` and `integration_test`. From the build graph, Mill is able to determine:

- `resolve_deps`, `code_gen`, and `asset_pipeline` can immediately start running in parallel (green)
- `compile`, `package`, and `integration_test` must run sequentially (red),
  only starting once `resolve_deps` and `code_gen` is complete
- `unit_test` (blue) can run in parallel with `asset_pipeline`, `package` or `integration_test`,
  but only can start once `compile` is complete

[graphviz]
....
digraph G {
  rankdir=LR
  node [shape=box width=0 height=0]
  direct_deps -> resolve_deps -> compile
  code_gen -> compile
  sources -> compile
  static_input_files -> asset_pipeline
  asset_pipeline -> integration_test
  compile -> unit_test
  compile -> package

  package -> integration_test
  direct_deps [shape=none]
  sources [shape=none]
  static_input_files [shape=none]
  resolve_deps [style=filled fillcolor=lightgreen]
  code_gen [style=filled fillcolor=lightgreen]
  asset_pipeline [style=filled fillcolor=lightgreen]
  compile [style=filled fillcolor=lightpink]
  package [style=filled fillcolor=lightpink]
  integration_test [style=filled fillcolor=lightpink]
  unit_test [style=filled fillcolor=lightblue]
}
....


Most modern build tools do this kind of graph-based parallelism automatically.
In contrast to most programming languages and application frameworks where you need
to set up parallelism yourself, in a build tool you don't need to fiddle
with threads, locks, semaphores, futures, actors, and so on. You just define the
shape of the build graph using whatever configuration format or language the build
tool provides, and you get parallelism for free.

=== Languages for defining your Build Graph

Every build tool provides some format for defining the build graph data structure.
There isn't any industry-wide standard text format for graph data structures, so each build
tool comes up with something on their own. Here we'll look at how it is done in
a few common build tools:

==== Bazel

Bazel uses the https://bazel.build/rules/language[Starlark language], a dialect of Python.
Below I show an example `BUILD` file from their https://bazel.build/start/cpp[documentation on using Bazel for C/C++].
The Python functions like `cc_library` or `cc_binary` are called _rules_ and by calling
the function you create a `target` with the given `name` and dependencies on upstream
targets (`deps`) and source files (`srcs`):

[source,python]
----
cc_library(
    name = "hello-greet",
    srcs = ["hello-greet.cc"],
    hdrs = ["hello-greet.h"],
)

cc_binary(
    name = "hello-world",
    srcs = ["hello-world.cc"],
    deps = [":hello-greet"],
)
----

[graphviz]
....
digraph G {
  rankdir=LR
  node [shape=box width=0 height=0 style=filled fillcolor=white]
  "main/hello-greet.cc\nmain/hello-greet.h" -> "//main:hello-greet"
  "//main:hello-greet" -> "//main:hello-world"
  "main/hello-world.cc" -> "//main:hello-world"
}
....

In Bazel, the implementation of what rules like `cc_library` actually _do_ is
done by upstream build libraries implemented in Java or Starlark. You can
also define your own custom rules in https://bazel.build/rules/lib/globals/bzl[.bzl files],
or use other people's rules from one of the
`rules_{lang}` repos on https://github.com/bazel-contrib[Bazel-Contrib],
if you need functionality that Bazel does not come built in with

==== Gradle

Gradle lets you define custom tasks in either Kotlin or Groovy. Below I show an
https://docs.gradle.org/current/userguide/implementing_custom_tasks.html[example custom task from their documentation]
written in Kotlin, that adds a new `packageApp` task:

[source,kotlin]
----
val packageApp = tasks.register<Zip>("packageApp") {
    from(layout.projectDirectory.file("run.sh"))                // input - run.sh file
    from(tasks.jar) {                                           // input - jar task output
        into("libs")
    }
    from(configurations.runtimeClasspath) {                     // input - jar of dependencies
        into("libs")
    }
    destinationDirectory.set(layout.buildDirectory.dir("dist")) // output - location of the zip file
    archiveFileName.set("myApplication.zip")                    // output - name of the zip file
}
----

[graphviz]
....
digraph G {
  rankdir=LR
  node [shape=box width=0 height=0 style=filled fillcolor=white]
  "run.sh" -> packageApp
  "tasks.jar" -> packageApp
  "configurations.runClassPath" -> packageApp
}
....

`packageApp` depends on the `run.sh` source file, the output of the tasks
`tasks.jar` and `configurations.runClasspath`, and some other miscellanious configuration.
The kotlin code is a bit idiosyncratic with the `from` and `into` helpers, and in this case
needs to integrate with the pre-defined `Zip` class that represents this task type and
comes from upstream in Gradle. But the end result of this syntax is to define a
small snippet of the build graph as shown above, and it is this graph that ends up
being important in your build system.

#### Mill

Mill xref:mill:ROOT:depth/why-scala.adoc[uses Scala for it's build file format], and lets you write
normal method ``def``s to define xref:mill:ROOT:javalib/intro.adoc#_custom_build_logic[custom tasks]
in your build graph. The different methods can call each other, e.g. `def resources` below can
call `lineCount()` or `super.resources()`, and these method calls become the edges in your build graph:

[source,scala]
----
/** Total number of lines in module source files */
def lineCount = Task {
  allSourceFiles().map(f => os.read.lines(f.path).size).sum
}

/** Generate resources using lineCount of sources */
override def resources = Task {
  os.write(Task.dest / "line-count.txt", "" + lineCount())
  super.resources() ++ Seq(PathRef(Task.dest))
}
----

[graphviz]
....
digraph G {
  rankdir=LR
  node [shape=box width=0 height=0 style=filled fillcolor=white]
  allSourceFiles -> lineCount -> resources -> run
  "resources.super" -> "resources"
  "resources.super" [color=white]
  allSourceFiles [color=white]
  run [color=white]
}
....

Perhaps the most interesting thing about Mill is that you can write arbitrary code as
part of your ``def``s: above you can see the code for iterating over files, counting
their lines, writing out a `line-count.txt` file, and so on. This is in contrast with Bazel
or Gradle, where your Starlark/Kotlin/Groovy code is really just a fancy YAML file that
configures build logic defined "elsewhere".

This arbitrary-code approach is similar to the approach taken by the venerable Make
tool, but running on the JVM comes with additional advantages:

* *IDE support* (e.g. in IntelliJ and VSCode): your IDE is able to provide autocomplete,
  error-highlighting, and navigation up and down your build graph. This is something impossible
  in `make` (which runs Bash scripts too dynamic for your IDE to analyze) or declarative tools
  like Bazel or Gradle (where the code you write just configures an execution-engine that
  your IDE doesn't understand)

* *The compiler can check for mistakes*: this is especially important in a build tool
  as unlike application code, build config tends to be written by non-experts and
  not covered by unit or integration tests. Thus any assistance in catching bugs is very
  valuable, and here the compiler provides a lot of help for free.

* *You can use any JVM library*. e.g. The `os.read.lines` or `os.write` functions are
  from https://github.com/com-lihaoyi/os-lib[OS-Lib], but you can
  download any library you want from
  Java's Maven Central repository and use it in your build with full functionality and IDE support.
  The JVM ecosystem is deep and broad, and you can find libraries to do almost anything
  imaginable available for free. Mill lets you make full use of this to customize
  your build to do exactly what you need.

Mill thus provides a safe, ease-to-use programming environment for working with your
build pipelines. And although you can write arbitrary code in each task, the
`Task{ }` wrapper automatically provides parallelism, caching, and other things
you want in your build tool.



== Conclusion

Although modern build tools may look very different on the surface, most of them
are surprisingly similar once you peek under the covers. Bazel's StarLark config,
Gradle's Groovy/Kotlin config, Mill's Scala config, all end up boiling down to
a build graph similar to the one above with only minor differences. And although the
way they execute their tasks using the build graph does differ, at their most
fundamental level they use the sort of graph traversals that I discuss above.

At their core, most build tools have the same goal. A build tool takes the wide
variety of tasks a developer needs to do during development, and automatically
run them as efficiently and quickly as possible, with caching and parallelism
and whatever other optimizations it can find.

Hopefully this blog post has given you a better appreciation for how build tools
do what they do, and perhaps give you some insight next time you need
to debug a build tool that is doing something wrong!
