Automating GO Module Dependency Updates

Go’s new “Minimal Version Selection” approach to modules provides consistency of versions, but projects can remain stuck on outdated dependencies without maintainer conscientiousness or automation. This article discusses how to automate dependency updates to free up module maintainer time for other tasks.

What needs updating?

A go.mod file typically looks something like this:

module github.com/gohugoio/hugo

require (
    github.com/BurntSushi/toml v0.0.0-20170626110600-a368813c5e64
    github.com/gobwas/glob v0.2.3
    github.com/mitchellh/mapstructure v1.0.0
)

In the above you can see three modules. BurntSush/toml has a pseudo-version while gobwas/glob and mitchellh/mapstructure have “real” versions, compatible with semver.

There are multiple possibilities for how these require statements might need updating in future:

• BurntSush/toml might need an update to a new pseudo-version (i.e. git commit)
• BurntSush/toml might have adopted semantic versioning and should get a “real” version number
• gobwas/glob or mitchellh/mapstructure might have updated versions available

In each of the above cases, there is a go.sum checksum file that needs updating too.

How can these be updated manually?

By hand

If you know exactly what you want, you can always update versions in go.mod by hand and then run go get to update go.sum as well. However knowing exactly what versions you need to upgrade to is usually the exception, not the rule.

By CLI

As described on the Go Modules page:

To upgrade to the latest version for all transitive dependencies of the current module:

run go get -u to use the latest minor or patch releases run go get -u=patch to use the latest patch releases

The challenge is that you might have dozens of packages needing updating, and should ideally run a full suite of tests against each new version separately. Doing so can take up a lot of time and is prone to human error.

Using automation

Renovate is an open-source tool designed specifically to help automate dependency updates using branches and Pull Requests. In addition, Renovate will fetch and embed Release Notes whenever found to save the additional steps of looking those up.

Renovate automation for Go Modules

The Renovate open source CLI tool is distributed via npmjs (i.e. installable via npm or yarn) and also as a pre-built Docker image compatible with Kubernetes. For this example though, we will be using the hosted Renovate App on github.com, which is free for open source repositories and has a free trial private repositories if necessary.

To demonstrate modules updates on a real and active project, I’ve copied go.mod and go.sum into a new repository rarkins/lazygit-example.

The next step is to add Renovate to this repository, using GitHub’s app controls at github.com/apps/marketplace.

Once this is done, you will get a “Configure Renovate” Pull Request that allows you to configure Renovate first before fully activating it to get “real” pull requests containing module updates.

As you can see above, Renovate will limit branch creation to two per hour by default, so as not to swamp CI. You can also place a limit on concurrent PRs open, which defaults to 20.

In addition, Renovate will raise PRs in this order:

• commit hash updates
• patch updates
• minor updates
• major upgrades

For the purpose of this demo, we’ll set "prHourlyLimit": 10 in renovate.json and merge. The resulting Pull Requests can be found in github.com/rarkins/lazygit-example/pulls.

golang.org/x/net

Here we have an example of a commit hash update in this module’s source repository on GitHub. The PR looks like this:

And the file diffs look like this:

You can see that both the go.mod and go.sum files have been updated.

aws-sdk-go

Next up we have a minor update to aws. The nice thing about this PR is that we can see an example of an embedded release note informing us of what’s changed:

hashicorp/go-httpclean

In the final PR example we have a module that has been converted from a pseudo version to a “real” version:

This can also be verified by looking at the go.mod diff:

Configuration options

This demo of Go module updating has just scratched the surface by using default config options. Renovate is a multi-language tool that has been battle-hardened on nearly 30,000 repositories on GitHub over the past year, and the majority of existing features will “just work” now that Go Module support has been plugged in. Such features include scheduling, grouping and customizing commit and PR templates. For more details check out docs.renovatebot.com

Go get it

Go Module updating is available today via Renovate’s hosted GitHub App, or via self-hosted Open Source for GitHub Enterprise, GitLab and BitBucket Cloud. Help keep the ecosystem fresh by adding automation to your module.