Updating NPM Packages – The Definitive Guide
Table of Contents
Selecting and installing a dependency with Node Package Manager (npm) is only half the job. This guide will explain why you need to keep your npm dependencies updated, and the most efficient ways to do so. It starts out with simple concepts to get everybody aligned, and goes on to present cutting-edge approaches to npm dependency management that weren’t available even a year or two ago. In the following guide we’ll use the term npm to keep things simple, however it also applies almost universally to yarn, pnpm, or lerna.
Defining the ranges and constraints of npm updates
Before you dive into updating npm dependencies, take the time to first reconsider how you define your package versions to begin with. You might not have given this much thought before because npm and other clients pick defaults for you. However, your choice of constraints can be just as important as your choice for how to update them.
Ranges are not part of SemVer
As you may already know, npm uses “Semantic Versioning” (SemVer) to define versions, i.e. major.minor.patch. Furthermore, npm allows the use of “ranges” (also known as “constraints”). For example, ^1.1.0 means >= 1.1.0 < 2.0.0. This type of range syntax is widely misunderstood despite its common use.
^1.0.0
1.*
1.x
>= 1.0.0 < 2.0.0
The above are all equivalent, but only in npm.
It is the SemVer specification that decrees that all 1.x releases should be backwards-compatible with other 1.x releases. Therefore, the fact that 1.3.0 should be backwards-compatible with 1.2.0 is not changed, regardless of whether you specify exact versions (e.g. 1.2.0) or ranges (e.g. ^1.2.0).
So then why do people use ranges?
Early use of ranges
Before the advent of lock files, you only needed a package.json file. If you defined a npm dependency with a constraint such as ^1.0.0 then it was an instruction to npm to use whatever the latest 1.x was at the time of installation. In what must have seemed like a good idea at the time, people reasoned that “according to SemVer, anything in 1.x should be backwards-compatible, so it’s safe to always install the latest 1.x”.
This assumption of compatibility did not meet the realities of software development, and led to a large amount of “works on my machine” and “broken build” problems, because people in a project could often be running different versions of a dependency —and not only among developers, but also between developers and CI or between CI and production. This risky “version roulette” needed to stop, and so lock files came to be.
Lock files
First introduced into the JavaScript ecosystem by Yarn, lock files for package.json provided a leap forward in reliability and reproducibility. Installing with a lock file meant that you could be assured of the same installation of dependencies (i.e. what’s in your node_modules) both within developer teams and in all environments dev, test, staging, production, etc. Instead of only knowing as little as “We are running some version 1.x”, now you could be sure which exact version of 1.x it is.
Locking the entire npm dependency tree
So far we’ve focused on what’s in the package.json (“direct” dependencies) but there is a metaphorical iceberg of indirect, or “transitive“, dependencies that also get installed, all of which also needs to be “locked” in order to have a reproducible node_modules.
A good rule of thumb is that your indirect dependencies are usually around 10-100x in size compared to your direct ones specified in package.json.
No more automatic updates
Now that you have a lock file in place, the randomness of installation results is gone. However, so is the idea of automatic updates. Instead, newer versions of dependencies won’t install no matter how long you wait or how many times you run npm install (which is a good thing, if you care about builds not breaking). Updating dependencies is now a choice and you need to decide how and when.
What if I never update npm dependencies?
If it ain’t broke, don’t fix it, right? You’re probably not reading so far into this article about updating npm dependencies if you think it’s a good idea not to update. Still, since “don’t update” is a valid option, we’ll discuss it here.
New versions of npm packages are usually released for one of two reasons:
- A feature was added (typically a minor release)
- A bug was fixed (typically a patch release)
However it’s quite common that bugs will also be fixed in minor releases, e.g. version 1.3.0 may include one new feature and three bug fixes compared to version 1.2.0.
If your software requires one of the newly added features then naturally you need to update. Instead let’s consider the case where you have no known functionality motivations for updating a dependency.
An important question is: how certain can you be that the fixed bugs in later releases aren’t impacting your users right now? For example, if you’re importing a framework for UI, would you really know if there was a buggy drop-down for certain browsers or misplaced pixels for others?
Perhaps the biggest problem with an approach of “upgrade nothing unless I have a reason to” is that sometimes there’s a critical reason to update — vulnerabilities. On those dreaded days when you get notified that there’s a critical security vulnerability in one of your npm dependencies, and you have little choice but to immediately update, you may have a particularly stressful day if you have fallen badly behind in dependency versions. In such a case you might need to review a year’s worth of release notes to work out if anything there poses a risk to your product, and if so, then what to do? Compare this to a scenario where you were already reasonably up-to-date and could apply a simple low-risk patch and be done with it.
In summary, the two main problems with a “reactive” approach to updating dependencies are:
- A high chance that there are unfixed bugs in your product
- The risk of rushing months or years of npm updates to your dependencies when vulnerability patches are required is greatly increased
Updating npm dependencies manually
The next strategy to consider is one of updating npm dependencies manually.
As discussed earlier, once you have a lock file in place then none of your dependency versions should ever change without you or a colleague doing so intentionally. So let’s walk through the process.
First, you have to remember to do it. The reality is that most of us let it slip.
Next, you need a way to know what needs updating. Most of the npm-compatible tools have an “outdated” command that can list all the updates. Here’s an example from a test project with two dependencies:
❯ npm outdated
Package Current Wanted Latest Location
chalk 2.2.0 2.2.0 3.0.0 test-project
left-pad 1.1.0 1.1.0 1.3.0 test-project
While this is accurate, it does not provide all of the information that you need in order to make an educated choice. First of all, it provides no information about what changed. It’s also missing information about releases in-between. For example, chalk has a 2.4.2 release that includes fixes for 2.0.0, but it’s not displayed because a new major version 3.0.0 exists.
Typing the command npm docs chalk will take you to https://github.com/chalk/chalk#readme. From there you could look for a CHANGELOG.md file, or check the GitHub “Releases” tab, etc. Usually, you can look up what’s changed.
So what’s next? Assuming you want your CI to run against the new version, you’d probably:
- Create a branch like fix/update-chalk
- Run npm update chalk@3.0.0
- Commit the files locally, then push to your repository (e.g. GitHub)
- Create a Pull Request for that branch
- Write a Pull Request title and description describing what it does, and copy/pasting any release notes you found for your colleagues to read
Now repeat that for the 15 other dependencies that are out of date in your project. It’s quite a lot of work, and understandable why some people have a “do nothing” approach with dependency updating. But don’t worry, there’s a better way.
In summary, the two major problems with manual dependency updating are:
- You’re human and will probably forget
- There’s a lot of manual work required, and you’re probably better off spending your time on your core product than manually looking up release notes for someone else’s
Automating npm package updates
Today, tools like Mend Renovate can automate away all the manual process described above. Let’s hear from one of our earliest users:
“I really believe in 2020 most popular open source projects will use automatic dependency updates, fixing bugs and security holes in 3rd party dependencies even before they become known.”
— Gleb Bahmutov, VP of Engineering, Cypress.io
At a high level, Mend Renovate’s approach is quite simple:
- Scans repositories for package.json files (amongst others)
- Extracts the full list of dependencies within
- Looks up if any of the dependencies have updates available
- Creates Pull Requests according to your desired groupings and schedule
- Fetches and embeds release notes for each updated dependency
Dependency update automation may be the missing link you’ve been looking for that makes staying up-to-date with npm packages finally achievable, and probably requires less hours than you might have spent managing it using any previous ad-hoc approach.
Mend Renovate includes a lot of advanced features too, including granular rules (e.g. by dependency type, or by major/minor/patch type), and can even auto merge Pull Requests if tests pass, and it fits your needs.
The tool has brought new capabilities to developers and DevOps staff alike, and many are not afraid to write about it either. The following articles are all by Renovate users describing how they are using it and the benefits it brings:
- Bring In The Bots, And Let Them Maintain Our Code! (Patrick Lee Scott)
- Better Dependency Management Using Renovate (Daniel Lemay)
- How to Automatically Update Your JavaScript Dependencies (Scott Vandehey)
Getting started with Renovate — updating npm packages
If you are using github.com or gitlab.com, the simplest way to get started updating npm dependencies is via the hosted Mend Renovate app. Visit Mend Renovate and select the relevant link to view installation instructions.
If you are using GitHub Enterprise or GitLab self-hosted, the free Mend Renovate Community is your best approach.
For Azure DevOps or Bitbucket, check out the Open Source Renovate Project for a CLI-based version of the tool that you can run locally.
