Manage Go tools via Go modules
Marco Franssen /
6 min read • 1196 words
In this blog I will cover how I'm managing and versioning the tools my Go projects depend on. Go Modules are available since Go 1.11. Using Go Modules you can manage the dependencies for your project. You can compare it to NPM in Nodejs projects or Maven in Java project or Nuget in .NET projects.
In general Go Modules are used to manage your compile time dependencies. However in my projects I also like to manage the tools required for Continuous Integration in my projects. To ensure all developers have same versions of tools installed and to ensure my CI server (Jenkins, Travis, CircleCI) can install and use the same version of the tools. I found a way using Go Modules, by default you will have some issue with go mod tidy
if you would just manually add the tools as dependencies to your go.mod file.
Initialize
To start with Go Modules we first have to initialize a new Go Module. We do that by creating a new folder and run the go mod init
command.
This will result in following go.mod
file.
Add your tools as dependency
Now we want to add our tools as dependency to our go.mod
. In general you would think to just add them using go get
, which works at first sight perfectly fine. See below example.
As you can see it added indirect dependencies, and you will notice the goreleaser dependencies are not there at all as the second go get
has removed them. Indirect means there is none of your own code which has a direct dependency on the module, which is correct as these are tools I would use and not dependencies of my own to be written code. You will also see there is a go.sum file generated, which I will skip for now as it is not relevant for this explanation.
Now there is also the go mod tidy
command which cleans your dependencies etc. This is a recommended command to run before you make a release to ensure all dependencies are cleaned and accurate with the real needs of your code. So lets run that command now and check what happens.
As you can see all the dependencies are again removed from the go.mod
file and the go.sum
file is cleaned up as well. This happens because there is no dependency in any .go
file.
TL;DR
To ensure my tool dependencies are not removed and can leverage the Go Modules, I create a file tools.go
. In this file I will list all my tool dependencies using an import statement.
As you can notice I have also added a build constraint a.k.a. build tag comment in the top of the file, to ensure it is not compiled into the binary, when running go build
. Now with this file in place I can very easily install all my tools using a simple bash command. go install
will make all the tools available in your $GO_WORKSPACE/bin
folder. Normally this folder is available in your PATH
so you can use the binaries in any folder of your choice.
Above bash script will read all the lines starting with an _
from the file and it will strip the "
before it passes them to go install
. As you can see now the go.mod
and go.sum
files are updated. Also notice the dependencies are not cleared anymore when running go mod tidy
.
Also notice the dependencies are no longer indirect as you now have code which depends on the given module. Don't worry, due to the build constraint this code will not be compiled into the binary when running go build .
, unless you ofcourse provide a build argument with the tools constraint.
Now I can imagine you don't want to type this command all the time, so the next thing I do in my projects is adding a Makefile including a install-tools task.
This now allows me to run simply make install-tools
.
Now in your Makefile you can add more tasks for compiling, testing, benchmarking and running your application, so you have less manual commands to type in your project.
Summarized you will now have following files in your folder, which you could now start committing in your repo before you continue setting up your project and adding the code.
References
Thanks you for your attention! Also consider to share this with your friends and colleagues on social media and leave me a comment below. See you next time.