Docker tips and tricks for your Go projects

In this blogpost I would like to show you some basic Docker setup I have been using so far in my Go projects. We will be looking at multi-stage Docker builds and how to utilize docker-compose.

In a typical project setup in Go you would most probably start with a file main.go. In addition to that I usually add a Dockerfile for building a Docker image and a docker-compose file to easily spin up my dependencies like databases and queues.

1
2
3
4
- go-docker/
|- docker-compose.yml
|- Dockerfile
\- main.go

To start we create a new folder to work in and initiales the folder as a Go module.

1
2
3
mkdir go-docker-webserver
cd go-docker-webserver
go mod init github.com/go-docker-webserver

I would also like to reuse the webserver from my last blogpost and run it in our Docker setup which we are about to create in this blog post. For convenience I also put the code in below codeblock.

main.goview raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
package main

import (
"context"
"flag"
"log"
"net/http"
"os"
"os/signal"
"time"
)

var (
listenAddr string
)

func main() {
flag.StringVar(&listenAddr, "listen-addr", ":5000", "server listen address")
flag.Parse()

logger := log.New(os.Stdout, "http: ", log.LstdFlags)

done := make(chan bool, 1)
quit := make(chan os.Signal, 1)

signal.Notify(quit, os.Interrupt)

server := newWebserver(logger)
go gracefullShutdown(server, logger, quit, done)

logger.Println("Server is ready to handle requests at", listenAddr)
if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
logger.Fatalf("Could not listen on %s: %v\n", listenAddr, err)
}

<-done
logger.Println("Server stopped")
}

func gracefullShutdown(server *http.Server, logger *log.Logger, quit <-chan os.Signal, done chan<- bool) {
<-quit
logger.Println("Server is shutting down...")

ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

server.SetKeepAlivesEnabled(false)
if err := server.Shutdown(ctx); err != nil {
logger.Fatalf("Could not gracefully shutdown the server: %v\n", err)
}
close(done)
}

func newWebserver(logger *log.Logger) *http.Server {
router := http.NewServeMux()
router.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
logger.Println(r.Method, r.URL.Path, r.RemoteAddr, r.UserAgent())
w.WriteHeader(http.StatusOK)
})

return &http.Server{
Addr: listenAddr,
Handler: router,
ErrorLog: logger,
ReadTimeout: 5 * time.Second,
WriteTimeout: 10 * time.Second,
IdleTimeout: 15 * time.Second,
}
}

Please download this file and put it as main.go in our new folder go-docker-webserver. With all of this in place we should now be able to start our webserver using go run. So far all good, but now it is time to come to the essence of this blogpost.

Docker

Now I would like to add a Dockerfile which is able to compile the binary and package that as a docker image. I expect you to have some basic Docker knowledge as I will not discuss all the Docker specifics in detail. I have previously written some blogs on Docker which you can find here. These include the following:

Today I will use a multi stage build process to get an image which is as small as possible for our Go binary to run. In my Docker builds I always prefer the alpine images as they are small, quick to download and consume not to much storage on my development machine. This isn’t mandatory, just my personal preference.

In below Dockerfile I defined the first stage of this build as builder utilizing the golang:1.12-alpine image. To fix cgo usecases for the docker image I have added some missing tools to the alpine image. After we COPY our code from our workspace into the build folder within the alpine image. Once that is done we run go get which will download our dependencies as they are defined in the go.mod file. Once that is done we can build the binary using go build. Now first have a peek at the first half of below Dockerfile before I will continue the further explanation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
FROM golang:1.12-alpine as builder

# To fix go get and build with cgo
RUN apk add --no-cache --virtual .build-deps \
bash \
gcc \
git \
musl-dev

RUN mkdir build
COPY . /build
WORKDIR /build

RUN go get
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o webserver .
RUN adduser -S -D -H -h /build webserver
USER webserver

FROM scratch
COPY --from=builder /build/webserver /app/
WORKDIR /app
EXPOSE 5000
CMD ["./webserver"]

In the second half the above Dockerfile we are about to build a Docker image based on the scratch image. This is an image without OS and no further things installed. It is the smallest image you will be able to start with. As we have compiled our binary with some specific options it will not have any dependencies, meaning we can run it as bare bone as possible. -extldflags "-static" means do not link against shared libraries.

Let’s give this a quick try.

1
2
3
4
5
$ docker build -t go-docker-webserver:latest .
$ docker run --rm -p "5000:5000" go-docker-webserver
http: 2019/03/19 19:16:00 Server is ready to handle requests at :5000
http: 2019/03/19 19:24:41 GET / 172.17.0.1:50864 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
http: 2019/03/19 19:24:42 GET /favicon.ico 172.17.0.1:50864 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36

Also have a peek at the image size and be amazed how small it is :).

1
2
3
4
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
go-docker-webserver latest 8d5a7f09ec9e 2 minutes ago 7.43MB
.........

Docker-compose

Now lets also have a look on a docker-compose file. In this example I will first show you how we can utilize this to have an easier way to build our Docker image using docker-compose.

docker-compose.yml
1
2
3
4
5
6
7
8
version: '3.7'

services:
web:
image: go-docker-webserver
build: .
ports:
- "5000:5000"

To run our application using docker-compose we can now utilize this using the following command.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
$ docker-compose up --build
Creating network "gracefull-webserver_default" with the default driver
Building web
Step 1/14 : FROM golang:1.12-alpine as builder
.........
.............
...........
......
Step 10/14 : FROM scratch
--->
Step 11/14 : COPY --from=builder /build/webserver /app/
---> Using cache
---> a6d1afd58117
Step 12/14 : WORKDIR /app
---> Using cache
---> 8e482ff5aacf
Step 13/14 : EXPOSE 5000
---> Using cache
---> 2b19c892fa2b
Step 14/14 : CMD ["./webserver"]
---> Using cache
---> 8d5a7f09ec9e

Successfully built 8d5a7f09ec9e
Successfully tagged go-docker-webserver:latest
Creating gracefull-webserver_web_1 ... done
Attaching to gracefull-webserver_web_1
web_1 | http: 2019/03/19 19:36:11 Server is ready to handle requests at :5000

As you can see this will save us some time to build a new image and run it.

In order to have a nice example on how to further use docker-compose I would like to add another container with an external tool to have a nice showcase. So lets add Traefik to our setup which is an awesome.

A reverse proxy / load balancer that’s easy, dynamic, automatic, fast, full-featured, open source, production proven, provides metrics, and integrates with every major cluster technology… No wonder it’s so popular!

Treafik has a nice option to have it configure itself by using Docker labels. In our Docker example I will use this approach to configure Traefik. Traefik needs access to the Docker sock to be able to do these configurations Therefore we will add a volumes so the Traefik container will have access to the sock. Furthermore we will boot Traefik with the --api and --docker option.

docker-compose.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
version: '3.7'

services:
api-gateway:
image: traefik:1.7-alpine
command: --api --docker
ports:
- "80:80"
- "8888:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock

web:
image: go-docker-webserver
build: .
labels:
- "traefik.frontend.rule=Host:localhost; PathPrefixStrip: /api/v1/awesome/"

As you can see we have also add a label on our web container to configure it using a Traefik frontend rule. With above rule we make our service available to the outside world at http://localhost/api/v1/awesome/. Therefore I also removed the previously public exposed port 5000. This means we can not directly access our webserver anymore from outside the Docker network.

Now lets boot our Docker environment again. This time I will also use the -d parameter to run it in the background and not block our terminal.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
$ docker-compose up --build -d
Creating network "gracefull-webserver_default" with the default driver
Building web
Step 1/14 : FROM golang:1.12-alpine as builder
.........
.............
...........
......
Step 10/14 : FROM scratch
--->
Step 11/14 : COPY --from=builder /build/webserver /app/
---> Using cache
---> a6d1afd58117
Step 12/14 : WORKDIR /app
---> Using cache
---> 8e482ff5aacf
Step 13/14 : EXPOSE 5000
---> Using cache
---> 2b19c892fa2b
Step 14/14 : CMD ["./webserver"]
---> Using cache
---> 8d5a7f09ec9e

Successfully built 8d5a7f09ec9e
Successfully tagged go-docker-webserver:latest
Creating gracefull-webserver_api-gateway_1 ... done
Creating gracefull-webserver_web_1 ... done

Using docker-compose logs -f web we can for example still get the logs from our web container whenever we need that.

Now lets have a look at http://localhost:8888 which is the management interface of Traefik. As you can see there is one frontend to our web container with currently one backend. We can now scale our web container to run it as multiple instances using docker-compose. As Traefik is utilizing the Docker sock it will automatically pick up the new instances available. All requests will now be loadbalanced on these containers. Let’s give this a try.

1
2
3
4
5
$ docker-compose up -d --scale web=3
gracefull-webserver_api-gateway_1 is up-to-date
Starting gracefull-webserver_web_1 ... done
Creating gracefull-webserver_web_2 ... done
Creating gracefull-webserver_web_3 ... done

Using above we will scale our web container to 3 instances. In the Traefik Dashboard you will see there are 3 backends available for your web deployment. When you make requests to our Traefik endpoint which we configured for our web deployment http://localhost/api/v1/awesome/ you will see all still works as expected and gives us the HTTP 200 OK response. To see that there is actual load balancing happening you can see multiple containers are booted using following command, and you can check the logs to see requests are showing up in the different instances.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2b1a608d27e1 go-docker-webserver "./webserver" 3 minutes ago Up 3 minutes 5000/tcp gracefull-webserver_web_3
4b9e3a03312d go-docker-webserver "./webserver" 3 minutes ago Up 3 minutes 5000/tcp gracefull-webserver_web_2
35454b1d1f78 traefik:1.7-alpine "/entrypoint.sh --ap…" 4 minutes ago Up 4 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:8888->8080/tcp gracefull-webserver_api-gateway_1
7dcd45abeea2 go-docker-webserver "./webserver" 4 minutes ago Up 4 minutes 5000/tcp gracefull-webserver_web_1

$ docker-compose logs -f web
Attaching to gracefull-webserver_web_3, gracefull-webserver_web_2, gracefull-webserver_web_1
web_1 | http: 2019/03/19 20:40:21 Server is ready to handle requests at :5000
web_1 | http: 2019/03/19 20:43:04 GET / 172.21.0.3:40670 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_3 | http: 2019/03/19 20:41:15 Server is ready to handle requests at :5000
web_2 | http: 2019/03/19 20:41:15 Server is ready to handle requests at :5000
web_2 | http: 2019/03/19 20:43:48 GET / 172.21.0.3:51194 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_3 | http: 2019/03/19 20:43:49 GET / 172.21.0.3:33668 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_1 | http: 2019/03/19 20:43:51 GET / 172.21.0.3:40680 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_2 | http: 2019/03/19 20:43:52 GET / 172.21.0.3:51194 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_3 | http: 2019/03/19 20:43:53 GET / 172.21.0.3:33668 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_1 | http: 2019/03/19 20:43:54 GET / 172.21.0.3:40680 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_2 | http: 2019/03/19 20:43:56 GET / 172.21.0.3:51194 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_3 | http: 2019/03/19 20:43:57 GET / 172.21.0.3:33668 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36

Just invoke the request a few times and you will see the requests are every time handled by different instances of the web container. When you try to access any url different then the configured path you will get a HTTP 404 response from Traefik as these endpoints have no backend.

Now you can do similar for defining you dependencies like databases in the docker-compose file to have a fully fledged local development environment. A fully working example can be downloaded from here. Thanks for reading, please share with your friends and colleagues and see you next time.

Share