Blog.

Docker tips and tricks for your Go projects

Marco Franssen

Marco Franssen /

10 min read • 1896 words

Cover Image for Docker tips and tricks for your Go projects

In this blogpost I would like to show you some basic Docker setup I have been using so far in my Go projects. We will be looking at multi-stage Docker builds and how to utilize docker-compose.

In a typical project setup in Go you would most probably start with a file main.go. In addition to that I usually add a Dockerfile for building a Docker image and a docker-compose file to easily spin up my dependencies like databases and queues.

terminal
$ tree go-docker
go-docker
├── Dockerfile
├── docker-compose.yml
└── main.go
 
0 directories, 3 files

To start we create a new folder to work in and initiales the folder as a Go module.

terminal
mkdir go-docker-webserver
cd go-docker-webserver
go mod init github.com/go-docker-webserver

I would also like to reuse the webserver from my last blogpost and run it in our Docker setup which we are about to create in this blog post. For convenience I also put the code in below codeblock.

main.go
package main

import (
	"context"
	"flag"
	"log"
	"net/http"
	"os"
	"os/signal"
	"time"
)

var (
	listenAddr string
)

func main() {
	flag.StringVar(&listenAddr, "listen-addr", ":5000", "server listen address")
	flag.Parse()

	logger := log.New(os.Stdout, "http: ", log.LstdFlags)

	done := make(chan bool, 1)
	quit := make(chan os.Signal, 1)

	signal.Notify(quit, os.Interrupt)

	server := newWebserver(logger)
	go gracefullShutdown(server, logger, quit, done)

	logger.Println("Server is ready to handle requests at", listenAddr)
	if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
		logger.Fatalf("Could not listen on %s: %v\n", listenAddr, err)
	}

	<-done
	logger.Println("Server stopped")
}

func gracefullShutdown(server *http.Server, logger *log.Logger, quit <-chan os.Signal, done chan<- bool) {
	<-quit
	logger.Println("Server is shutting down...")

	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
	defer cancel()

	server.SetKeepAlivesEnabled(false)
	if err := server.Shutdown(ctx); err != nil {
		logger.Fatalf("Could not gracefully shutdown the server: %v\n", err)
	}
	close(done)
}

func newWebserver(logger *log.Logger) *http.Server {
	router := http.NewServeMux()
	router.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		logger.Println(r.Method, r.URL.Path, r.RemoteAddr, r.UserAgent())
		w.WriteHeader(http.StatusOK)
	})

	return &http.Server{
		Addr:         listenAddr,
		Handler:      router,
		ErrorLog:     logger,
		ReadTimeout:  5 * time.Second,
		WriteTimeout: 10 * time.Second,
		IdleTimeout:  15 * time.Second,
	}
}

Please download this file and put it as main.go in our new folder go-docker-webserver. With all of this in place we should now be able to start our webserver using go run. So far all good, but now it is time to come to the essence of this blogpost.

Docker

Now I would like to add a Dockerfile which is able to compile the binary and package that as a docker image. I expect you to have some basic Docker knowledge as I will not discuss all the Docker specifics in detail. I have previously written some blogs on Docker which you can find here. These include the following:

Today I will use a multi stage build process to get an image which is as small as possible for our Go binary to run. In my Docker builds I always prefer the alpine images as they are small, quick to download and consume not to much storage on my development machine. This isn't mandatory, just my personal preference.

In below Dockerfile I defined the first stage of this build as builder utilizing the golang:1.12-alpine image. To fix cgo usecases for the docker image I have added some missing tools to the alpine image. After we COPY our code from our workspace into the build folder within the alpine image. Once that is done we run go get which will download our dependencies as they are defined in the go.mod file. Once that is done we can build the binary using go build. Now first have a peek at the first half of below Dockerfile before I will continue the further explanation.

Dockerfile
FROM golang:1.12-alpine as builder
 
# To fix go get and build with cgo
RUN apk add --no-cache --virtual .build-deps \
    bash \
    gcc \
    git \
    musl-dev
 
RUN mkdir build
COPY . /build
WORKDIR /build
 
RUN go get
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o webserver .
RUN adduser -S -D -H -h /build webserver
USER webserver
 
FROM scratch
COPY --from=builder /build/webserver /app/
WORKDIR /app
EXPOSE 5000
CMD ["./webserver"]

In the second half the above Dockerfile we are about to build a Docker image based on the scratch image. This is an image without OS and no further things installed. It is the smallest image you will be able to start with. As we have compiled our binary with some specific options it will not have any dependencies, meaning we can run it as bare bone as possible. -extldflags "-static" means do not link against shared libraries.

Let's give this a quick try.

terminal
$ docker build -t go-docker-webserver:latest .
$ docker run --rm -p "5000:5000" go-docker-webserver
http: 2019/03/19 19:16:00 Server is ready to handle requests at :5000
http: 2019/03/19 19:24:41 GET / 172.17.0.1:50864 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
http: 2019/03/19 19:24:42 GET /favicon.ico 172.17.0.1:50864 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36

Also have a peek at the image size and be amazed how small it is :).

terminal
$ docker images
REPOSITORY             TAG       IMAGE ID        CREATED          SIZE
go-docker-webserver    latest    8d5a7f09ec9e    2 minutes ago    7.43MB
.........

Docker-compose

Now lets also have a look on a docker-compose file. In this example I will first show you how we can utilize this to have an easier way to build our Docker image using docker-compose.

docker-compose.yml
version: "3.7"
 
services:
  web:
    image: go-docker-webserver
    build: .
    ports:
      - "5000:5000"

To run our application using docker-compose we can now utilize this using the following command.

terminal
$ docker-compose up --build
Creating network "graceful-webserver_default" with the default driver
Building web
Step 1/14 : FROM golang:1.12-alpine as builder
.........
.............
...........
......
Step 10/14 : FROM scratch
 --->
Step 11/14 : COPY --from=builder /build/webserver /app/
 ---> Using cache
 ---> a6d1afd58117
Step 12/14 : WORKDIR /app
 ---> Using cache
 ---> 8e482ff5aacf
Step 13/14 : EXPOSE 5000
 ---> Using cache
 ---> 2b19c892fa2b
Step 14/14 : CMD ["./webserver"]
 ---> Using cache
 ---> 8d5a7f09ec9e
 
Successfully built 8d5a7f09ec9e
Successfully tagged go-docker-webserver:latest
Creating graceful-webserver_web_1 ... done
Attaching to graceful-webserver_web_1
web_1  | http: 2019/03/19 19:36:11 Server is ready to handle requests at :5000

As you can see this will save us some time to build a new image and run it.

In order to have a nice example on how to further use docker-compose I would like to add another container with an external tool to have a nice showcase. So lets add Traefik to our setup which is an awesome.

Simplifynetworking complexity while designing, deploying, and running applications.

Treafik has a nice option to have it configure itself by using Docker labels. In our Docker example I will use this approach to configure Traefik. Traefik needs access to the Docker sock to be able to do these configurations Therefore we will add a volumes so the Traefik container will have access to the sock. Furthermore we will boot Traefik with the --api and --docker option.

docker-compose.yml
version: "3.7"
 
services:
  api-gateway:
    image: traefik:1.7-alpine
    command: --api --docker
    ports:
      - "80:80"
      - "8888:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
 
  web:
    image: go-docker-webserver
    build: .
    labels:
      - "traefik.frontend.rule=Host:localhost; PathPrefixStrip: /api/v1/awesome/"

As you can see we have also add a label on our web container to configure it using a Traefik frontend rule. With above rule we make our service available to the outside world at http://localhost/api/v1/awesome/. Therefore I also removed the previously public exposed port 5000. This means we can not directly access our webserver anymore from outside the Docker network.

Now lets boot our Docker environment again. This time I will also use the -d parameter to run it in the background and not block our terminal.

terminal
$ docker-compose up --build -d
Creating network "graceful-webserver_default" with the default driver
Building web
Step 1/14 : FROM golang:1.12-alpine as builder
.........
.............
...........
......
Step 10/14 : FROM scratch
 --->
Step 11/14 : COPY --from=builder /build/webserver /app/
 ---> Using cache
 ---> a6d1afd58117
Step 12/14 : WORKDIR /app
 ---> Using cache
 ---> 8e482ff5aacf
Step 13/14 : EXPOSE 5000
 ---> Using cache
 ---> 2b19c892fa2b
Step 14/14 : CMD ["./webserver"]
 ---> Using cache
 ---> 8d5a7f09ec9e
 
Successfully built 8d5a7f09ec9e
Successfully tagged go-docker-webserver:latest
Creating graceful-webserver_api-gateway_1 ... done
Creating graceful-webserver_web_1         ... done

Using docker-compose logs -f web we can for example still get the logs from our web container whenever we need that.

Now lets have a look at http://localhost:8888 which is the management interface of Traefik. As you can see there is one frontend to our web container with currently one backend. We can now scale our web container to run it as multiple instances using docker-compose. As Traefik is utilizing the Docker sock it will automatically pick up the new instances available. All requests will now be loadbalanced on these containers. Let's give this a try.

terminal
$ docker-compose up -d --scale web=3
graceful-webserver_api-gateway_1 is up-to-date
Starting graceful-webserver_web_1 ... done
Creating graceful-webserver_web_2 ... done
Creating graceful-webserver_web_3 ... done

Using above we will scale our web container to 3 instances. In the Traefik Dashboard you will see there are 3 backends available for your web deployment. When you make requests to our Traefik endpoint which we configured for our web deployment http://localhost/api/v1/awesome/ you will see all still works as expected and gives us the HTTP 200 OK response. To see that there is actual load balancing happening you can see multiple containers are booted using following command, and you can check the logs to see requests are showing up in the different instances.

terminal
$ docker ps
CONTAINER ID   IMAGE                 COMMAND                  CREATED          STATUS         PORTS                                        NAMES
2b1a608d27e1   go-docker-webserver   "./webserver"            3 minutes ago    Up 3 minutes   5000/tcp                                     graceful-webserver_web_3
4b9e3a03312d   go-docker-webserver   "./webserver"            3 minutes ago    Up 3 minutes   5000/tcp                                     graceful-webserver_web_2
35454b1d1f78   traefik:1.7-alpine    "/entrypoint.sh --ap…"   4 minutes ago    Up 4 minutes   0.0.0.0:80->80/tcp, 0.0.0.0:8888->8080/tcp   graceful-webserver_api-gateway_1
7dcd45abeea2   go-docker-webserver   "./webserver"            4 minutes ago    Up 4 minutes   5000/tcp                                     graceful-webserver_web_1
 
$ docker-compose logs -f web
Attaching to graceful-webserver_web_3, graceful-webserver_web_2, graceful-webserver_web_1
web_1  | http: 2019/03/19 20:40:21 Server is ready to handle requests at :5000
web_1  | http: 2019/03/19 20:43:04 GET / 172.21.0.3:40670 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_3  | http: 2019/03/19 20:41:15 Server is ready to handle requests at :5000
web_2  | http: 2019/03/19 20:41:15 Server is ready to handle requests at :5000
web_2  | http: 2019/03/19 20:43:48 GET / 172.21.0.3:51194 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_3  | http: 2019/03/19 20:43:49 GET / 172.21.0.3:33668 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_1  | http: 2019/03/19 20:43:51 GET / 172.21.0.3:40680 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_2  | http: 2019/03/19 20:43:52 GET / 172.21.0.3:51194 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_3  | http: 2019/03/19 20:43:53 GET / 172.21.0.3:33668 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_1  | http: 2019/03/19 20:43:54 GET / 172.21.0.3:40680 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_2  | http: 2019/03/19 20:43:56 GET / 172.21.0.3:51194 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_3  | http: 2019/03/19 20:43:57 GET / 172.21.0.3:33668 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36

Just invoke the request a few times and you will see the requests are every time handled by different instances of the web container. When you try to access any url different then the configured path you will get a HTTP 404 response from Traefik as these endpoints have no backend.

Now you can do similar for defining you dependencies like databases in the docker-compose file to have a fully fledged local development environment. A fully working example can be downloaded from here. Thanks for reading, please share with your friends and colleagues and see you next time.

You have disabled cookies. To leave me a comment please allow cookies at functionality level.

More Stories

Cover Image for Improved graceful shutdown webserver

Improved graceful shutdown webserver

Marco Franssen

Marco Franssen /

In a previous blogpost I wrote how to create a Webserver in Go with graceful shutdown. This time I want to show you a more improved version which you can utilize better in your projects as it can be used as a drop in server.go file in your project where I also make use of some popular high performing libraries. In previous example I coded the full example in main.go. Although nothing wrong with that I learned while building microservices for a while it would be more convenient for me if I could…

Cover Image for Go client for Elasticsearch using Docker

Go client for Elasticsearch using Docker

Marco Franssen

Marco Franssen /

In this blog post I would like to cover the recently released Elasticsearch 7.0-rc1 Go client for Elasticsearch. In this blogpost I want to show you a small example with a simple Docker setup using to build a Elasticsearch cluster. In my previous blogpost I covered some Docker tips and tricks we will utilize again in this blog post. Initializing your project To start with we first have to create a project folder. In this folder we will have to initialize our Go module, add our Dockerfile and…

Cover Image for Go webserver with graceful shutdown

Go webserver with graceful shutdown

Marco Franssen

Marco Franssen /

In this blogpost I want to show you how you can make a http webserver in Go with gracefull shutdown. Using this approach you allow the server to clean up some resources before it actually shuts down. Think about finishing a database transaction or some other long operation. We will be using the things we learned in my blogpost on concurency. So expect to see channels and go routines as part of the solution. When I create new http servers I usually start with an commandline flag to provide the p…

Cover Image for Go interfaces and type assertions

Go interfaces and type assertions

Marco Franssen

Marco Franssen /

In this blog I would like to zoom in on Interfaces and type assertions in Go. Compared to language like c# and Java implementing interfaces works slightly different. In the remainder of this blog I want to give you a bit of theory and practical usecases. In case this is your first time working with Go you might want to check out this blog which shows you how to setup your development environment including a small hello world. The empty interface (interface{}) is an interface which defines zero…