Blog.

Docker for Go projects with multi-stage builds and Docker Compose

Marco Franssen

March 19, 2019

9 min read•1608 words

Docker for Go projects with multi-stage builds and Docker Compose

In this post, I'll show you the Docker setup I've been using for my Go projects. We'll build a small image with multi-stage Docker builds, run it with Docker Compose, and add Traefik 1.7 for routing and load balancing.

In a typical Go project, you'll probably start with a main.go file. I usually add a Dockerfile to build an image and a Docker Compose file to start dependencies such as databases and queues.

terminal

$ tree go-docker
go-docker
├── Dockerfile
├── docker-compose.yml
└── main.go
 
0 directories, 3 files

To start, we'll create a folder and initialize it as a Go module.

terminal

mkdir go-docker-webserver
cd go-docker-webserver
go mod init github.com/go-docker-webserver

I'd like to reuse the web server from my last post in the Docker setup we're about to create. For convenience, I've included the code below.

main.go

Loading graceful-webserver/main.go…

Download this file and save it as main.go in our new go-docker-webserver folder. We should now be able to start the web server with go run. So far, so good. Now let's get to the Docker setup.

Docker

We'll add a Dockerfile to compile the binary and package it as a Docker image. I assume you have some basic Docker knowledge, so I won't explain every detail. You can find my previous Docker posts here, including:

We'll use a multi-stage build to keep the image for our Go binary as small as possible. I prefer Alpine images for my Docker builds because they're small, quick to download, and don't take up much storage on my development machine. That's just my preference, not a requirement.

In the Dockerfile below, I named the first stage builder and used the golang:1.12-alpine image. I've added some tools to Alpine to support cgo build use cases. We then COPY our code from the workspace into the image's build folder. Next, go get downloads the dependencies defined in go.mod, and go build builds the binary. Take a look at the first half of the Dockerfile before we continue.

Dockerfile

FROM golang:1.12-alpine as builder
 
# To fix go get and build with cgo
RUN apk add --no-cache --virtual .build-deps \
    bash \
    gcc \
    git \
    musl-dev
 
RUN mkdir build
COPY . /build
WORKDIR /build
 
RUN go get
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o webserver .
RUN adduser -S -D -H -h /build webserver
USER webserver
 
FROM scratch
COPY --from=builder /build/webserver /app/
WORKDIR /app
EXPOSE 5000
CMD ["./webserver"]

In the second half of the Dockerfile, we build the final image from scratch: an empty image with no operating system utilities or other software installed. It's the smallest starting point for an image. We've compiled our binary with options that avoid shared-library dependencies, so it can run in this minimal environment. -extldflags "-static" tells the linker not to link against shared libraries.

Let's give this a quick try.

terminal

$ docker build -t go-docker-webserver:latest .
$ docker run --rm -p "5000:5000" go-docker-webserver
http: 2019/03/19 19:16:00 Server is ready to handle requests at :5000
http: 2019/03/19 19:24:41 GET / 172.17.0.1:50864 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
http: 2019/03/19 19:24:42 GET /favicon.ico 172.17.0.1:50864 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36

Take a look at the image size too. It's pretty small :).

terminal

$ docker images
REPOSITORY             TAG       IMAGE ID        CREATED          SIZE
go-docker-webserver    latest    8d5a7f09ec9e    2 minutes ago    7.43MB
.........

Docker-compose

Now let's look at a Docker Compose file. First, I'll show you how it makes building and running our Docker image easier.

docker-compose.yml

version: "3.7"
 
services:
  web:
    image: go-docker-webserver
    build: .
    ports:
      - "5000:5000"

We can now build and run our application with the following command.

terminal

$ docker-compose up --build
Creating network "graceful-webserver_default" with the default driver
Building web
Step 1/14 : FROM golang:1.12-alpine as builder
.........
.............
...........
......
Step 10/14 : FROM scratch
 --->
Step 11/14 : COPY --from=builder /build/webserver /app/
 ---> Using cache
 ---> a6d1afd58117
Step 12/14 : WORKDIR /app
 ---> Using cache
 ---> 8e482ff5aacf
Step 13/14 : EXPOSE 5000
 ---> Using cache
 ---> 2b19c892fa2b
Step 14/14 : CMD ["./webserver"]
 ---> Using cache
 ---> 8d5a7f09ec9e
 
Successfully built 8d5a7f09ec9e
Successfully tagged go-docker-webserver:latest
Creating graceful-webserver_web_1 ... done
Attaching to graceful-webserver_web_1
web_1  | http: 2019/03/19 19:36:11 Server is ready to handle requests at :5000

As you can see, this saves us some steps when building a new image and running it.

To show you more of what Docker Compose can do, let's add another container. We'll use Traefik, a handy router for our setup.

Simplifynetworking complexity while designing, deploying, and running applications.
Traefik - The cloud native edge Router

Traefik can configure itself using Docker labels. That's the approach we'll use here. It needs access to the Docker socket to discover this configuration, so we'll add a volume mount for the socket. We'll also start Traefik with the --api and --docker options.

docker-compose.yml

version: "3.7"
 
services:
  api-gateway:
    image: traefik:1.7-alpine
    command: --api --docker
    ports:
      - "80:80"
      - "8888:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
 
  web:
    image: go-docker-webserver
    build: .
    labels:
      - "traefik.frontend.rule=Host:localhost; PathPrefixStrip: /api/v1/awesome/"

We've also added a label to our web container to configure a Traefik frontend rule. This rule exposes our service at http://localhost/api/v1/awesome/. I've removed the previously published port 5000, so we can no longer access the web server directly from outside the Docker network.

Let's start our Docker environment again. This time, I'll use the -d parameter to run it in the background without blocking our terminal.

terminal

$ docker-compose up --build -d
Creating network "graceful-webserver_default" with the default driver
Building web
Step 1/14 : FROM golang:1.12-alpine as builder
.........
.............
...........
......
Step 10/14 : FROM scratch
 --->
Step 11/14 : COPY --from=builder /build/webserver /app/
 ---> Using cache
 ---> a6d1afd58117
Step 12/14 : WORKDIR /app
 ---> Using cache
 ---> 8e482ff5aacf
Step 13/14 : EXPOSE 5000
 ---> Using cache
 ---> 2b19c892fa2b
Step 14/14 : CMD ["./webserver"]
 ---> Using cache
 ---> 8d5a7f09ec9e
 
Successfully built 8d5a7f09ec9e
Successfully tagged go-docker-webserver:latest
Creating graceful-webserver_api-gateway_1 ... done
Creating graceful-webserver_web_1         ... done

We can still view the web container's logs whenever we need them with docker-compose logs -f web.

Now open http://localhost:8888, Traefik's management interface. You'll see one frontend for our web container, with one backend. We can use Docker Compose to scale the web service to multiple instances. Because Traefik watches the Docker socket, it automatically discovers the new instances and load-balances requests across them. Let's try it.

terminal

$ docker-compose up -d --scale web=3
graceful-webserver_api-gateway_1 is up-to-date
Starting graceful-webserver_web_1 ... done
Creating graceful-webserver_web_2 ... done
Creating graceful-webserver_web_3 ... done

This scales our web service to three instances. In the Traefik dashboard, you'll see three backends for the deployment. Requests to our configured endpoint, http://localhost/api/v1/awesome/, still return HTTP 200 OK. To see load balancing in action, use the following commands to list the running containers and watch requests appear in each instance's logs.

terminal

$ docker ps
CONTAINER ID   IMAGE                 COMMAND                  CREATED          STATUS         PORTS                                        NAMES
2b1a608d27e1   go-docker-webserver   "./webserver"            3 minutes ago    Up 3 minutes   5000/tcp                                     graceful-webserver_web_3
4b9e3a03312d   go-docker-webserver   "./webserver"            3 minutes ago    Up 3 minutes   5000/tcp                                     graceful-webserver_web_2
35454b1d1f78   traefik:1.7-alpine    "/entrypoint.sh --ap…"   4 minutes ago    Up 4 minutes   0.0.0.0:80->80/tcp, 0.0.0.0:8888->8080/tcp   graceful-webserver_api-gateway_1
7dcd45abeea2   go-docker-webserver   "./webserver"            4 minutes ago    Up 4 minutes   5000/tcp                                     graceful-webserver_web_1
 
$ docker-compose logs -f web
Attaching to graceful-webserver_web_3, graceful-webserver_web_2, graceful-webserver_web_1
web_1  | http: 2019/03/19 20:40:21 Server is ready to handle requests at :5000
web_1  | http: 2019/03/19 20:43:04 GET / 172.21.0.3:40670 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_3  | http: 2019/03/19 20:41:15 Server is ready to handle requests at :5000
web_2  | http: 2019/03/19 20:41:15 Server is ready to handle requests at :5000
web_2  | http: 2019/03/19 20:43:48 GET / 172.21.0.3:51194 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_3  | http: 2019/03/19 20:43:49 GET / 172.21.0.3:33668 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_1  | http: 2019/03/19 20:43:51 GET / 172.21.0.3:40680 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_2  | http: 2019/03/19 20:43:52 GET / 172.21.0.3:51194 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_3  | http: 2019/03/19 20:43:53 GET / 172.21.0.3:33668 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_1  | http: 2019/03/19 20:43:54 GET / 172.21.0.3:40680 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_2  | http: 2019/03/19 20:43:56 GET / 172.21.0.3:51194 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_3  | http: 2019/03/19 20:43:57 GET / 172.21.0.3:33668 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36

Make a few requests, and you'll see different instances of the web container handling them. If you request a URL that doesn't match the configured path, Traefik returns HTTP 404 because it has no backend for that endpoint.

You can use the same approach to define dependencies such as databases in your Docker Compose file and build a complete local development environment. Download the full example here. Thanks for reading, please share with your friends and colleagues, and see you next time!

Docker for Go projects with multi-stage builds and Docker Compose

Marco Franssen

March 19, 2019

9 min read•1608 words

In a typical Go project, you'll probably start with a main.go file. I usually add a Dockerfile to build an image and a Docker Compose file to start dependencies such as databases and queues.

terminal

$ tree go-docker
go-docker
├── Dockerfile
├── docker-compose.yml
└── main.go
 
0 directories, 3 files

To start, we'll create a folder and initialize it as a Go module.

terminal

mkdir go-docker-webserver
cd go-docker-webserver
go mod init github.com/go-docker-webserver

I'd like to reuse the web server from my last post in the Docker setup we're about to create. For convenience, I've included the code below.

main.go

package main
 
import (
	"context"
	"flag"
	"log"
	"net/http"
	"os"
	"os/signal"
	"time"
)
 
var (
	listenAddr string
)
 
func main() {
	flag.StringVar(&listenAddr, "listen-addr", ":5000", "server listen address")
	flag.Parse()
 
	logger := log.New(os.Stdout, "http: ", log.LstdFlags)
 
	done := make(chan bool, 1)
	quit := make(chan os.Signal, 1)
 
	signal.Notify(quit, os.Interrupt)
 
	server := newWebserver(logger)
	go gracefullShutdown(server, logger, quit, done)
 
	logger.Println("Server is ready to handle requests at", listenAddr)
	if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
		logger.Fatalf("Could not listen on %s: %v\n", listenAddr, err)
	}
 
	<-done
	logger.Println("Server stopped")
}
 
func gracefullShutdown(server *http.Server, logger *log.Logger, quit <-chan os.Signal, done chan<- bool) {
	<-quit
	logger.Println("Server is shutting down...")
 
	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
	defer cancel()
 
	server.SetKeepAlivesEnabled(false)
	if err := server.Shutdown(ctx); err != nil {
		logger.Fatalf("Could not gracefully shutdown the server: %v\n", err)
	}
	close(done)
}
 
func newWebserver(logger *log.Logger) *http.Server {
	router := http.NewServeMux()
	router.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		logger.Println(r.Method, r.URL.Path, r.RemoteAddr, r.UserAgent())
		w.WriteHeader(http.StatusOK)
	})
 
	return &http.Server{
		Addr:         listenAddr,
		Handler:      router,
		ErrorLog:     logger,
		ReadTimeout:  5 * time.Second,
		WriteTimeout: 10 * time.Second,
		IdleTimeout:  15 * time.Second,
	}
}

Loading graceful-webserver/main.go…

Download this file and save it as main.go in our new go-docker-webserver folder. We should now be able to start the web server with go run. So far, so good. Now let's get to the Docker setup.

Docker

Dockerfile

FROM golang:1.12-alpine as builder
 
# To fix go get and build with cgo
RUN apk add --no-cache --virtual .build-deps \
    bash \
    gcc \
    git \
    musl-dev
 
RUN mkdir build
COPY . /build
WORKDIR /build
 
RUN go get
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o webserver .
RUN adduser -S -D -H -h /build webserver
USER webserver
 
FROM scratch
COPY --from=builder /build/webserver /app/
WORKDIR /app
EXPOSE 5000
CMD ["./webserver"]

Let's give this a quick try.

terminal

$ docker build -t go-docker-webserver:latest .
$ docker run --rm -p "5000:5000" go-docker-webserver
http: 2019/03/19 19:16:00 Server is ready to handle requests at :5000
http: 2019/03/19 19:24:41 GET / 172.17.0.1:50864 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
http: 2019/03/19 19:24:42 GET /favicon.ico 172.17.0.1:50864 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36

Take a look at the image size too. It's pretty small :).

terminal

$ docker images
REPOSITORY             TAG       IMAGE ID        CREATED          SIZE
go-docker-webserver    latest    8d5a7f09ec9e    2 minutes ago    7.43MB
.........

Docker-compose

Now let's look at a Docker Compose file. First, I'll show you how it makes building and running our Docker image easier.

docker-compose.yml

version: "3.7"
 
services:
  web:
    image: go-docker-webserver
    build: .
    ports:
      - "5000:5000"

We can now build and run our application with the following command.

terminal

$ docker-compose up --build
Creating network "graceful-webserver_default" with the default driver
Building web
Step 1/14 : FROM golang:1.12-alpine as builder
.........
.............
...........
......
Step 10/14 : FROM scratch
 --->
Step 11/14 : COPY --from=builder /build/webserver /app/
 ---> Using cache
 ---> a6d1afd58117
Step 12/14 : WORKDIR /app
 ---> Using cache
 ---> 8e482ff5aacf
Step 13/14 : EXPOSE 5000
 ---> Using cache
 ---> 2b19c892fa2b
Step 14/14 : CMD ["./webserver"]
 ---> Using cache
 ---> 8d5a7f09ec9e
 
Successfully built 8d5a7f09ec9e
Successfully tagged go-docker-webserver:latest
Creating graceful-webserver_web_1 ... done
Attaching to graceful-webserver_web_1
web_1  | http: 2019/03/19 19:36:11 Server is ready to handle requests at :5000

As you can see, this saves us some steps when building a new image and running it.

To show you more of what Docker Compose can do, let's add another container. We'll use Traefik, a handy router for our setup.

Simplifynetworking complexity while designing, deploying, and running applications.
Traefik - The cloud native edge Router

docker-compose.yml

version: "3.7"
 
services:
  api-gateway:
    image: traefik:1.7-alpine
    command: --api --docker
    ports:
      - "80:80"
      - "8888:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
 
  web:
    image: go-docker-webserver
    build: .
    labels:
      - "traefik.frontend.rule=Host:localhost; PathPrefixStrip: /api/v1/awesome/"

Let's start our Docker environment again. This time, I'll use the -d parameter to run it in the background without blocking our terminal.

terminal

$ docker-compose up --build -d
Creating network "graceful-webserver_default" with the default driver
Building web
Step 1/14 : FROM golang:1.12-alpine as builder
.........
.............
...........
......
Step 10/14 : FROM scratch
 --->
Step 11/14 : COPY --from=builder /build/webserver /app/
 ---> Using cache
 ---> a6d1afd58117
Step 12/14 : WORKDIR /app
 ---> Using cache
 ---> 8e482ff5aacf
Step 13/14 : EXPOSE 5000
 ---> Using cache
 ---> 2b19c892fa2b
Step 14/14 : CMD ["./webserver"]
 ---> Using cache
 ---> 8d5a7f09ec9e
 
Successfully built 8d5a7f09ec9e
Successfully tagged go-docker-webserver:latest
Creating graceful-webserver_api-gateway_1 ... done
Creating graceful-webserver_web_1         ... done

We can still view the web container's logs whenever we need them with docker-compose logs -f web.

terminal

$ docker-compose up -d --scale web=3
graceful-webserver_api-gateway_1 is up-to-date
Starting graceful-webserver_web_1 ... done
Creating graceful-webserver_web_2 ... done
Creating graceful-webserver_web_3 ... done

terminal

$ docker ps
CONTAINER ID   IMAGE                 COMMAND                  CREATED          STATUS         PORTS                                        NAMES
2b1a608d27e1   go-docker-webserver   "./webserver"            3 minutes ago    Up 3 minutes   5000/tcp                                     graceful-webserver_web_3
4b9e3a03312d   go-docker-webserver   "./webserver"            3 minutes ago    Up 3 minutes   5000/tcp                                     graceful-webserver_web_2
35454b1d1f78   traefik:1.7-alpine    "/entrypoint.sh --ap…"   4 minutes ago    Up 4 minutes   0.0.0.0:80->80/tcp, 0.0.0.0:8888->8080/tcp   graceful-webserver_api-gateway_1
7dcd45abeea2   go-docker-webserver   "./webserver"            4 minutes ago    Up 4 minutes   5000/tcp                                     graceful-webserver_web_1
 
$ docker-compose logs -f web
Attaching to graceful-webserver_web_3, graceful-webserver_web_2, graceful-webserver_web_1
web_1  | http: 2019/03/19 20:40:21 Server is ready to handle requests at :5000
web_1  | http: 2019/03/19 20:43:04 GET / 172.21.0.3:40670 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_3  | http: 2019/03/19 20:41:15 Server is ready to handle requests at :5000
web_2  | http: 2019/03/19 20:41:15 Server is ready to handle requests at :5000
web_2  | http: 2019/03/19 20:43:48 GET / 172.21.0.3:51194 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_3  | http: 2019/03/19 20:43:49 GET / 172.21.0.3:33668 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_1  | http: 2019/03/19 20:43:51 GET / 172.21.0.3:40680 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_2  | http: 2019/03/19 20:43:52 GET / 172.21.0.3:51194 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_3  | http: 2019/03/19 20:43:53 GET / 172.21.0.3:33668 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_1  | http: 2019/03/19 20:43:54 GET / 172.21.0.3:40680 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_2  | http: 2019/03/19 20:43:56 GET / 172.21.0.3:51194 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
web_3  | http: 2019/03/19 20:43:57 GET / 172.21.0.3:33668 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36

Blog.

Blog.

Docker for Go projects with multi-stage builds and Docker Compose

Docker

Docker-compose

Docker for Go projects with multi-stage builds and Docker Compose

Docker

Docker-compose

More Stories

Improve a Go web server with graceful shutdown, chi, and zap

Build a Go Elasticsearch client with Docker

Build a Go web server with graceful shutdown

Go interfaces and type assertions

More Stories

Improve a Go web server with graceful shutdown, chi, and zap

Build a Go Elasticsearch client with Docker

Build a Go web server with graceful shutdown

Go interfaces and type assertions