Build a Go web server with HTTP/2 and Let's Encrypt

I often see developers struggle to set up a web server with HTTPS. Why use HTTPS during development? In this tutorial, I want to test HTTP/2 features such as server push through the http.Pusher interface early in the development process. I'll show you how I set up a Go HTTP/2 server using Let's Encrypt, with a self-signed certificate for working offline.

In my previous post, I showed you how to use self-signed certificates in Nginx to enable HTTP/2 features. I also wrote a post some time ago about getting a Let's Encrypt certificate for your Azure website.

To retrieve a certificate with Let's Encrypt in this setup, your server has to be publicly reachable. Let's Encrypt provides trusted certificates for free. Here, we'll build support for both self-signed and Let's Encrypt certificates into our Go web server.

Self signed certificates

To start developing on my offline machine, we'll first create a self-signed certificate.

mkdir -p certs
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
    -keyout certs/localhost.key -out certs/localhost.crt \
    -subj "/C=NL/ST=Limburg/L=Geleen/O=Marco Franssen/OU=Development/CN=localhost/emailAddress=marco.franssen@gmail.com"

Next, we'll write a small web server that uses this self-signed certificate.

package main
 
import (
  "crypto/tls"
  "fmt"
  "net/http"
)
 
func main() {
  mux := http.NewServeMux()
  mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    fmt.Fprint(w, "Hello HTTP/2")
  })
 
  server := http.Server{
    Addr:    ":443",
    Handler: mux,
    TLSConfig: &tls.Config{
      NextProtos: []string{"h2", "http/1.1"},
    },
  }
 
  fmt.Printf("Server listening on %s", server.Addr)
  if err := server.ListenAndServeTLS("certs/localhost.crt", "certs/localhost.key"); err != nil {
    fmt.Println(err)
  }
}

playground

Time to give this a try.

go run .

For Google Chrome to accept the self-signed certificate in this setup, enable allow-insecure-localhost by navigating to the localhost flag in your address bar. You can also navigate to the localhost site settings. At the bottom, you can Allow the Insecure Content option to use the self-signed certificate.

To allow only the currently blocked certificate, type thisisunsafe while the Your connection is not private page has focus. The page will refresh automatically once you've typed the full phrase. In older versions of Chrome, you had to type badidea or danger.

Now, when we run our server and navigate to https://localhost, you'll see the Hello HTTP/2 message. In the developer tools' Network tab, the protocol column also shows that the page is served over HTTP/2.

If you'd like to compare this with HTTP/1.1, you can run the server on port :80 without TLS. To do so, add the following line.

  go http.ListenAndServe(":80", mux)
  if err := server.ListenAndServeTLS("certs/localhost.crt", "certs/localhost.key"); err != nil {
    fmt.Println(err)
  }

Let's try this out.

go run .

This runs the same handler on http://localhost. In the Network tab, the protocol column shows that the page is now served over HTTP/1.1.

A best practice is to redirect http traffic to https. Users don't have to remember to type https://, and we still serve the page over https.

We can achieve that by replacing the line we just added with the following.

  go http.ListenAndServe(":80", http.HandlerFunc(redirectHTTP))

At the top of main.go, we'll also add two functions: redirectHTTP, used in the previous code sample, and a small utility function, stripPort.

import(
  "crypto/tls"
  "fmt"
  "net"
  "net/http"
)
 
func redirectHTTP(w http.ResponseWriter, r *http.Request) {
  if r.Method != "GET" && r.Method != "HEAD" {
    http.Error(w, "Use HTTPS", http.StatusBadRequest)
    return
  }
  target := "https://" + stripPort(r.Host) + r.URL.RequestURI()
  http.Redirect(w, r, target, http.StatusFound)
}
 
func stripPort(hostport string) string {
  host, _, err := net.SplitHostPort(hostport)
  if err != nil {
    return hostport
  }
  return net.JoinHostPort(host, "443")
}

Let's give these changes a try.

go run .

Now when you navigate to http://localhost, you will automatically be redirected to https://localhost.

Autocert

Great, now that this is working, it's time to retrieve a valid certificate for production through Let's Encrypt. We can do this using the golang.org/x/crypto/acme/autocert package. The following lines configure autocert to retrieve a certificate.

  certManager := autocert.Manager{
    Prompt:     autocert.AcceptTOS,
    HostPolicy: autocert.HostWhitelist(domain),
    Cache:      autocert.DirCache("certs"),
  }
 
  tlsConfig := certManager.TLSConfig()
  server := http.Server{
    Addr:    ":443",
    Handler: mux,
    TLSConfig: tlsConfig,
  }

We also need to provide one or more domains in the HostPolicy to specify which certificate requests are allowed. With the following lines, I let you set the domain from the command line when starting the server.

var (
  domain string
)
 
func main() {
  flag.StringVar(&domain, "domain", "", "domain name to request your certificate")
  flag.Parse()
 
  // left for brevity
}

When we run our server, we can provide the domain name through this flag. Of course, the server has to be publicly accessible for Let's Encrypt to issue the certificate. If you're working from home, configure your router to forward ports 80 and 443 to your laptop's IP address. Also create a DNS entry for mydomain.com pointing to your home's public IP address. Otherwise, this setup won't work.

go run . -domain mydomain.com

If we try to run the server at https://localhost, it fails because Let's Encrypt doesn't consider localhost a valid domain name for a certificate. You might also be traveling and unable to make your server publicly available, but still want to develop locally. We'll customize the GetCertificate function to look for an existing self-signed certificate first.

func getSelfSignedOrLetsEncryptCert(certManager *autocert.Manager) func(hello *tls.ClientHelloInfo) (*tls.Certificate, error) {
  return func(hello *tls.ClientHelloInfo) (*tls.Certificate, error) {
    dirCache, ok := certManager.Cache.(autocert.DirCache)
    if !ok {
      dirCache = "certs"
    }
 
    keyFile := filepath.Join(string(dirCache), hello.ServerName+".key")
    crtFile := filepath.Join(string(dirCache), hello.ServerName+".crt")
    certificate, err := tls.LoadX509KeyPair(crtFile, keyFile)
    if err != nil {
      fmt.Printf("%s\nFalling back to Letsencrypt\n", err)
      return certManager.GetCertificate(hello)
    }
    fmt.Println("Loaded selfsigned certificate.")
    return &certificate, err
  }
}

This function looks in the configured certs directory for a .key and .crt pair. If it can load them, it uses that certificate; otherwise, it tries to get one through Let's Encrypt.

To use this function, we add the following line of code.

  // left for brevity
  tlsConfig := certManager.TLSConfig()
  tlsConfig.GetCertificate = getSelfSignedOrLetsEncryptCert(&certManager)
  // left for brevity

Now you can test this approach with different certificates by generating another one for a custom domain.

openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
    -keyout certs/dev.local.io.key -out certs/dev.local.io.crt \
    -subj "/C=NL/ST=Limburg/L=Geleen/O=Marco Franssen/OU=Development/CN=dev.local.io/emailAddress=marco.franssen@gmail.com"

Also make sure you update your hosts file.

127.0.0.1 localhost dev.local.io

Now run the solution as follows.

go run . -domain dev.local.io
# or
go build .
./go-web-letsencrypt -domain dev.local.io

When opening http://dev.local.io, you'll notice that the redirect to https://dev.local.io still works and the self-signed certificate is used. The same applies to the certificate we created earlier for https://localhost.

Now that you have an HTTP/2-capable server, you can use Go's http.Pusher.

TL;DR

You'll find the complete solution from this post in the following code block.

Run in development

For development, you can use self-signed certificates for https://localhost or a custom domain such as https://dev.local.io. This lets you work offline or in an environment you can't expose to the public internet.

openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
    -keyout certs/dev.local.io.key -out certs/dev.local.io.crt \
    -subj "/C=NL/ST=Limburg/L=Geleen/O=Marco Franssen/OU=Development/CN=dev.local.io/emailAddress=marco.franssen@gmail.com"
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
    -keyout certs/localhost.key -out certs/localhost.crt \
    -subj "/C=NL/ST=Limburg/L=Geleen/O=Marco Franssen/OU=Development/CN=localhost/emailAddress=marco.franssen@gmail.com"

Ensure custom domains are added to your /etc/hosts file.

127.0.0.1 localhost dev.local.io

Once you have your certificates, run the server as follows.

go run . -domain localhost
# or
go run . -domain dev.local.io
# or
go build .
./go-web-letsencrypt -domain localhost
# or
./go-web-letsencrypt -domain dev.local.io

Now simply navigate to http://localhost or http://dev.local.io.

Run on production

For production, host the solution on a publicly accessible endpoint. Don't create a self-signed certificate for this domain. Make sure your DNS record points to the correct public IP address and that ports 80 and 443 are forwarded to the machine running the web server.

Then simply launch the server.

go run . -domain fictional-domain.marcofranssen.nl
# or
go build .
./go-web-letsencrypt -domain fictional-domain.marcofranssen.nl

Now that you have an HTTP/2-capable server, you can use Go's http.Pusher.

References

• http.Pusher
• Let's Encrypt

Also consider adding Graceful shutdown to your server.

Please share this post with your friends. I'm looking forward to your feedback, and I hope to see you back next time.