Loading...
Use the ACME DNS-01 challenge to get a TLS certificate
Marco Franssen
8 min read•1589 words
In my previous two posts, I showed you how to build an HTTP/2 web server, covering both self-signed TLS certificates and certificates from Let's Encrypt. I mentioned that you needed to expose the server publicly on the internet. But I've since found another way: the ACME DNS-01 challenge. Let's take a look.
Let's Encrypt is a free, automated, and open certificate authority brought to you by the nonprofit Internet Security Research Group (ISRG).
Let's Encrypt implements the ACME (Automated Certificate Management Environment) protocol. The challenge types documentation describes four challenge types, including one that has been retired. Let's briefly go through them and relate them to my previous posts before trying the DNS challenge.
HTTP-01 challenge
This challenge can only be performed on port 80. The client temporarily places a file on the web server at the following path:
/.well-known/acme-challenge/<TOKEN>
Once the ACME client tells Let's Encrypt the file is ready, Let's Encrypt retrieves it to validate the challenge. If validation succeeds, you receive the certificate. Your web server must be publicly reachable on port 80 for this challenge.
DNS-01 challenge
This challenge requires you to prove control of a domain name and also supports wildcard certificates. After receiving a token, the client creates a DNS TXT record at the following name:
_acme-challenge.<YOUR_DOMAIN>
Let's Encrypt queries this DNS record. Once it's verified, the client can request the certificate. Many DNS providers expose an API, making it possible to automate this process. Your web server doesn't need to be exposed to the internet, which makes this challenge convenient for issuing development certificates for a domain you own.
TLS-SNI-01 challenge
This challenge appeared in draft versions of ACME. It performed a TLS handshake on port 443 and sent a specific SNI header, looking for a certificate containing the token. It was disabled in March 2019 because it wasn't secure enough, so let's move on.
TLS-ALPN-01 challenge
This challenge runs on port 443 over TLS. It's mainly suited to authors of TLS-terminating reverse proxies who want host-based validation like HTTP-01, but entirely at the TLS layer to keep those concerns separate.
Recap
Now that we know a bit about the ACME challenge types, let's look at what we used in my earlier Go web server post. We had to expose the web server publicly to request a certificate because golang.org/x/crypto/acme/autocert doesn't implement the DNS-01 challenge. Instead, we used HTTP-01.
Let's now focus on requesting a certificate through DNS-01. As a Go fan, I found several other libraries and tools that implement ACME, including this challenge type.
Lego
Lego works as both a command-line tool and a library you can use in your own code. It supports a wide range of DNS providers.
Certmagic
CertMagic is the library used by the Caddy web server. In the version discussed here, it also supports Lego's DNS providers for the DNS-01 challenge.
Example
I'll use the Lego CLI to show you a small example of requesting a certificate for your domain. My DNS provider is Gandi LiveDNS (v5).
First, I'll install Lego from source using Go. The version used in this example requires Go 1.12+.
$ GO111MODULE=on go get -u github.com/go-acme/lego/v3/cmd/lego
go: found github.com/go-acme/lego/v3/cmd/lego in github.com/go-acme/lego/v3 v3.5.0
...
...
...
$ lego -h
NAME:
lego - Let's Encrypt client written in Go
USAGE:
lego [global options] command [command options] [arguments...]
VERSION:
dev
COMMANDS:
run Register an account, then create and install a certificate
revoke Revoke a certificate
renew Renew a certificate
dnshelp Shows additional help for the '--dns' global option
list Display certificates and accounts information.
help, h Shows a list of commands or help for one command
...
...
...If you don't have Go installed, there's also a Docker image available.
$ docker run goacme/lego -h
NAME:
lego - Let's Encrypt client written in Go
USAGE:
lego [global options] command [command options] [arguments...]
VERSION:
v3.5.0
COMMANDS:
run Register an account, then create and install a certificate
revoke Revoke a certificate
renew Renew a certificate
dnshelp Shows additional help for the '--dns' global option
list Display certificates and accounts information.
help, h Shows a list of commands or help for one command
...
...
...For Gandi LiveDNS, I need to provide my Gandi API key so Lego can communicate with the API. The key below is a dummy, so you'll need to use your own.
$ export GANDIV5_API_KEY=G4nD1v5L1v3DNSDummyK3y
$ lego --dns gandiv5 -d marcofranssen.nl -d '*.marcofranssen.nl' -a -m marco.franssen@gmail.com run
2020/04/11 13:31:42 No key found for account marco.franssen@gmail.com. Generating a P384 key.
2020/04/11 13:31:42 Saved key to /Users/marco/.lego/accounts/acme-v02.api.letsencrypt.org/marco.franssen@gmail.com/keys/marco.franssen@gmail.com.key
2020/04/11 13:31:43 [INFO] acme: Registering account for marco.franssen@gmail.com
!!!! HEADS UP !!!!
Your account credentials have been saved in your Let's Encrypt
configuration directory at "/Users/marco/.lego/accounts".
You should make a secure backup of this folder now. This
configuration directory will also contain certificates and
private keys obtained from Let's Encrypt so making regular
backups of this folder is ideal.
2020/04/11 13:36:11 [INFO] [mac-dev.marcofranssen.nl] acme: Obtaining bundled SAN certificate
2020/04/11 13:36:12 [INFO] [mac-dev.marcofranssen.nl] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/3894178945
2020/04/11 13:36:12 [INFO] [mac-dev.marcofranssen.nl] acme: Could not find solver for: tls-alpn-01
2020/04/11 13:36:12 [INFO] [mac-dev.marcofranssen.nl] acme: Could not find solver for: http-01
2020/04/11 13:36:12 [INFO] [mac-dev.marcofranssen.nl] acme: use dns-01 solver
2020/04/11 13:36:12 [INFO] [mac-dev.marcofranssen.nl] acme: Preparing to solve DNS-01
2020/04/11 13:36:13 [INFO] API response: DNS Record Created
2020/04/11 13:36:13 [INFO] [mac-dev.marcofranssen.nl] acme: Trying to solve DNS-01
2020/04/11 13:36:13 [INFO] [mac-dev.marcofranssen.nl] acme: Checking DNS record propagation using [8.8.8.8:53 8.8.4.4:53]
2020/04/11 13:36:13 [INFO] Wait for propagation [timeout: 20m0s, interval: 20s]
2020/04/11 13:36:14 [INFO] [mac-dev.marcofranssen.nl] acme: Waiting for DNS record propagation.
2020/04/11 13:36:34 [INFO] [mac-dev.marcofranssen.nl] acme: Waiting for DNS record propagation.
2020/04/11 13:36:54 [INFO] [mac-dev.marcofranssen.nl] acme: Waiting for DNS record propagation.
2020/04/11 13:37:14 [INFO] [mac-dev.marcofranssen.nl] acme: Waiting for DNS record propagation.
2020/04/11 13:37:41 [INFO] [mac-dev.marcofranssen.nl] The server validated our request
2020/04/11 13:37:41 [INFO] [mac-dev.marcofranssen.nl] acme: Cleaning DNS-01 challenge
2020/04/11 13:37:41 [INFO] [mac-dev.marcofranssen.nl] acme: Validations succeeded; requesting certificates
2020/04/11 13:37:42 [INFO] [mac-dev.marcofranssen.nl] Server responded with a certificate.Now let's inspect the certificate we received. There's also a .key file in the same location, which you'll need to deploy alongside the certificate on your web server. For security reasons, I won't share my private key here.
$ cat .lego/certificates/mac-dev.marcofranssen.nl.crt
-----BEGIN CERTIFICATE-----
MIIEuTCCA6GgAwIBAgISBLbVpvyGBSKWR+C/qPw+WAijMA0GCSqGSIb3DQEBCwUA
MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0yMDA0MTExMDM3NDJaFw0y
MDA3MTAxMDM3NDJaMCMxITAfBgNVBAMTGG1hYy1kZXYubWFyY29mcmFuc3Nlbi5u
bDB2MBAGByqGSM49AgEGBSuBBAAiA2IABJVkLy+89NjDUcJ7Ym0tOSODV8h/HrGF
29u7sTbg6yicu4uETIjNn5yNeFXVaz9NfJ39iiwPCkyP6G35g46YNqv5xAlg2uid
Io0jiaR/A4S/JX4loD+U0b2FUxXPCyUwpaOCAmwwggJoMA4GA1UdDwEB/wQEAwIH
gDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAd
BgNVHQ4EFgQUhTdwoXTajiDC0WkwLzIhO/0jV4YwHwYDVR0jBBgwFoAUqEpqYwR9
3brm0Tm3pkVl7/Oo7KEwbwYIKwYBBQUHAQEEYzBhMC4GCCsGAQUFBzABhiJodHRw
Oi8vb2NzcC5pbnQteDMubGV0c2VuY3J5cHQub3JnMC8GCCsGAQUFBzAChiNodHRw
Oi8vY2VydC5pbnQteDMubGV0c2VuY3J5cHQub3JnLzAjBgNVHREEHDAaghhtYWMt
ZGV2Lm1hcmNvZnJhbnNzZW4ubmwwTAYDVR0gBEUwQzAIBgZngQwBAgEwNwYLKwYB
BAGC3xMBAQEwKDAmBggrBgEFBQcCARYaaHR0cDovL2Nwcy5sZXRzZW5jcnlwdC5v
cmcwggEDBgorBgEEAdZ5AgQCBIH0BIHxAO8AdgDwlaRZ8gDRgkAQLS+TiI6tS/4d
R+OZ4dA0prCoqo6ycwAAAXFpB/hLAAAEAwBHMEUCIGngr8eDf1aGEuueWIzdx8GV
JGY5WhMOCM15Ij6VwJ29AiEA9WBTGf8xg3SoQXN3Sjgy/fYZu3Cs2lphHg3v0B4h
tH8AdQCyHgXMi6LNiiBOh2b5K7mKJSBna9r6cOeySVMt74uQXgAAAXFpB/g/AAAE
AwBGMEQCIEeoOJrnb1UH0XIorYLz9koKp4j0GVgqYgRvnTT517bnAiBgcrTEnvKw
tCe7R3Ci9XS8dcJDjQVXTEs4rPV4gCRW8TANBgkqhkiG9w0BAQsFAAOCAQEATcqt
lugYm7/7Z1R/IvBWct3E/bapMAhVz7alsClVeZHuv2vSX2LPgTp8ZXcgnf5RsXTw
iYJ/ZXhEtUwQfrPYGWYuXeINEbQ2hjKivGLYrWRfHw2MEC/R7Z1/5t6ODtUuYYbr
EHtCJ1I5XmmnqQvSoRBjbY1WtYskIZV2i1mk8eJv6z1FpBBkrrGFUaEXrw2HIBo1
K4zXwfxdnAb0PuBCVf+ds+EORhlMUWKf0yfSq9oAhZoitm4hS4BFC2ey8Y2dmuI9
I/EeVnxDPBFwWQ1Sbhinu0o1kjdYM2zO1nPZJjjyUDYJ+7Utq1zjvY7vENL7pmr/
pGu3XUZC2cD8LZRFkg==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIEkjCCA3qgAwIBAgIQCgFBQgAAAVOFc2oLheynCDANBgkqhkiG9w0BAQsFADA/
MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
DkRTVCBSb290IENBIFgzMB4XDTE2MDMxNzE2NDA0NloXDTIxMDMxNzE2NDA0Nlow
SjELMAkGA1UEBhMCVVMxFjAUBgNVBAoTDUxldCdzIEVuY3J5cHQxIzAhBgNVBAMT
GkxldCdzIEVuY3J5cHQgQXV0aG9yaXR5IFgzMIIBIjANBgkqhkiG9w0BAQEFAAOC
AQ8AMIIBCgKCAQEAnNMM8FrlLke3cl03g7NoYzDq1zUmGSXhvb418XCSL7e4S0EF
q6meNQhY7LEqxGiHC6PjdeTm86dicbp5gWAf15Gan/PQeGdxyGkOlZHP/uaZ6WA8
SMx+yk13EiSdRxta67nsHjcAHJyse6cF6s5K671B5TaYucv9bTyWaN8jKkKQDIZ0
Z8h/pZq4UmEUEz9l6YKHy9v6Dlb2honzhT+Xhq+w3Brvaw2VFn3EK6BlspkENnWA
a6xK8xuQSXgvopZPKiAlKQTGdMDQMc2PMTiVFrqoM7hD8bEfwzB/onkxEz0tNvjj
/PIzark5McWvxI0NHWQWM6r6hCm21AvA2H3DkwIDAQABo4IBfTCCAXkwEgYDVR0T
AQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAYYwfwYIKwYBBQUHAQEEczBxMDIG
CCsGAQUFBzABhiZodHRwOi8vaXNyZy50cnVzdGlkLm9jc3AuaWRlbnRydXN0LmNv
bTA7BggrBgEFBQcwAoYvaHR0cDovL2FwcHMuaWRlbnRydXN0LmNvbS9yb290cy9k
c3Ryb290Y2F4My5wN2MwHwYDVR0jBBgwFoAUxKexpHsscfrb4UuQdf/EFWCFiRAw
VAYDVR0gBE0wSzAIBgZngQwBAgEwPwYLKwYBBAGC3xMBAQEwMDAuBggrBgEFBQcC
ARYiaHR0cDovL2Nwcy5yb290LXgxLmxldHNlbmNyeXB0Lm9yZzA8BgNVHR8ENTAz
MDGgL6AthitodHRwOi8vY3JsLmlkZW50cnVzdC5jb20vRFNUUk9PVENBWDNDUkwu
Y3JsMB0GA1UdDgQWBBSoSmpjBH3duubRObemRWXv86jsoTANBgkqhkiG9w0BAQsF
AAOCAQEA3TPXEfNjWDjdGBX7CVW+dla5cEilaUcne8IkCJLxWh9KEik3JHRRHGJo
uM2VcGfl96S8TihRzZvoroed6ti6WqEBmtzw3Wodatg+VyOeph4EYpr/1wXKtx8/
wApIvJSwtmVi4MFU5aMqrSDE6ea73Mj2tcMyo5jMd6jmeWUHK8so/joWUoHOUgwu
X4Po1QYz+3dszkDqMp4fklxBwXRsW10KXzPMTZ+sOPAveyxindmjkW8lGy+QsRlG
PfZ+G6Z6h7mjem0Y+iWlkYcV4PIWL1iwBi8saCbGS5jN2p8M+X+Q7UNKEkROb3N6
KOqkqm57TH2H3eDJAkSnh6/DNFu0Qg==
-----END CERTIFICATE-----To inspect the certificate's contents, run the following OpenSSL command.
$ openssl x509 -in .lego/certificates/mac-dev.marcofranssen.nl.crt -noout -text
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
04:b6:d5:a6:fc:86:05:22:96:47:e0:bf:a8:fc:3e:58:08:a3
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=US, O=Let's Encrypt, CN=Let's Encrypt Authority X3
Validity
Not Before: Apr 11 10:37:42 2020 GMT
Not After : Jul 10 10:37:42 2020 GMT
Subject: CN=mac-dev.marcofranssen.nl
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (384 bit)
pub:
04:95:64:2f:2f:bc:f4:d8:c3:51:c2:7b:62:6d:2d:
39:23:83:57:c8:7f:1e:b1:85:db:db:bb:b1:36:e0:
eb:28:9c:bb:8b:84:4c:88:cd:9f:9c:8d:78:55:d5:
6b:3f:4d:7c:9d:fd:8a:2c:0f:0a:4c:8f:e8:6d:f9:
83:8e:98:36:ab:f9:c4:09:60:da:e8:9d:22:8d:23:
89:a4:7f:03:84:bf:25:7e:25:a0:3f:94:d1:bd:85:
53:15:cf:0b:25:30:a5
ASN1 OID: secp384r1
NIST CURVE: P-384
X509v3 extensions:
X509v3 Key Usage: critical
Digital Signature
X509v3 Extended Key Usage:
TLS Web Server Authentication, TLS Web Client Authentication
X509v3 Basic Constraints: critical
CA:FALSE
X509v3 Subject Key Identifier:
85:37:70:A1:74:DA:8E:20:C2:D1:69:30:2F:32:21:3B:FD:23:57:86
X509v3 Authority Key Identifier:
keyid:A8:4A:6A:63:04:7D:DD:BA:E6:D1:39:B7:A6:45:65:EF:F3:A8:EC:A1
Authority Information Access:
OCSP - URI:http://ocsp.int-x3.letsencrypt.org
CA Issuers - URI:http://cert.int-x3.letsencrypt.org/
X509v3 Subject Alternative Name:
DNS:mac-dev.marcofranssen.nl
X509v3 Certificate Policies:
Policy: 2.23.140.1.2.1
Policy: 1.3.6.1.4.1.44947.1.1.1
CPS: http://cps.letsencrypt.orgYou can also renew your certificate with Lego.
$ lego --dns gandiv5 -d mac-dev.marcofranssen.nl -a -m marco.franssen@gmail.com renew
2020/04/11 13:49:23 [mac-dev.marcofranssen.nl] The certificate expires in 89 days, the number of days defined to perform the renewal is 30: no renewal.Feel free to explore the other CLI options yourself. There are plenty of DNS providers available.
To use the certificate on your laptop during development, update your /etc/hosts file.
127.0.0.1 mac-dev.marcofranssen.nlThis lets me access my web server on localhost at https://mac-dev.marcofranssen.nl. See my other two posts for how to build an HTTPS web server in Go or serve it with Nginx. Use the crt and key files we just received from Let's Encrypt.
I've also included a reference to Traefik, which uses Lego to obtain certificates.
References
- Build a Go web server on HTTP/2 using Let's Encrypt
- React Router and Nginx over HTTP/2
- Let's Encrypt
- ACME challenge types
- Let’s Encrypt client and ACME library written in Go
- Lego DNS providers
- CertMagic: Easy and Powerful TLS Automation
- Caddy Server
- Traefik
Thanks for reading! Don't forget to share this post with your friends and colleagues. Together, we can make the web a safer place, starting with TLS in our development environments so we can use modern web features such as HTTP/2 and gRPC, which require TLS.
