Secretless authentication and keyless signing with SPIFFE Vault

Securing the software supply chain has been a hot topic lately. Many projects focus on adding security to the supply chain and zero-trust capabilities to the infrastructure running our software.

In this post, I want to introduce a small command-line utility, spiffe-vault, that enables use cases such as:

• Secretless deployments
• Keyless code signing
• Keyless encryption

spiffe-vault combines two projects for this secretless and keyless approach: SPIFFE and HashiCorp Vault.

The project was founded on 25 August 2021. Why am I only blogging about spiffe-vault now?

Honestly, there's no good reason beyond not having made the time. One of my community followers, Batuhan Apaydın (a.k.a. developer-guy), also explicitly asked me to write about it.

This might sound a bit fuzzy at first, but bear with me. I'll introduce the concepts and explain why I built the tool before diving into the implementation.

Let's start with some threats to the software supply chain.

This diagram from the SLSA project shows several points where our software supply chain can be attacked. At many of these steps, we use secrets as part of the measures to protect ourselves.

Examples of these secrets are:

• The credentials to authenticate to a container registry
• The passphrase for our signing/encryption key
• The API keys to run an integration test against an API
• The credentials to push our release assets to artifact storage
• The secrets to deploy our release to our dev/qa/prod environments
• …

So why do we need all these secrets? To protect against many of these attacks, we want to verify the operations we perform. That's the idea behind zero trust: trust nothing, verify everything. For the rest of this article, I'll focus on keyless code signing to explain how spiffe-vault works.

The signature on a container image or another release asset is only as trustworthy as the way we protect our signing key. If that key is compromised, an attacker could inject malicious code into a release, sign it, and make it look like it came from the project's maintainer. So we need to store the key securely and keep it out of other people's hands. We also protect it with a strong passphrase in case someone does get hold of it. Great, now we're all set, right?

Unfortunately, no. What if I accidentally leak the passphrase too? I hear you: I should store it securely in a password manager such as KeePass or LastPass. But accessing that tool requires another password, which I also need to protect. I could keep going with this story forever. This is called the bottom turtle problem, and it's where SPIFFE comes in to solve it.

What if we could sign code without handing our workload a signing key or a passphrase to unlock it? That's the keyless and secretless approach I'm referring to here.

SPIFFE

SPIFFE lets our infrastructure assign cryptographically verifiable identities to workloads. For example, we could register a self-hosted GitHub Actions runner to receive a SPIFFE ID. We can then use that identity to authenticate with HashiCorp Vault without a preconfigured secret, with cryptographic verification before access is granted. For this setup, you'll also need SPIRE, the reference implementation of SPIFFE. Its components issue workload identities and provide the verification mechanisms.

To make deploying SPIRE on Kubernetes easier, I've also open-sourced a Helm chart.

HashiCorp Vault

Another prerequisite is a HashiCorp Vault server in your infrastructure. This small piece of Terraform configures the transit engine for code signing and enables JWT authentication for our SPIFFE-based identities. If you're less familiar with Terraform, here's the equivalent setup.

vault secrets enable transit
vault write -f transit/keys/cosign type=ecdsa-p256
vault auth enable jwt
vault write auth/jwt/config oidc_discovery_url=http://spire-oidc.my-spire default_role=""
vault policy write local-policy -<<EOF
path "transit/keys" {
   capabilities = ["list"]
}
path "transit/keys/cosign" {
   capabilities = ["read"]
}
path "transit/hmac/cosign/*" {
   capabilities = ["update"]
}
path "transit/sign/cosign/*" {
   capabilities = ["update"]
}
path "transit/sign/cosign/sha1" {
   capabilities = ["deny"]
}
path "transit/verify/cosign" {
   capabilities = ["write"]
}
path "transit/verify/cosign/sha1" {
   capabilities = ["deny"]
}
EOF
vault write auth/jwt/role/local role_type=jwt user_claim=sub bound_audiences=CI bound_subject=spiffe://dev.localhost/ns/my-app/sa/my-app-spiffe-vault token_ttl=60s token_policies=local-policy

This enables JWT authentication and configures it to use the SPIRE OIDC endpoint deployed by the Helm chart. It also configures a JWT role with access to the transit engine. The policy allows signature creation and verification but denies the insecure sha1 hashing algorithm.

Now we can use the Cosign signing key managed by HashiCorp Vault to sign our Docker images, without providing a Vault authentication secret in our CI pipeline.

$ export VAULT_ADDR=http://localhost:8200
$ eval "$(spiffe-vault auth -role local)"
$ cosign sign -key hashivault://cosign marcofranssen/busybox:latest
Pushing signature to: index.docker.io/marcofranssen/busybox:sha256-febcf61cd6e1ac9628f6ac14fa40836d16f3c6ddef3b303ff0321606e55ddd0b.sig
$ cosign verify -key hashivault://cosign marcofranssen/busybox:latest
 
Verification for marcofranssen/busybox:latest --
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key
  - Any certificates were verified against the Fulcio roots.
 
[{"critical":{"identity":{"docker-reference":"index.docker.io/marcofranssen/busybox"},"image":{"docker-manifest-digest":"sha256:febcf61cd6e1ac9628f6ac14fa40836d16f3c6ddef3b303ff0321606e55ddd0b"},"type":"cosign container image signature"},"optional":null}]

In the HashiCorp Vault setup, we give the token a lifetime of only 60 seconds. For this particular use case, we could reduce it to a few seconds, limiting the time someone could abuse a leaked token.

The spiffe-vault auth -role local command succeeds only if the workload, whether a server or Docker container, has the required SPIFFE identity. In this Kubernetes setup, registration is handled by the k8s-registrar component.

Run it yourself

The complete example, including automation scripts and detailed steps, is available below. It lets you deploy the setup on a Kubernetes cluster.

github.com/philips-labs/spiffe-vault/tree/main/example#readme

What if you'd rather use your own EC2 instance for a self-hosted GitHub runner that registers with the SPIRE server? This guide is a good starting point: spiffe.io/docs/latest/try/getting-started-linux-macos-x.

Wrapping up

In short, spiffe-vault lets you authenticate with HashiCorp Vault on the command line without providing a password or another preconfigured authentication secret. Once authenticated, you can run your usual vault commands or use the REST API to build your own use cases. That's useful for CI/CD. You could retrieve secrets at deployment time and inject them into a backend, but I recommend making the backend SPIFFE-aware instead and integrating the few lines of code from this tool. Then it can retrieve secrets, encryption keys, and mTLS certificates from HashiCorp Vault on demand.

authPath := "jwt"
audience := "CI"
role := "dev"
 
jwt, err := spiffe.FetchJWT(ctx, audience)
if err != nil {
  return err
}
 
c, err := vault.NewClient(authPath)
if err != nil {
  return err
}
 
err = c.Authenticate(jwt, role)
if err != nil {
  return err
}
 
// Now utilize any hashicorp Vault features you need, e.g. encrypt some data or retrieve a secret

That way, you don't need to distribute plaintext secrets through deployment configuration or environment variables.

References

• spiffe-vault
• The complete example on how to use spiffe-vault
• SPIRE Helm chart

Background materials:

• Solving the bottom turtle
• SPIFFE
• SPIRE
• Kubernetes
• HashiCorp Vault