Loading...
Test and benchmark your code in Go
Marco Franssen
9 min read•1736 words
When I started writing my first programs in Go, I noticed that the tooling ships with testing and benchmarking features out of the box. Follow a few filename conventions, import the standard "testing" package, aaaand… ready, set, off you Go! In my earlier blog post, I briefly covered writing a test in Go. I recommend starting there if you're completely new to the language.
The conventions
In Go, you put your tests in the same folder as the code. Test filenames end in _test.go. Don't worry: the tests won't be included in your compiled binary. Let's start with an example folder structure to see what that looks like.
$ tree my-awesome-project
my-awesome-project
├── api
│ ├── handlers.go
│ ├── handlers_test.go
│ ├── routes.go
│ └── routes_test.go
├── main.go
├── main_test.go
├── storage.go
└── storage_test.go
1 directory, 8 filesAt first, I had to get used to this structure. I'd always put tests in separate packages in Java, separate assemblies in C#, or a test folder in Javascript projects. I resisted it a bit in the beginning, but as I wrote more code, I found that it helped me practice TDD. Keeping tests close to the code encouraged me to follow TDD even more strictly, with less effort spent navigating folders and files. The structure above shows a main package containing main.go and storage.go, and an api package containing routes and their handlers.
If you've recently worked on React web development, you might have noticed developers using a similar structure, keeping tests in the same folder as the component.
Writing tests
In Go, tests go in a file named after the code you're testing. For example, tests for main.go go in main_test.go. The test file needs to import the testing package. Each test has the signature func TestXxx(*testing.T), where the first X must be uppercase.
package main
import "testing"
func TestSomething(t *testing.T) {
expected := "Something Awesome Happened!"
something := Something("Awesome")
if something != expected {
t.Errorf("Expected %s, got %s", expected, something)
}
}
Benchmarking
Go also has built-in benchmarking support as part of the testing package. A benchmark has a signature similar to a test: func BenchmarkXxx(*testing.B), where the first X must also be uppercase.
func BenchmarkSomething(b *testing.B) {
for i := 0 ; i < b.N ; i++ {
Something("Awesome")
}
}Example
In this example, we'll define a struct to use as a data transfer object (DTO) for JSON responses from our API. Let's approach this with TDD. I usually start by creating the files that will contain my code.
- api/models.go
- api/models_test.go
First, I'll write some tests to describe what I want to achieve.
package api
import "testing"
var awesomeJSON = []byte(`{
"id": "123456789",
"message": "Total awesomeness",
"score": 9.99,
"confirmed": true
}`)
func TestAwesomeToJSON(t *testing.T) {
awesome := NewAwesome("123456789", "Total awesomeness", 9.99, true)
testJSON, err := awesome.ToJSON()
if err != nil {
t.Error("Failed to create json from awesome")
}
if string(testJSON) != string(awesomeJSON) {
t.Errorf("JSON output\n%s\nis not as expected\n%s", testJSON, awesomeJSON)
}
}
func TestAwesomeFromJSON(t *testing.T) {
awesome := NewAwesomeFromJSON(awesomeJSON)
if awesome == nil {
t.Error("Unmarshalling json into awesome failed")
}
if awesome.Id != "123456789" {
t.Error("Awesome Id does not match expected value")
}
if awesome.Message != "Total awesomeness" {
t.Error("Awesome Message does not match expected value")
}
if awesome.Score != 9.99 {
t.Error("Awesome Id does not match expected value")
}
if !awesome.Confirmed {
t.Error("Awesome Confirmed does not match expected value")
}
}With the tests in place, I can define the struct, constructor functions and instance method.
package api
import "encoding/json"
type Awesome struct {
Id string `json:"id"`
Message string `json:"message"`
Score float64 `json:"score"`
Confirmed bool `json:"confirmed"`
}
func NewAwesome(id string, message string, score float64, confirmed bool) *Awesome {
return &Awesome{
id,
message,
score,
confirmed,
}
}
func NewAwesomeFromJSON(jsonData []byte) *Awesome {
var awesome *Awesome
err := json.Unmarshal(jsonData, &awesome)
if err != nil {
return nil
}
return awesome
}
func(a *Awesome) ToJSON() ([]byte, error) {
return json.Marshal(a)
}Now we're ready to run our tests and see whether the code produces the expected output.
$ go test ./api
--- FAIL: TestAwesomeToJSON (0.00s)
models_test.go:24: JSON output
{"id":"123456789","message":"Total awesomeness","score":9.99,"confirmed":true}
is not as expected
{
"id": "123456789",
"message": "Total awesomeness",
"score": 9.99,
"confirmed": true
}
FAIL
FAIL test-go/api 0.296sAs you can see, one of our tests is failing. We expected nicely formatted JSON, but got compact JSON without formatting. We have two options: change our requirement (the test), or change the implementation to format the output. I'll provide a solution for the failing test later in this post. Before that, I'd like to take a closer look at benchmarking.
Let's also add benchmarks to measure the performance of our implementation.
func BenchmarkAwesomeFromJSON(b *testing.B) {
for i := 0; i < b.N; i++ {
NewAwesomeFromJSON(awesomeJSON)
}
}
func BenchmarkAwesomeToJSON(b *testing.B) {
for i := 0; i < b.N; i++ {
awesome := NewAwesome("123456789", "Total awesomeness", 9.99, true)
awesome.ToJSON()
}
}Now let's run our benchmarks. If you still have the failing test, you'll notice that the benchmarks don't run. No need to benchmark broken code, right? If you haven't taken the previous challenge yet, you can temporarily skip the failing test using t.Skip().
$ go test -bench=. ./api
goos: windows
goarch: amd64
pkg: github.com/marcofranssen/test-go/api
BenchmarkAwesomeFromJSON-8 500000 3736 ns/op
BenchmarkAwesomeToJSON-8 1000000 1189 ns/op
PASS
ok github.com/marcofranssen/test-go/api 3.408sThe -8 in these benchmark names means they ran with GOMAXPROCS=8. Running with only one CPU would probably make little difference here, since our code doesn't use parallelism.
If you haven't taken the previous challenge, here's another chance. Let's add support for both formatted and unformatted JSON, then compare their performance by adding a benchmark for formatted JSON.
Once you're ready, keep reading. I'll provide a solution to the challenge as well.
In the following example, I've put all the code in main.go so you can easily try it in the playground.
package main
import (
"encoding/json"
"fmt"
)
type Awesome struct {
Id string `json:"id"`
Message string `json:"message"`
Score float64 `json:"score"`
Confirmed bool `json:"confirmed"`
}
func NewAwesome(id string, message string, score float64, confirmed bool) *Awesome {
return &Awesome{
id,
message,
score,
confirmed,
}
}
func NewAwesomeFromJSON(jsonData []byte) *Awesome {
var awesome *Awesome
err := json.Unmarshal(jsonData, &awesome)
if err != nil {
return nil
}
return awesome
}
func(a *Awesome) ToJSON(pretty bool) ([]byte, error) {
if pretty {
return json.MarshalIndent(a, "", " ")
} else {
return json.Marshal(a)
}
}
func main() {
awesome := NewAwesome("123456789", "Total awesomeness", 9.99, true)
awesomeJSON, _ := awesome.ToJSON(false)
fmt.Printf("%s\n", awesomeJSON)
moreAwesomeJSON, _ := awesome.ToJSON(true)
fmt.Printf("%s\n", moreAwesomeJSON)
}I've also updated the benchmark and test.
func TestAwesomeToJSON(t *testing.T) {
t.Skip()
awesome := NewAwesome("123456789", "Total awesomeness", 9.99, true)
testJSON, err := awesome.ToJSON(true)
if err != nil {
t.Error("Failed to create json from awesome")
}
if string(testJSON) != string(awesomeJSON) {
t.Errorf("JSON output\n%s\nis not as expected\n%s", testJSON, awesomeJSON)
}
}
func BenchmarkAwesomeToJSON(b *testing.B) {
for i := 0; i < b.N; i++ {
awesome := NewAwesome("123456789", "Total awesomeness", 9.99, true)
awesome.ToJSON(false)
}
}
func BenchmarkAwesomeToJSONPretty(b *testing.B) {
for i := 0; i < b.N; i++ {
awesome := NewAwesome("123456789", "Total awesomeness", 9.99, true)
awesome.ToJSON(true)
}
}Let's run the updated benchmarks to compare the performance of both implementations.
$ go test -bench=. ./api
goos: windows
goarch: amd64
pkg: github.com/marcofranssen/test-go/api
BenchmarkAwesomeFromJSON-8 500000 2685 ns/op
BenchmarkAwesomeToJSON-8 2000000 862 ns/op
BenchmarkAwesomeToJSONPretty-8 500000 2308 ns/op
PASS
ok github.com/marcofranssen/test-go/api 5.409sAs you can see, nicely formatted JSON with newlines and two-space indentation is a factor of four slower than unformatted JSON.
Bonus
Below, I'll show you a few more options for running tests and benchmarks.
Test
The following command runs the tests with code coverage.
$ go test -covermode=atomic -coverprofile=coverage.out ./...
? github.com/marcofranssen/test-go [no test files]
ok github.com/marcofranssen/test-go/api 0.259s coverage: 77.8% of statements-covermode=atomic is the safest option in multithreaded environments. ./... runs the tests for all packages in the project.
The following command runs the tests using GOMAXPROCS=1 and GOMAXPROCS=4.
$ go test -cpu 1,4 ./...
ok github.com/marcofranssen/test-go 0.286sAssertions
As you've noticed, the standard testing package doesn't include an assertion library. If you want assertion functions like those in other languages, you can install a third-party package, such as Testify.
This lets you write assertions in the following style.
package main
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestSomething(t *testing.T) {
assert := assert.New(t)
var a string = "Hello"
var b string = "Hello"
assert.Equal(a, b, "The two words should be the same.")
}Testify also offers other features, including mocking.
Benchmark
The following command runs the benchmarks for 15 seconds rather than the default of one second. It runs each benchmark twice for every CPU configuration we specify, and only selects benchmarks whose names match the regular expression. Using . as the expression would select all benchmarks.
$ go test -bench=^BenchmarkAwesomeToJSON.* -benchtime 15s -count 2 -cpu 1,4 ./api
goos: windows
goarch: amd64
BenchmarkAwesomeToJSON 30000000 870 ns/op
BenchmarkAwesomeToJSON 30000000 855 ns/op
BenchmarkAwesomeToJSON-4 30000000 859 ns/op
BenchmarkAwesomeToJSON-4 30000000 851 ns/op
BenchmarkAwesomeToJSONPretty 10000000 2476 ns/op
BenchmarkAwesomeToJSONPretty 10000000 2446 ns/op
BenchmarkAwesomeToJSONPretty-4 10000000 2468 ns/op
BenchmarkAwesomeToJSONPretty-4 10000000 2399 ns/op
PASS
ok github.com/marcofranssen/test-go/api 214.339sFor more complicated scenarios, these options can give you a more reliable report by running the benchmarks for longer and repeating them. As predicted earlier, using one or multiple CPUs makes little difference here. Notice that the benchmarks using four CPUs have a -4 suffix. Also note that pretty formatting now appears to be only a factor of three slower.
For more options, check go test --help. There are options for creating memory and CPU profiles, for example, which let you analyze performance. I'm looking forward to your feedback: there's never a limit to what a person can learn.
The full solution can be downloaded here.
