Go client for Elasticsearch using Docker
Marco Franssen /
9 min read • 1736 words
In this blog post I would like to cover the recently released Elasticsearch 7.0-rc1 Go client for Elasticsearch. In this blogpost I want to show you a small example with a simple Docker setup using to build a Elasticsearch cluster.
In my previous blogpost I covered some Docker tips and tricks we will utilize again in this blog post.
To start with we first have to create a project folder. In this folder we will have to initialize our Go module, add our Dockerfile and docker-compose.yml. Once we have this in place we will be able to start our development. As I like as few mouseclicks as possible to get my projects bootstrapped I share below a few lines of shell script to easily create the outline of our project.
For our Dockerfile we will utilize the same definition as we discussed in our previous blogpost. See below for the contents of the Dockerfile.
In my previous blogpost I explain every single line of above Dockerfile, please have a look there if you want to have a better understanding on the details of this Dockerfile.
Next I would like to put in place the contents of our docker-compose.yml, so we have our runtime in place from an infra and packaging point of view.
As you can see we only open up the web container port to our host. That way all other services will not be directly accessible. We can access our api on port 5000.
Last but not least I would like to use the graceful webserver which I have written about earlier as a starting point to build our api on top of Elasticsearch.
Now with all of this in place we are able to boot up our infrastructure and have a first look on the setup from our browser.
NOTE: we didn't do anything yet with the Go Elasticsearch client, but we will come to that once we have seen our infrastructure is working so far.
Now we can run out solution by doing
docker-compose up --build It will build our api image and download the Elasticsearch docker containers. By accessing http://localhost:5000 you should be able to see a blank response with http status 200. You will also notice http://localhost:9200 is not available due to the fact we don't expose the port from our docker network.
Now we are ready to modify our webserver to interact with Elasticsearch. For that we will utilize the go-elasticsearch client we installed already in the first few steps using the shell commands. In order to use it we will add an import statement and I will add a new function to create a new Elasticsearch client.
… left for brevity …
Now we can use our function to create a Elasticsearch client from our main function, so we can use it on our http server. Lets add another commandline flag to configure the elasticsearch addresses and create the client from this.
With the last step I want to update our webserver function so we can return our Elasticsearch cluster info using the Elasticsearch client, when we call the endpoint. For that we will add the client as a parameter to our webserver and update the HttpHandler which registered at
/ to get the Elasticsearch cluster info.
In above example we are piping the Body of the Elasticsearch response to our httpWriter using
io.Copy(). So above example is a nice way for you to be able to add your own logic like authentication on top of elasticsearch and for the rest just directly pipe the elasticsearch response to your api response. This is very memory efficient as you will not take the full response into memory and directly pipe it on your http response.
Last thing left is to change the function call to newWebserver, so we provide the Elasticsearch client to the function.
Let's try out our example by running
docker-compose up --build -d, this will rebuild web container and run Docker in background so we can more easily continue development on next example. To get access to console logs you can do
docker-compose logs -f or
docker-compose logs -f web to filter out only web container logs.
To give you a more tensible example I would like to show you how to search an indice as a starting point for your Elasticsearch Go journey.
First we will define a new HttpHandlerFunc on our http router which will forward out search request to ElasticSearch. In addition I will add a small helper function to build an elasticsearch query and some constant string for the constant part of our query.
Above code will add a handler for a
/search endpoint that will take the querystring parameter
q to build up the Elasticsearch search request which will utilize the advanced POST request where the query will be formed as a JSON body. Let's try it out,
docker-compose up --build -d. Now when you try to hit the API you will notice we are getting an error response as we don't have a index called people yet. Try for yourself at http://localhost:5000/search?q=marco.
To resolve this we will add a bit more code to our project to create the index and add some initial records to Elasticsearch.
Last but not least call this bootstrap function in your main function right after creating the Elasticsearch client.
Now we can run the solution again and hit some searches.
docker-compose up -d --build and then run
docker-compose logs -f to see your requests reach your handler. As you might have noticed before we only made it possible to search by firstname, lastname, country and title. For those searchable fields we also defined priorities in case your query matches one multiple of those fields to calculate the score. Some queries you could try for example:
For this last search you will not find any results as we don't match on email addresses. Now go ahead yourself by editing the query template to also be able to search by email address, then rerun your code using
docker-compose up -d --build to give the last search another try.
A collection of resources you might want to checkout to enrich your knowledge.
- Go Docker Tips and Tricks
- Go webserver with Graceful shutdown
- Elasticsearch Docker image
- Go Elasticsearch github project
Thank you for reading my blogpost. I hope this gave you a good starting point to start developing your own Go Elasticsearch projects. Please share it on social media with your friends and colleagues. Oh and before I forget, you can download the entire solution here as a zip.