Packer.io machine building and provisioning part 2

In the previous part of this series we had a look on building a bare Debian VM with the bare minimum packages installed to run a web server. In this part we will have a look on how we can improve our packer script with user variables and how to use the file and shell provisioner.

User variables

Variables can be easily added to the packer script by adding following JSON.

{
  "variables": {
    "username": "root",
    "password": "r00tme",
    "memory": "1024",
    "cpus": "1",
    "database_name": "{{env `DB_NAME`}}"
  },
  "builders": [{
    // Left for brevity
}

Best practice is to put your variables as the first property in your JSON, before your builders. This way you have all the configurable values to your script quickly accessible. As you can see we define for each variable a default value, which will be used as the default when the user doesn't provide one. For the database_name variable I used a special default. This default will be retrieved from your environment variables. You can set this kind of variable just as you would set any other variable from your command line/shell.

set DB_NAME=mydatabase

export DB_NAME=mydatabase

When the environment variable is not set the value of database_name will be an empty string. Environment variables can only be used in your user variables and nowhere else in your template. This to prevent confusion about possible input for the template. When you want to override the other values during the build of your packer VM, you need to set them when executing the packer build. This can be done using following command.

packer build \
  -var 'cpus=1' \
  -var 'memory=512' \
  packer-debian-x64-webserver.json

All the user variables not overridden here will use the default value. You can inspect your template by running the following command from your command line/shell.

packer inspect packer-debian-x64-webserver.json

The command will show you the contents of your template. In our case our variables and our VirtualBox-iso builder.

So now we know how to define and use the user variables I only need to explain you how to get the value of these user variables in the template. To do so I want you to replace the following parts of the packer template with the following JSON.

  "ssh_username": "{{user `username`}}",
  "ssh_password": "{{user `password`}}",
  //Left for brevity...
  "vboxmanage": [
    ["modifyvm", "{{.Name}}", "--memory", "{{user `memory`}}"],
    ["modifyvm", "{{.Name}}", "--cpus", "{{user `cpus`}}"],
    ["modifyvm", "{{.Name}}", "--vram", "10"]
  ],

Now the values for ssh_username, ssh_password, --memory and --cpus will be populated with the values from our user variables. Feel free to also make your disk_size configurable via a user variable. So take this minute to apply the things you just learned to also have the disk_size as a user variable.

Provisioning

In order to execute shell scripts on our VM we need these scripts available on our VM. The easiest way to do so is by using the file provisioner to upload the script files to our VM. As soon as the scripts are uploaded to our VM we can execute them using the shell provisioner. We probably also want our source files for our webpage available on our VM. After the builder property in our template we will now add the following provisioners property.

  "builders": [{
    //Left for brevity...
  }],
  "provisioners": [{
    "type": "file",
    "source": "scripts",
    "destination": "/tmp"
  }, {
    "type": "shell",
    "script": "prepare_data_folder"
  }, {
    "type": "file",
    "source": "webpage",
    "destination": "/data"
  }, {
    "type": "shell",
    "script": "setup_database",
    "environment_vars": [
      "DB_NAME={{user `database_name`}}"
    ]
  }, {
    "type": "shell",
    "script": "configure_apache"
  }]

The provisioners are executed in the order we provide them here. So first all scripts from our scripts folder will be uploaded to the /tmp folder. Then we will execute the prepare_data_folder so we can upload our website scripts to the /data folder. Then we will execute the setup_database script which we will provide an environment variable which can be used in the script. Notice we are using one of our user variables defined before. And last but not least we are executing the configure_apache script.

So let me first show you a simple script to create the data folder.

#!/bin/sh
 
mkdir -p /data
chown www-data:www-data /data
chmod 755 /data

This script will create the /data folder and make apache the owner of the folder. We also made directory writeable. Feel free to modify the script to your own needs. The file provisoner will now upload the files for our webpage folder to this /data folder. Then we could execute a script to setup our database.

#!/bin/sh
 
# if database name not is empty
if [[ ! -z "$DB_NAME" -a "$DB_NAME" != " " ]]; then
    MYSQLPASS=r00tmysql #as defined in the debian preseed file
    mysqladmin create $DB_NAME -p$MYSQLPASS
    mysql -u root -p$MYSQLPASS -e "GRANT ALL ON $DB_NAME.* TO $DB_NAME@localhost IDENTIFIED BY '$DB_NAME'"
    mysql -u $DB_NAME -p$DB_NAME -e "CREATE TABLE user (id INT NOT NULL AUTO_INCREMENT PRIMARY KEY, username VARCHAR(50), email VARCHAR(50), password BINARY(64)) TYPE=innodb"
    # mysql -u $DB_NAME -p$DB_NAME $DB_NAME < /data/sql/mydatabase_creationscript.sql
fi

This example script will only create a database when you provide a name for the database. The database will have a username and password with the same name as the database. In the example we create a table user in this newly created database. You could also use a file containing sql statements to create your database (see the last hashed line). When you use this *.sql file don't forgot to upload it using the file provisioner. You could for example put this file in your webpage folder which will be uploaded to the /data folder on your vm.

Last but not least we would like to configure Apache in this example to setup a virtualhost to host the website in our data folder.

#!/bin/sh
 
echo "" >> /apache2/sites-available/mywebpage
echo "  ServerName mywebpage.vm" >> /apache2/sites-available/mywebpage
echo "  DocumentRoot /data" >> /apache2/sites-available/mywebpage
echo "" >> /apache2/sites-available/mywebpage
 
# cp /tmp/sites-available/mywebapge /etc/apache2/sites-available
 
/etc/init.d/apache2 restart

In the example above I create the most simple virtual host as possible and then restart Apache to load this virtual host configuration. You can reach this webpage from mywebpage.vm. Therefore you need to make sure you have this value in your hosts file (10.0.2.15 mywebpage.vm). Do not forget to configure portforwarding for your network adapter in VirtualBox, since it is an NAT adapter, otherwise you won't be able to access your VM from your host. You could choose for copy pasting a file which was uploaded using the file provisioner, or even better let the file provisioner upload it directly in the correct directory.

Considering we will be on Windows we can execute following from our command line to build our VM.

SET DB_NAME=MyAwesomeDB
packer build \
    -var 'cpus=1' \
    -var 'memory=512' \
    packer-debian-x64-webserver.json

The result is a *.ova file which we can import in VirtualBox. As soon as you boot the VM you will notice all steps done by the provisoning are there. Now it is up to you to make your provisioning work for your own project. Things you could do is things like setting up your git repository, configure the git user, upload your ssh key, clone the repository etc. All this can be done with everything you have learned in this and previous blog post. The most cool thing is you have a complete VM available for all your developers which they can get up and running with their own username, password and ssh keys provided via variables and the file provisioner. This all would require just a few kilobytes of text files containing your template and provisioner files instead of having Gigabytes of VM's which still have to be configured by your devs to be personalized. You could even store your packer script in Git to have version control.

I hope you enjoyed this blog series of two posts, so it gave you enough inspiration to create your own packer scripts. Don't forget to share them on Github so we all can benefit from your awesome work. Feel free to share your own creations here in the comments. You can fork my full example on Github.