Packer.io machine building and provisioning part 1

Large development teams are often coping with the "It works on my machine" syndrome. One solution to solve these kind of issues is by give each single developer the same VM, which is most preferably the same as your production server. So imagine your company is building a web application. The web application is hosted on a Debian server using Apache, MySQL and PHP. So considering these preconditions I will give you a simple example to get your machines scripted and fully provisioned. In this first part of these series we will zoom in on the packer.io builders.

So let me first explain you what packer is by quoting some statements of their webpage.

Packer is a tool for creating identical machine images for multiple platforms from a single source configuration

Modern, Automated

Packer is easy to use and automates the creation of any type of machine image. It embraces modern configuration management by encouraging you to use automated scripts to install and configure the software within your Packer-made images. Packer brings machine images into the modern age, unlocking untapped potential and opening new opportunities.

Works Great With

Out of the box Packer comes with support to build images for Amazon EC2, DigitalOcean, VirtualBox, and VMware. Support for more platforms is on the way, and anyone can add new platforms via plugins.

In order to create an image for a specific platform packer uses builders. Since I don't want to zoom in on creating your own builder plugin for your own platform and I don't want you to have more cost to get up your VM I will use the VirtualBox builder in this example. VirtualBox is free to use, so if you don't have it already installed on your machine please first install VirtualBox to continue with this example. VirtualBox will run on following OSes: Windows, Linux and Mac, so no matter what OS you're on, you can continue reading.

Please download the proper package for your operating system and architecture. Packer has packages for MacOS, Linux, Windows, FreeBSD and OpenBSD. Once you have installed packer we can start with creating our packer script. First create a folder on your machine called "my-first-packer-machine". In this folder we create a file "packer-debian-x64-webserver.json". This JSON file is used to define our packer script to build and provision our VM. Let's start with defining our VirtualBox builder.

{
  "builders": [
    {
      "name": "debian64-webserver-vbox",
      "type": "virtualbox-iso",
      "guest_os_type": "Debian_64",
      "guest_additions_mode": "disable",
      "iso_url": "http://cdimage.debian.org/debian-cd/7.5.0/amd64/iso-cd/debian-7.5.0-amd64-netinst.iso",
      "iso_checksum": "0a86e2fdef365f8bd626de7e754189291174b86a312c655fa6585eec3c6e5caa5e8f128e37a6989ae8d45743e158191d1b178d76bf338cb4512f0ee4aa9cda3c",
      "iso_checksum_type": "sha512",
      "ssh_username": "root",
      "ssh_password": "r00tme",
      "ssh_wait_timeout": "30m",
      "shutdown_command": "echo 'packer' | shutdown -h now",
      "vm_name": "my-webserver",
      "format": "ova",
      "hard_drive_interface": "sata",
      "disk_size": 20000,
      "vboxmanage": [
        ["modifyvm", "{{.Name}}", "--memory", "1024"],
        ["modifyvm", "{{.Name}}", "--cpus", "1"],
        ["modifyvm", "{{.Name}}", "--vram", "10"]
      ],
      "boot_wait": "5s",
      "boot_command": [
        "auto url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/debian64.cfg "
      ],
      "http_directory": "http"
    }
  ]
}

As you can see we use the latest stable Debian iso ("Wheezy") to install on our VM. The checksum is required, so packer can check for its integrity as soon it is downloaded. We configure our VirtualBox image to have a 20GB SATA harddisk (Don't worry it will be a dynamic disk and will dynamically grow so it won't consume 20GB immediately). We configure the VM to have 1 GB of memory, 1CPU and 10MB of Video memory (minimum requirement for VirtualBox Seamless mode). The most important is the boot_command, since this is how we make our Debian installation unattended. You can grab an example Debian preseed file here. We will create the file debian64-webserver.cfg in a sub folder "http", in our "my-first-packer-machine" folder, which packer will serve via HTTP during build. Some important options to configure are the following. Make sure you have a long ssh_wait_timeout because packer will abort the build after this timeout.

In case you want to use another OS you can use vboxmanage from your command line to figure out which guest OS types are supported.

VBoxManage list ostypes

### Localization
d-i debian-installer/locale string en_US
d-i debian-installer/language string en
d-i debian-installer/country string US
d-i debian-installer/locale string en_US.UTF-8
d-i localechooser/supported-locales multiselect en_US.UTF-8
# Keyboard selection.
d-i console-tools/archs select at
d-i console-keymaps-at/keymap select us
d-i keyboard-configuration/xkb-keymap select us
 
### Network configuration
d-i netcfg/choose_interface select auto
d-i netcfg/get_hostname string debian-wheezy
d-i netcfg/hostname string debian-wheezy
d-i netcfg/wireless_wep string
 
d-i hw-detect/load_firmware boolean true
 
### Mirror settings
d-i mirror/country string manual
d-i mirror/http/hostname string http.debian.net
d-i mirror/http/directory string /debian
d-i mirror/http/proxy string
d-i mirror/suite string wheezy
 
### Account setup
### Root
d-i passwd/root-password password r00tme
d-i passwd/root-password-again password r00tme
d-i passwd/make-user boolean false
 
### Clock and time zone setup
d-i clock-setup/utc boolean true
d-i time/zone string GMT+1
d-i clock-setup/ntp boolean true
 
### Partitioning
d-i partman-auto/disk string /dev/sda
d-i partman-auto/method string regular
d-i partman-auto/choose_recipe select atomic
 
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true
d-i partman/mount_style select uuid
 
### Apt setup
d-i apt-setup/non-free boolean true
d-i apt-setup/contrib boolean true
d-i apt-setup/services-select multiselect security, volatile
d-i apt-setup/security_host string security.debian.org
d-i apt-setup/volatile_host string volatile.debian.org
 
### Package selection
tasksel tasksel/first multiselect
 
d-i pkgsel/include string openssh-server build-essential nfs-common ssh ca-certificates curl linux-headers-amd64 apache2 php5 php5-mysql mysql-server-5.5 mysql-client samba vim
 
mysql-server-5.5 mysql-server/root_password password r00tmysql
mysql-server-5.5 mysql-server/root_password_again password r00tmysql
 
d-i pkgsel/upgrade select safe-upgrade
popularity-contest popularity-contest/participate boolean false
 
### GRUB
d-i grub-installer/only_debian boolean true
 
### Finishing up the installation
d-i finish-install/keep-consoles boolean true
d-i finish-install/reboot_in_progress note

As you can see we make sure the Debian installation is in the right timezone and has the right packages installed. I also choose to install samba so we can have a shared folder which can be accessed by our host and vim to have a decent text editor to change configuration files like php.ini etc. Feel free to add any php packages you need (php5-memcache etc.) to the di pkgsel/include string line.

Now we are good to test our packer.io script. To do so first open a shell, or on windows your command line, and navigate to your my-first-packer-machine folder. In this folder we can execute the following packer commands.

# validates your packer template file
packer validate packer-debian-x64-webserver.json
# take the template and actually run the builds within it,
# producing any resulting machine images.
packer build packer-debian-x64-webserver.json

The first command will validate your JSON file, the latter will run the builds specified in your packer template file. If you have multiple builders specified and only want to execute one you can use the -only=debian64-webserver-vbox command line parameter to only build using the builder with specified name. As you may see the debian.iso will be cached by packer in the packer_cache folder, so it won't have to download it again when you need to rerun your packer build. So please don't throw them away since you will need them for the next part of these series. The result of this build should be a my-webserver.ova file with a size of about 700MB. By double clicking this file you can import this VirtualBox appliance and start using your VM.

Next week we will have a look at packer variables and provisioners, in the next part, to further automate our machine building.