Deploy VMware virtual machines with Ansible

Posted by

Why would I deploy virtual machines with Ansible

[Update]I wrote another article about VM automation, for Azure this time. Check it out![/Update]

[French readers]La version française de l’article « Déployer des machines virtuelles VMware avec Ansible) est disponible à l’adresse suivante[/French readers]

Since a few month, I automate all that can be automated with Ansible (FR). Still, there was something that I still had to do manually and it bothered me (a lot).

Until then, I used Ansible both as a deployment tools (to install things on my machines) and also as a « Configuration Management » tool.

Let me explain :

  • From a virtual machine template, I can deploy default configuration (depending of the machine role). I mean by configuration, the first sense of the term like configuration files, but also package installations.
  • If I need to modify something on a subset of (or all!) my servers, I can push it on them with one command line.
  • Should something/someone modify something on the server managed configuration, I can correct this with my Ansible magic wand by just playing all my playbooks on all the servers once in a while. All « non compliant » servers will be automatically corrected WITHOUT TOUCHING those that ARE compliant.

For the last part, it’s not really « magic ». Playbooks have to be written while keeping I mind that they must be « idempotent ». This is really important and that’s why I had to develop my own Ansible module for Centreon.

Still, I was frustrated because one last step was still missing: Provisionning VMs directly from Ansible!

What is the problem?

2 obstacles prevented me from doing it.

Gather facts

First, Ansible works with an inventory. If you try to tell Ansible that you want to create new machines and you add them to the inventory, the first thing Ansible will try to do is to connect to them… before they exists!

ansible-playbook -l 192.168.100.103 zwindler_deploy.yml

PLAY [xxxxxxaaaaaaaaaaa] ********************************************************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************************************************************
fatal: [192.168.100.103]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: connect to host 192.168.100.103 port 22: No route to host\r\n", "unreachable": true}

PLAY RECAP **********************************************************************************************************************************************************************************
192.168.100.103 : ok=0 changed=0 unreachable=1 failed=0

We need to tell Ansible not to connect, at least at first.

You can do that by using the option gather_facts: false, which, you guessed, tells Ansible not to gather facts…

But that’s still not enough. If you try to run the playbook with just this option, Ansible will still try to connect to the yet to be created server, failing.

Most of the time, people use the delegate_to option, which tells Ansible to connect to another server for one specific task. I had used it previously to feed informations in Centreon with CLAPI.

Here, if you need to run several tasks and none of them have to be executed on the remote machine (that we are trying to deploy), you can just specify at the head of the playbook that all tasks are to be executed from the local machine, with connection: local :

  gather_facts: false
  connection: local

To sum this up, with these 2 options, you can tell Ansible not to « gather facts » as a preliminary step of every playbook, and to execute everything locally, thus avoiding the failing connection to the yet inexistant servers.

Problem solved.

vsphere_guest versus vmware_guest

My second issue was that 2 groups of modules were available in Ansible core, and one of them was abandonned. Of course, I was using the oldest abandonned one, because it’s the one that posts up in all the tutorials/examples I could find on Internet ;-) .

You can find it here vsphere_guest. It « works », and it has been integrated to the core way sooner (v 1.6) and needs the python dependancy pysphere (not maintained since 2013)!

There is no reason « not » to use vsphere_guest to provision your virtual machine. Latest version of this module works for vSphere v5 and should work for 6.X. A good example of what can be done with this module can be found on this nice blog-post from EverythingShouldBeVirtual.

Yet, you won’t be able to:

  • Modify the virtual machine (like vCPU, …) if you generate it from a vm template
  • Customize it on the OS level (change hostname, IP, …)

I massively use templates so this was clearly an issue. My typical usecase is this : I deploy a virtual machine from a given template (depending which type of service I want) and then finalize customisation through Ansible playbooks.

In this case, I would have to deploy my virtual machine, and still have to go into them to modify them (like hostname and IP) before playing my playbooks to customise them.

Hopefully, people at VMware didn’t stop at VIPerl and PowerCLI. It’s been a few years than there is a Python module, maintained by VMware, and regularly updated to take advantage of the new API functionnalities: pyvmomi.

From there, a new Ansible module, taking advantage of this python API, you have much more possibilities.

Let there be light

The list of Ansible modules taking advantage from pyvmomi is quite impressive. You can find them, they all start with vmware_ (you can find the list on the official Ansible documentation, VMware section).

Two of them particularly got my attention (but there are a lot more!):

  • vmware_guest can help manage virtual machines
  • vmware_vm_shell can execute commands directly inside the guest OS, provided it executes VMware Tools

Prerequisites

To execute this modules, you need:

  • pyvmomi and its dependancies (python 2.6+ and sphinx)
pip install pyvmomi sphinx
  • Ansible 2.2 minimum (but you’d better have 2.3 or even 2.4!)

On paper, the module « works » with 2.2, the actual version on stable depots. But most interesting functions appear in 2.3 so for now you have to compile sources!

The actual version on RHEL/CentOS 7 is 2.3.X. You don’t need to compile sources anymore, UNLESS you need the very latest additions which can be found in 2.4. I give you the necessary steps at the end of the article.

And now, the playbook !

The following playbook will:

  • ask you for a priviledged account for vCenter
  • deploy a virtual machine on the vCenter from a template

This deployment needs some variables that can be given directly from the Ansible host inventory like:

  • virtual disk size
  • vCPU number
  • vRAM quantity
  • target ESXi and target datasotre
  • VM IP address and hostname

The virtual machine will be deployed as expected, except for hostname and IP. Those 2 parameters will be modified after first boot by VMware Tool.

cat zwindler_vmware_add_normal.yml
---
- hosts: all
  gather_facts: false
  connection: local

  vars_prompt:
    - name: "vsphere_password"
      prompt: "vSphere Password"
    - name: "notes"
      prompt: "VM notes"
      private: no
      default: "Deployed with ansible"

  tasks:
  # get date
  - set_fact: creationdate="{{lookup('pipe','date "+%Y/%m/%d %H:%M"')}}"

  # Create a VM from a template
  - name: create the VM
    vmware_guest:
      hostname: '{{ vsphere_host }}'
      username: '{{ vsphere_user }}'
      password: '{{ vsphere_password }}'
      validate_certs: no
      esxi_hostname: esxi_server
      datacenter: 'ZWINDLER'
      folder: A_DEPLOYER
      name: '{{ inventory_hostname }}'
      state: poweredon
      guest_id: rhel7_64Guest
      annotation: "{{ notes }} - {{ creationdate }}"
      disk:
      - size_gb: 150
        type: thin
        datastore: '{{ vsphere_datastore }}'
      networks:
      - name: server_network
        ip: '{{ custom_ip }}'
        netmask: 255.255.252.0
        gateway: 192.168.100.1
        dns_servers:
        - 192.168.100.10
        - 192.168.101.10
      hardware:
        memory_mb: 4096
        num_cpus: 2
      customization:
        dns_servers:
        - 192.168.100.10
        - 192.168.101.10
        domain : zwindler.fr
        hostname: '{{ inventory_hostname }}'
      template: tmpl-rhel-7-3-app
      wait_for_ip_address: yes

  - name: add to ansible hosts file
    lineinfile:
      dest: /etc/ansible/hosts
      insertafter: '^\[{{ ansible_host_group }}\]'
      line: '{{ item }}'
    with_items: '{{play_hosts}}'
    run_once: true

From there… well, you can do anything that pops in your mind!

Useful links

« pause » module might be useful in some cases when you really have to wait some time. I don’t use it and I wouldn’t even consider it for production.

Also, I might write the same article with the Proxmox VE module. Stay tuned!

New functionnalities

Until 2.3 2.4 becomes available in your OS repository, you might encounter this error if you use functions that aren’t included (yet) in your own version of Ansible (linked_clone or snapshot_src released in 2.4 for example) :

ansible-playbook zwindler_vmware_add_micro.yml
vSphere Password:
VM notes [Deployed with ansible]:

PLAY [ansible_node] **********************************************************

TASK [create the VM] ***********************************************************
fatal: [ansible_node]: FAILED! => {"changed": false, "failed": true, "msg": "unsupported parameter for module: linked_clone"}

PLAY RECAP *********************************************************************
ansible_node : ok=0 changed=0 unreachable=0 failed=1

Should you need a « yet to be released » functionnality, you can compile the latest version of Ansible from the sources (2.4 in my example here):

git clone https://github.com/ansible/ansible

cd ansible
make
make install

ansible --version
    ansible 2.4.0

Vous avez aimé cet article ? Partagez-le avec vos amis !   Twitter Facebook Linkedin email

Vous pouvez également soutenir le blog financièrement :

|| Tipeee

15 comments

  1. need help how we could specify if network id DVS. this script works fine in standard switch but fail to connect network label if we using DVS. Please help

  2. If it doesn’t work and you can’t find the option, then this is probably a limitation on the vmware ansible module. If you confirm this is indeed the issue here, don’t hesitate to contact them to report the bug or ask for the missing feature

  3. Very good article..indeed! Kudos!!.
    Question… If I have several guest VMs within an ansible host inventoy group to clone from a template… how do I provide the different IPs to the several guests? That is looping over the members of the host inventory group and each having it’s unique ip address when finished.

  4. Hi Harrison,

    Super interesting question, so much to say I’ll try to stay brief (but no promises ;p)

    There are several ways to achieve this I think. The easiest way, especially if you already have the inventory file like you say, is simply to add a variable just after the hostname.

    In the example given in the article, you’d write something like that in your inventory

    [vms_to_deploy]
    host1 custom_ip=1.1.1.1
    host2 custom_ip=1.1.1.2
    ...
    

    and so on. On each host, {{custom_ip}} value will be overwritten and each server will have it’s own value.

    Aside from using a custom variable (custom_ip here), you could override an existing variable like ansible_host which is indeed used by ansible as the IP to address you’re VM (this can be useful if you want to do some things on this host once it’s provisionned)

    That’s really easy and that’s what I used at the time of this article.
    But you could also don’t use an inventory at all and simply use a table/dict or even use the tools provided but Ansible to « build » your own inventory on the fly with statements like « set_facts » and « with_sequence »

    I’ll try to write a bit more about this in a future article

  5. Can you describe the contents of your inventory file in the example above? From my understanding, the hosts specified in the inventory have the tasks run in the playbook against them. In this case, the vmware_guest module is run. However, doesn’t this module hit the esxi server host IP? I’m a bit confused as to how this is supposed to work….pls elaborate.

  6. You are right when you say that hosts in the inventory file should be the targets of the playbooks. Most of the time anyway because that’s the trick here ;)

    vmware_guest doesn’t run inside ESXi servers. It runs on a machine that can run Ansible and python (so definitely NOT an ESXi server) and can reach a vCenter server. The ansible/python modules then use the API to do the tasks set in the module.

    In the article, I add « connection: local » or « delegate_to » to achieve this (run the tasks on the local machine, not on the ESXi, nor the VMs). So, here, the only host needed in the inventory could have been localhost.

    I then use the inventory as a way to pass variables for the VMs awaiting to be created. Why do that? Because then, when the VMs are indeed deployed, I don’t have to update my inventory and then launch another playbook to install software in the VMs; I can just chain the installation of the software, post vm deployment (I wrote another article about this, but it’s French only for now, sorry).

    Is it more clear for you this way?

  7. The error being received is: « PyVmomi Python module required. Install using
    « pip install PyVmomi »

    However, PyVmomi is installed. pip show pyvmomi reports it as installed. It looks like ansible is not finding it in the installation in /usr/lib/python2.6/site-packages where is gets installed.

    I ran strace -f on it and it is attempting to look for it in:

    stat(« /usr/lib/python2.6/site-packages/ansible/module_utils/vmware/PyVmomi »)

    and doesn’t find it there….it’s like the pyvmomi install is not setup correctly…

  8. That’s a fairly common issue with python modules I think.

    Somethimes, you have multiple versions of python installed, or you installed ansible with your package manager (yum) and pyvmomi with pip (or the other way around).

    I think you can displayed what is installed with « pip freeze ». You can also try to launch a python interpreter and try to import it manually (I just did it)

    # pip freeze | grep pyvmomi
    pyvmomi==6.7.0
    # which python
    /usr/bin/python
    # python --version
    Python 2.7.12
    # python
    Python 2.7.12 (default, Dec  4 2017, 14:50:18)
    [GCC 5.4.0 20160609] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import pyVmomi
    >>>
    
  9. I think the core issue with our environment is the server where you can do software installs using pip, yum, etc (internet connectivity) is separate from the server that runs ansible with the PyVmomi plugin. As such, I had to install PyVmomi using pip on the server with connectivity and then tarred up the /usr/lib/python2.6/site-packages/pyvmomi directory and the other pyvmomi directory with the egg info. I suspect that this is not the correct way of bring installed python modules from one system to another.

    Is there a proper way of doing this so all the « hooks » being done by the pip install are captured?

  10. 2 zwindler

    Thanks a lot, it works (vCenter 6.5 + ESXi 6.0)!
    Except one thing – VM template should be located not in the Content Library.

  11. Very informative however when I run this I’m not able to get the interface to configure. My VM is created from my template just fine but I want to set a static ip. When the vm comes up it is set for dhcp and gets the next address from my router. My template is Centos 7 with open vm tools installed. Any ideas why this isn’t working? esx 6.5 vshpere 6.5

  12. OS is recent and vsphere also. I personally did it successfully on such platform & guest OS.
    If you do have openvm tools and if you did specify the hostname and IP you want like in my example it should work. OS customisation part should run just after first boot (and VM should reboot again just after).

    For some reason, the customisation is not happening and it never happened to me. You might want to increase the verbosity of the playbook but I think you’ll find more informations on the vsphere platform itself, as it’s the component that handles the customisation.

  13. Hi,
    I have used both.
    Short version: It’s simply not the same philosophy AT ALL.
    Long version:
    If you are fluent in powershell and already have scripted everything, you may not want to invest in Ansible, even though the learning curve is really smooth.
    Another thing to consider is the fact that data manipulation and complex conditionals (get the list of VM or resources, then provision this, else provision that) is not always easy with Ansible.
    On the other hand, Ansible is an incredible tool, with a vast amount of module to manage practicaly Anything. But most of all, it allows you to describe what you want (you might have heard about infrastructure as code) rather than script it. You don’t program anything, you don’t handle the errors if infrastructure is in an unusual state (ex. Working on a VM that should but doesn’t exist might crash your script so you have to check it first with powershell). Ansible module does that for you and the result is exactly what you asked.
    Furthermore, you can run it again if you made some modifications. Only the delta will be applied, you don’t have to run a modified version of your first script that can apply the modification while not touching the rest.
    You guessed, even though sometimes it’s harder to describe than to script, I strongly recommend using Ansible (or another configuration management/infra as code tool) rather than scripting. In my opinion, it will be much more future proof.

Leave a Reply

Votre adresse de messagerie ne sera pas publiée. Les champs obligatoires sont indiqués avec *

Ce site utilise Akismet pour réduire les indésirables. En savoir plus sur comment les données de vos commentaires sont utilisées.