Devops (2): Vagrant

After looking a bit into Docker, I went on to look into Vagrant, another well-known tool to provision and configure virtual machines.



To learn about Vagrant, I read the book "Vagrant: Up and Running". This O'Reilly book is well-written and the examples all work. While doing the exercises on a Windows Server 2012R2 VM, I hardly encountered any problems. Starting the Ubuntu VM with the vagrant up command and removing it with vagrant destory.

The Vagrant command line tool allows to create a Virtual Machine in a reproducible and neutral manner. Vagrant was initially developed around the the free VM product Oracle VirtualBox. But Vagrant now comes with many other providers, e.g. AWS, Rackspace, IBM SoftLayer and Microsoft Azure. Support for VMWare however is not free ($79).

Vagrant focuses on the creation of Virtual Machines in a neutral manner. Contrary to Docker, it uses an actual Virtualization solution to provision the virtual machines. This allows Vagrant to support multiple Operating Systems in parallel. And offers support for automating the creation of Windows based virtual machines.

When Vagrant is used in combination with Oracle VirtualBox, Vagrant will use the VBoxManage.exe of VirtualBox. To create machines with a cloud provider, the respective Vagrant provider will leverage the API and tools of the specific Infrastructure-As-A-Service solution. Vagrant configure all sorts of attributes of the virtual machine, incl. e.g. networking (and port forwarding).

For the actual provisioning of the machines, Vagrant supports many options, including command line. But most often, Vagrant will be used in combination with Chef or Puppet. E.g. the Chef development kit uses Vagrant as its default "driver".

Boxes
Vagrant does not start from an ISO image, but from an already prepared "box". The more such box is pre-configured, the fewer configuration needs to be done afterwards. Vagrant uses its own software format to package the virtual machines that are taken as a starting point (compare to Amazon Machine Images). Vagrantbox.es and many others make pre-packaged Vagrant boxes available.

Windows specific

• VagrantManager makes Vagrant accessible from the Windows (or iOS) Taskbar 
• The company modern.IE makes Windows boxes with all sorts of IE versions available.
• Interesting blog on how to create Variant Windows boxes
• Vagrant can directly access the command line of Linux boxes over SSH (secure shell). For Windows boxes this cal also be arranged wen cygwin (or other SSH server) is installed. But Vagrant can also use WinRM to access the Windows command line
• Where the installation of software on Linux boxes leverages apt-get or yum to install software packages, Chocolatery wants to bring a similar solution to the Microsoft world; many packages are available for quick and easy installation
• Boxstarter leverages Chocolatey packages to automate the installation of software and create repeatable, scripted Windows environments.

Vagrant and Integration Tools
In my own domain of Application Integration and SOA, I expect that both vendors and customers will pickup tools such as Vagrant for creating and provisioning (virtual) machines. Combined with Chef or Puppet to actually install and configure the software on these machines.

Author: Guy

Devops and Docker

The holiday period between Christmas and New Year is an ideal period to catch up on some reading and experimenting. Devops and tools such as Docker, Vagrant, Chef, Puppet and Ansible were on my radar for a while. So finally some time to dive into this topics.


Nested VMs
Not to mess up my machine, I use VMWare workstation to spin up some test machines. As these Devops tools are all about creating and provisioning virtual machines, one must enable "Nested VMs" support. This allows one virtual machine to run in another.

Docker


Docker appeared on my radar while learning about Micro Services. Docker focuses on the creation of light-weight containers in which applications are configured in an automated manner.



The Linux Containers are very small by leveraging OS level virtualization of Linux. Is it some "chroot on rocks". The chroot system call on Unix/Linux changes the root directory for a program and all of its children. chroot allows programs - e.g. a web server - to run in a more protected mode. The OS level virtualization can limit all the resources used by child processes: CPU, memory, disk space, ... Because containers are so light-weight, many of them can be run on a single machine. This mechanism allows each application to run in its own container, its own virtualized OS.


To have a quick try of Docker, there is a great Online Tutorial consisting of 10 steps. Recommended!


As there aren't any books available on Docker, I watched the brand new training material of LiveLessons. As I couldn't find the text material, had to type over the instructions from the paused video. After wasting some time trying to get access the Fedora Atomic container on the Fedora 21 host, decided to switch to another topic, Vagrant. If I have some more time, I'll come back and retry with RHEL as used in the video training. Or switch to Windows and take a look boot2docker.

Author: Guy

Datapower XQuery replace

One of the clients that I’m working for discovered a problem with a SOAP web service querying an LDAP. The service could contain a ‘*’ in plain text in possibly different fields in the message. When the service is called it uses the ‘*’ as a wildcard. The system should handle the ‘*’ as plain text so we need to escape the character with ‘\2a’ (escape for a LDAP filter query). So they looked in complete web service chain where the least impact was. They decided that an update in the DataPower configuration was the best option.

This is a small message example, but the ‘*’ can occur in couple different WSDL operations and in different fields.

<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"
               xmlns:tem="http://tempuri.org/">

   <soap:Header/>

   <soap:Body>

      <tem:FindUser>

        <tem:UserName>KMe_*</tem:UserName>

      </tem:FindUser>

   </soap:Body>

</soap:Envelope>

I immediately thought to use the function str:replace(). But unfortunately it is not supported in Datapower, which brought me to XQuery, as an alternative for XSLT. So this is the solution that I developed.

Because the replacement is only necessary for 3 operations from the WSDL I defined the policy-rule on WSDL operation level.

Below the XQuery code used to replace the ‘*’ into ‘\2’a. The XQuery can be extended to handle other values that need to be escaped for example:  ( ) \ / NUL

xquery version "1.0";

declare namespace local = "http://example.org";

declare function local:copy-replace($element as element()) {

  element {node-name($element)}

               {$element/@*,

                for $child in $element/node()

                return if ($child instance of element())

                       then local:copy-replace($child)

                       else replace($child,'\*','\\2a')

               }

};

local:copy-replace(/*)

The total number of requests that have a ‘*’ or other wildcards in the username is limited. To improve the performance I adapted the standard SQL-injection filter to search for ‘*’ and output the number of hits. This way when the hit count is 0 I can skip the XQuery transform action.

Author: Kim