02
Sat, Nov
2 New Articles

Cloudify Your Applications by Creating a Software Virtual Appliance

High Availability / Disaster Recovery
Typography
  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times

Package your existing applications for automatic deployment into clouds.

 

The vast majority of IT professionals today understand that cloud computing constitutes a fundamental paradigm shift in how the software solutions are marketed, distributed, and deployed. I contend that there are two fundamental features of cloud computing that will guarantee its success in the marketplace:

 

  1. Self-service
  2. Automation

 

The concept of self-service empowers the end-users. They can browse a catalog of prepackaged software solutions and decide when to deploy the workloads that satisfy their requirements. In this scenario, the data center provides the required services much in the same way as your public utilities company delivers electricity to your home. The data center administrators are no more the gatekeepers; they become transparent to the end-users.

 

With a high level of automation, the companies are able to lower their data center operational expenses (OPEX) by orders of magnitude. To achieve these high levels of automation, however, the IT shops need to adopt a new deployment model. In the traditional model, a solution provider such as an ISV would send a specialist to a customer location to install or upgrade their solution. Often, it is a cumbersome, time-consuming, tedious process that needs to be repeated over and over again at each customer location. Cloud computing simplifies this problem by implementing a solution known as a software virtual appliance (software VA, for short).

 

A software virtual appliance is a self-contained and self-described entity that consists of one or more virtual machine images and a metadata descriptor that describes the content of the virtual appliance and the required features of the target deployment platform.

 

IBM—along with many other vendors in the virtualization space—have adopted the Distributed Management Task Force (DMTF) Open Virtualization Format (OVF) specification as a standard way of building and deploying software virtual appliances.

Why Implement Software Virtual Appliances?

The software virtual appliances can encourage a quicker transition to the cloud deployment model by dramatically lowering the initial requirements for in-depth virtualization skills and addressing the key issues that have previously hindered adoption of the virtualized solutions:

 

  • Software Licensing—The software license agreement can be included in the software virtual appliance so that the deployer must accept its terms and conditions before the appliance can be successfully activated.  
  • A standard way to accompany the executable image with contextual information—The metadata descriptor, called the OVF file, contains detailed information about the target system Hardware Abstraction Layer (HAL) requirements, such as how many virtual CPUs are needed, what the networking setup is, etc. In addition, the OVF file can contain precise descriptions of the software components that constitute the appliance. For example, there might be separate sections describing the attributes of the operating system, the middleware, and the end-user applications. 
  • Support for software topologies that require multiple virtual machines—The realistic multi-tier software architectures can be properly represented with each tier being mapped to a separate virtual machine image. Note that in the DMTF specification a virtual machine is called a virtual system.
  • Vendor and platform independence—The appliances can be ported across management stacks, making a migration among virtualization vendors and among data centers fairly easy and non-disruptive.
  • Industry standard content verification and integrity checking support—The appliance payloads are digitally signed, ensuring that the content was not tempered with or modified.

The Anatomy of a Software Virtual Appliance

As mentioned, a software virtual appliance can consist of one or many virtual systems (VSes). Each VS can contain one or many virtual disks. The relationships between components as well as their attributes are described in the OVF file. All of the components, including binary images of the virtual disks along with the corresponding OVF file, are packaged in a tar archive. This tar archive becomes a software VA. In its most basic form, a software VA consists of a single virtual server that contains just one virtual disk image and a corresponding OVF.

 

Figure 1 illustrates the contents of a software VA targeting KVM as the deployment hypervisor.

 

061311JarekFigure1V2

Figure 1: These are the contents of a sample software virtual appliance. (Click images to enlarge.)

 

The software VA archive shown in Figure 1 consists of several components. Let's discuss the most important ones:

 

  • OVF File—This is an XML document containing the metadata describing the content of the software VA. It must be present as the first file in the root directory of the archive.
  • Manifest File—This is used to digitally sign the software VA. It guarantees the integrity and the security of the content. The cloud deployment platform will reject the deployment of an appliance with a compromised digital signature.
  • Virtual Disks—Each virtual system may require one or many virtual disk images. A virtual disk is a binary representation of a disk that is needed by a given virtual system to boot and function properly. For instance, the first disk is a bootable image of the operating system, the second is the data disk, etc. The DMTF OVF standard does not deal with the virtual disk formats. It requires only that the format is published. In the above example, I use the raw disk format, which is supported by the open-source hypervisor called KVM.
  • Supporting Files—In certain cases, some software artifacts need to be loaded at the time when the software VA is booted for the first time. These artifacts can be packaged with the software VA as ISO images or zip archives.

The OVF File

The OVF file is a critical part of every DMTF-compliant software VA. The cloud deployment management system parses its content to retrieve the information that is necessary for a successful activation of the software solution contained in a given appliance. The OVF file is an XML document that contains a number of sections. Each section describes a different aspect of the appliance.

 

Let's examine the most important parts of a typical OVF file:

 

Disk Section

The Disk Section defines all virtual disk images at the global (virtual appliance) level. Here's an XML snippet that shows a typical Disk Section for a simple single-VS, single-disk appliance:

 

<ovf:References>

       <ovf:File ovf:href="vs0/disk0.raw" ovf:id="disk0"

ovf:size="10737418240" />

</ovf:References>

<ovf:DiskSection>

       <ovf:Info>Describes the set of virtual disks</ovf:Info>

       <ovf:Disk ovf:capacity="10737418240"                                    [1]

ovf:capacityAllocationUnits="byte"

ovf:diskId="vs0/vd0"                                     

ovf:fileRef="disk0"                                              [2]

              ovf:format="http://www.ibm.com/xmlns/ovf/diskformat/qemu.raw"  [3]  

              ovf:populatedSize="10737418240" />

</ovf:DiskSection>

 

In the sample above, at [1] the virtual disk capacity is specified, and at [2] a file reference to the disk image location is defined. The deployment platform uses this info to locate the file within the appliance tar archive that contains the binary image of a given virtual disk. At [3] a link to the disk format specification is provided.

 

Network Section

The Network Section provides all network definitions at the global level, listing the required private and public networks used in the software VA.

 

Virtual System Collection

The Virtual System Collection contains one or many virtual system definitions. This concept allows the deployment platform to manage the collection of the virtual systems as a unit of deployment. A successfully deployed Virtual System Collection becomes a workload that can be started, stopped, and migrated as a unit. 

    

Virtual System

The virtual system defines the metadata for each of the virtual machines that make up the appliance.  The virtual system section contains the following sections:

  • Eula Section—This is the license agreement for the given virtual server.
  • Product Section—This defines application, service, and static info that needs to be configured at the deployment of the appliance. For example, the local host name needs to be changed in the DB2 server configuration. 
  • Hardware Section—This defines the virtual hardware to be used per virtual server. This needs to be in the scope of the Hardware Abstraction Layer (HAL) of the hypervisor chosen for deployment. Typically, this section describes the following aspects of HAL: network (virtual or physical),  capacity of virtual servers that will boot from images (memory, CPU),  storage controller, and devices.

The Magic of the Virtual Appliance Self-Activation

The true value of the OVF-compliant software VAs lies in the ability to automatically install and configure all of the software components, including the operating system, middleware, and end-user applications. Unfortunately, the prevailing approach of many appliance architects is to ship them "half-baked" without automating the activation of the entire software stack. Often, appliance architects use, say, the VMWare tools to ensure the reconfiguration of the OS-level settings, such as host name, domain name, and root password. They tend not to address the reconfiguration needs of the software components above the OS at the initial deploy time. For example, they do not reconfigure the database settings. Consequently, at deployment time, the host name at the OS level gets set to the proper value, while the database configuration shows the original host name. This may lead to unpredictable results and often requires manual debug and repair. This approach is problematic for several reasons:

 

  • The need for a manual configuration of deployed workloads defeats the key value proposition of cloud computing—namely, automation.
  • The deployer (a cloud admin) may not have the necessary domain knowledge to properly reconfigure a third-party application or middleware or the inter-dependencies of a multi-tiered application construct.

 

A preferred solution to address these issues is to take advantage of the IBM Virtual Solutions Activation Engine (VSAE). The Activation Engine is a software tool that is used during the appliance creation and deployment phases. The appliance architect installs the Activation Engine in each virtual system image during the appliance construction phase. Then, when the appliance is deployed, the Activation Engine runs in each of the virtual systems and reconfigures the system and the installed applications.

 

Essentially, the VSAE is a scripting engine that starts on the first boot before the application services are activated. For example, the WebSphere Application Server (WAS) data source configuration requires the DB2 server host name, the port on which DB2 is listening, the database name, and DB2 user credentials. Please note that these settings may be different for each instance and are runtime environment–specific. They need to be set before the WAS application startup to prevent conflicts and security exposure.

 

The VSAE supports a pluggable architecture where the configuration tasks for a specific aspect of a software stack are performed by the specialized activation programs. These activation programs are invoked by the Activation Engine in a predefined sequence. 

 

The OVF standard recommends that the runtime parameters are passed to the Activation Engine using an XML configuration file (default name is ovf-env.xml) contained in a virtual CD drive. The virtual CD drive is attached to a given virtual system at the boot time. The Activation Engine parses the XML configuration file to retrieve the parameters and then calls a specific activation program that sets points of variability (change points) in the application stack.

 

These concepts are illustrated in Figure 2:

 

061311JarekFigure2V2

Figure 2: These are the Activation Engine components.

 

Let's analyze the activation flow in greater detail. For example, an activation program that reconfigures a DB2 instance requires a number of runtime-specific parameters that need to be passed by the deployment platform using the ovf-env.xml transport mechanism. Here's an excerpt from the ovf-env.xml file that illustrates how the DB2 runtime parameters are passed to the Activation Engine:

 

<PropertySection>

<Property ovfenv:key="com.ibm.vsae.2_1.db2-config.db2inst1_password"

                     ovfenv:value="" />

<Property ovfenv:key="com.ibm.vsae.2_1.db2-config.db2fenc1_password"

                     ovfenv:value="" />

<Property ovfenv:key="com.ibm.vsae.2_1.db2-config.dasusr1_password"

                     ovfenv:value="" />

<Property ovfenv:key="com.ibm.vsae.2_1.db2-config.db2inst1_username"

                     ovfenv:value="db2inst1" />

<Property ovfenv:key="com.ibm.vsae.2_1.db2-config.db2inst1_group"

                     ovfenv:value="db2iadm1" />

<Property ovfenv:key="com.ibm.vsae.2_1.db2-config.db_name_new"

                     ovfenv:value="SAMPLE" />

<Property ovfenv:key="com.ibm.vsae.2_1.db2-config.db_action"

                     ovfenv:value="do-nothing" />

<Property ovfenv:key="com.ibm.vsae.2_1.db2-config.db2_license_type"

                     ovfenv:value="paid" />

<Property ovfenv:key="com.ibm.vsae.2_1.db2-config.db2_hostname"            [1]

                     ovfenv:value="vs102" />

/PropertySection>

 

In the example above, at [1] the db2_hostname parameter is passed with the value of "vs102". Note that I use the fully qualified parameter name that will allow the activation engine to locate the proper activation program to which to pass the hostname parameter and thus avoid the name conflicts for the same-named parameters used by different activation programs. For instance, the WAS activation may also require a db2_hostname parameter to properly reset the server name in the data source object.

 

There are two methods for setting the parameters:

 

1. At the appliance construction time—The appliance architect can use the OVF Product Section for DB2 to preset the parameters to their default values. Consider the following excerpt from the corresponding OVF file:

 

<!-- db2_hostname = localhost -->

<ovf:Property ovf:key="db2_hostname" ovf:type="string"

       ovf:userConfigurable="true" ovf:value="localhost" password="false">   [1]

              <ovf:Label ovf:msgid="ConfigDB2.db2_hostname.label"></ovf:Label>

              <ovf:Description>Local Hostname</ovf:Description>

</ovf:Property>

 

 In the snippet above, at [1] the appliance architect sets the db2_hostname to a default value of "localhost". He or she also indicates that the value is user-configurable, meaning that it can be changed by the deployer.

 

This leads to the second method for parameter modification:

 

2. At the deployment time—The deployment platform identifies all the parameters that have the userConfigurable attribute set to true. The parameters will be presented to the deployer at one of the deployment steps, at which time they can be changed to the values that reflect the current runtime environment. For example, the db2_hostname gets set to "vs102". Note that the deployer can be a human being or the cloud management software, such as the appliance placement services.

 

The actual code within the activation program responsible for resetting the host name may look as shown below:

 

echo "Update the DB2 hostname to $db2_hostname"

set -x

$DB2_PATH/adm/db2set -g db2system=$db2_hostname

set -

echo

 

Following the methodology described in this section, a number of software VAs were built for IBM partners. These software VAs fully automated the deployment of the real-life, sometimes-quite-complex solutions.

Building Software Virtual Appliances

By now you should be excited about the tremendous possibilities offered by the software Virtual Appliance deployment model. You may ask yourself, "How do I package my applications into this format so that they become cloud-ready?" Well, there are several options:

 

1. Build the software VA manually using the open-source tooling. For example, you could install on your Linux workstation the open-source hypervisor KVM, along with the virtualization tools, such as virsh, virt-manager, and virt-viewer. You would then use the virt-manager to create the virtual machine images, boot those images on KVM, and install all the necessary software components. A simple text editor could then be used to create or modify an OVF file to describe the content of the virtual machine images you created. The final step would be to create a tar archive that contains the images and the OVF. This is, of course, easier said than done. The manual VA creation requires significant virtualization and OVF skills and probably is appealing only to geeks. So let's explore other options.

 

2. Use VMWare tooling. VMWare Studio and the OVF Toolkit are aimed at simplifying the process of virtual appliance creation. The virtual appliances created in VMWare Studio can be easily imported and deployed using vSphere Client. The process is well-documented in the VMWare publications. Please keep in mind that VMWare tools do not support open-source hypervisors such as KVM or architectures other than Intel.

 

3. Leverage Virtual Appliance Factory Service (VAF Service). With this no-fee service, IBM Business Partners and ISVs get assistance and resources to build virtual images and software virtual appliances. The VAF Service is a set of Web 2.0 tools, services, and processes that dramatically simplify and automate the process of ISV solutions "cloudification." Currently, this service is hosted by the IBM Innovation Center in Waltham, Massachusetts. Some of the benefits of the VAF Service include these:

 

  • Allows entry into cloud space with minimal up-front investment
  • Hides complexity of the virtualization infrastructure, meaning that partners can focus on their knowledge domain, which is multi-tier application composition rather than infrastructure setup
  • Creates a virtual appliance deployable into DMTF OVF–compatible data centers
  • Delivers on two key value propositions of cloud computing: automation and self-service (industry-strength IBM VSAE can be bundled with the VAs)

 

Please contact the author for further details on the VAF Service.

Closing Comments

The software virtual appliance model governs the packaging and deploying of software solutions into cloud environments. It does not have dependencies on any specific cloud architecture or cloud services. I believe that the software VA standard as codified by the DMTF OVF spec has been the best option for easing the pains of moving from the traditional deployment model to the cloud deployment model. It speeds up the process of "cloudifying" a range of applications. On the one end of the spectrum, you can take an existing matured, hardened, and market-tested COBOL app and with little effort package it as a single-image, single-disk software VA, which immediately becomes deployable into the cloud. On the other end of the spectrum, you can package a newly implemented, highly distributed, service-oriented application. In this case, the software VA may consist of multiple virtual systems, each requiring multiple disks and sophisticated activation logic. This newly created software may, in fact, integrate with the services provided by the cloud, such as database services or queuing services. Please note, however, that this integration with the cloud services is neither required nor governed by the DMTF OVF spec. Which underlying services to use is the solution architect's choice much in the same way as she or he may choose to use J2EE rather than .NET. This has nothing to do with how the solution is packaged.

 

I hope this helps clarify the scope and the benefits of the software VA deployment model.

 

Additional Material

The DMTF OVF specification Web site:

 

http://www.dmtf.org/standards/ovf

 

The IBM Web site for Partners who wish to participate in the Cloud Specialty program:

 

https://www-304.ibm.com/partnerworld/wps/servlet/ContentHandler/isv_com_spe_cloud_index

 

as/400, os/400, iseries, system i, i5/os, ibm i, power systems, 6.1, 7.1, V7, V6R1

Jarek Miszczyk

Jarek Miszczyk is a Lead Technical Consultant for System x Virtualization and Cloud Computing at the IBM STG Global ISV Enablement organization. He is located in Rochester, Minnesota. He can be reached by email at This email address is being protected from spambots. You need JavaScript enabled to view it..

BLOG COMMENTS POWERED BY DISQUS

LATEST COMMENTS

Support MC Press Online

$

Book Reviews

Resource Center

  • SB Profound WC 5536 Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. You can find Part 1 here. In Part 2 of our free Node.js Webinar Series, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Brian will briefly discuss the different tools available, and demonstrate his preferred setup for Node development on IBM i or any platform. Attend this webinar to learn:

  • SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.

  • SB HelpSystems ROBOT Generic IBM announced the E1080 servers using the latest Power10 processor in September 2021. The most powerful processor from IBM to date, Power10 is designed to handle the demands of doing business in today’s high-tech atmosphere, including running cloud applications, supporting big data, and managing AI workloads. But what does Power10 mean for your data center? In this recorded webinar, IBMers Dan Sundt and Dylan Boday join IBM Power Champion Tom Huntington for a discussion on why Power10 technology is the right strategic investment if you run IBM i, AIX, or Linux. In this action-packed hour, Tom will share trends from the IBM i and AIX user communities while Dan and Dylan dive into the tech specs for key hardware, including:

  • Magic MarkTRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now!  Request Now.

  • SB HelpSystems ROBOT GenericForms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.

  • SB HelpSystems ROBOT GenericIT security is a top priority for businesses around the world, but most IBM i pros don’t know where to begin—and most cybersecurity experts don’t know IBM i. In this session, Robin Tatam explores the business impact of lax IBM i security, the top vulnerabilities putting IBM i at risk, and the steps you can take to protect your organization. If you’re looking to avoid unexpected downtime or corrupted data, you don’t want to miss this session.

  • SB HelpSystems ROBOT GenericCan you trust all of your users all of the time? A typical end user receives 16 malicious emails each month, but only 17 percent of these phishing campaigns are reported to IT. Once an attack is underway, most organizations won’t discover the breach until six months later. A staggering amount of damage can occur in that time. Despite these risks, 93 percent of organizations are leaving their IBM i systems vulnerable to cybercrime. In this on-demand webinar, IBM i security experts Robin Tatam and Sandi Moore will reveal:

  • FORTRA Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:

  • FORTRAManaging messages on your IBM i can be more than a full-time job if you have to do it manually. Messages need a response and resources must be monitored—often over multiple systems and across platforms. How can you be sure you won’t miss important system events? Automate your message center with the Robot Message Management Solution. Key features include:

  • FORTRAThe thought of printing, distributing, and storing iSeries reports manually may reduce you to tears. Paper and labor costs associated with report generation can spiral out of control. Mountains of paper threaten to swamp your files. Robot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing. Manage your reports with the Robot Report Management Solution. Key features include:

  • FORTRAFor over 30 years, Robot has been a leader in systems management for IBM i. With batch job creation and scheduling at its core, the Robot Job Scheduling Solution reduces the opportunity for human error and helps you maintain service levels, automating even the biggest, most complex runbooks. Manage your job schedule with the Robot Job Scheduling Solution. Key features include:

  • LANSA Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.

  • LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed. Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

  • LANSASupply Chain is becoming increasingly complex and unpredictable. From raw materials for manufacturing to food supply chains, the journey from source to production to delivery to consumers is marred with inefficiencies, manual processes, shortages, recalls, counterfeits, and scandals. In this webinar, we discuss how:

  • The MC Resource Centers bring you the widest selection of white papers, trial software, and on-demand webcasts for you to choose from. >> Review the list of White Papers, Trial Software or On-Demand Webcast at the MC Press Resource Center. >> Add the items to yru Cart and complet he checkout process and submit

  • Profound Logic Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.

  • SB Profound WC 5536Join us for this hour-long webcast that will explore:

  • Fortra IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn: