f you have been following recent IT industry trends, you may have noticed that there has been a lot of activity in the area of Business Intelligence (BI). Mergers, major acquisitions, and announcements have dominated the headlines in the past year. IBM has been quite active in this area as well. The recent acquisition of Cognos and the announcement of the Dynamic Data Warehouse Initiative are evidence of this. But IBM's i5/OS investment in and commitment to BI has not just been recent; it has been a point of emphasis in the last several releases, going back to V4R1. Consider some of the following database enhancements that have been built into the operating system over the last decade and beyond:

• SQL Query Engine (SQE)
• On Demand Performance Center
• Database Parallelism
• Materialized Query Tables
• Encoded Vector Indexing
• Remote Journaling
• Autonomic Indexes
• Index Advisor
• Database Monitors
• OmniFind Text Search Server
• Grouping functions (cube, rollup, and grouping sets)

While many enhancements have been made in the database, one area where little was done was a query and reporting tool that could take advantage of these enhancements. Until recently, Query/400 was one of the few IBM i5/OS offerings in this area; and while it was a very successful and popular product, it simply wasn't keeping pace with today's demanding BI requirements. For starters, all aspects of the tool's interfaces (report development, end user invocation, and output formats) are text-based and limited to the green-screen. Another major drawback is that all Query/400 requests are processed by the Classic Query Engine (CQE). This query optimizer technology is not taking advantage of the latest in IBM technology, and products like Query/400 that do not leverage standard SQL interfaces will (for the foreseeable future) only be able to use the less "intelligent" and poorer performing CQE. In fact, CQE has received no major enhancements in several years. IBM has been focusing on SQE, a redesigned database engine that is more efficient and provides more features. At this time, only standard SQL requests are processed by SQE. The result is that Query/400 is unable to leverage the latest database technologies and generally exhibits poorer performance, with limited analysis tools to understand the poor performance.

Interestingly, Query/400 continues to be used by almost every customer. However, the usage of Query/400 is often relegated to being a data extraction tool in a process of moving the data to another database. Over the past several months, while teaching workshops or presenting at conferences or through Webinars, I often poll the audience to get a count of Query/400 usage. Nearly everyone responds that they are using Query/400. However, when I ask how many are using Query/400 as their primary database analysis tool, the number of respondents is a small percentage! It's quite clear that Query/400, while useful and easy to use, simply does not have the functions and features that customers are looking for. Some of those things include the following:

• Ability to select a column and drill down to the next level of detail
• Conditional styling (also known as traffic lighting) to make certain facts in the report stand out
• Graph generation: Bar charts, pie charts, and other graphical data representations are essential in order to give users the ability to quickly consume and comprehend the data
• Modern output formats such as spreadsheets, PDFs, and HTML to the Web browser
• Parameterization and ad hoc report: Ability to allow report consumers to select the information they want to see via selection parameters and then have the report filter the data based on the parameters specified
• Dashboards: A single view for executives or business analysts to see how the business is performing compared to key performance metrics. The single view is graphical and allows a "one-stop shopping" view for many data points that would otherwise be delivered in an untimely or incomprehensible manner. Information can be complex in nature and can originate from a variety of data sources. Graphical elements such as graphs, gauges, and charts are combined with high-level views to provide a visually rich, single interface so that users can quickly comprehend the information they are looking for.
• Interactive analysis: When users analyze a report, sometimes the data reveals aspects that raise even more questions. In order to answer those questions, the analyst needs to look at the data in different ways (drilling down, drilling up, "slicing and dicing" the data).

Time for a New Offering: DB2 Web Query

To fill this void, IBM partnered with Information Builders Incorporated to deliver DB2 Web Query for System i to the marketplace. Information Builders is an IBM Business Partner that has been in the information and reporting business for over 30 years and has long been a leader in delivering BI solutions to their customers. In fact, their WebFOCUS product, recognized by Gartner Group (an industry analyst) as a top BI software product, runs on many platforms (including i5/OS).  

The DB2 Web Query product is essentially a subset of WebFOCUS, one that runs natively on i5/OS. The product is sold, marketed, and supported by IBM and in fact is packaged as Licensed Program Product (LPP) 5733QU2 and is installed by using the familiar RSTLICPGM interface.

It must be stressed that while a very powerful product, DB2 Web Query is not WebFOCUS. There are many features in WebFOCUS that are not included in DB2 Web Query. The objective of this product is to provide a query and reporting tool to meet the majority of the BI needs of most System i shops. WebFOCUS is based on the powerful FOCUS 4GL programming language, which supports, behind the scenes, DB2 Web Query. But for a full 4GL implementation to enable highly customized query or application logic, WebFOCUS would be the recommended solution.   

What Is Different About DB2 Web Query?

Unlike many BI solutions, DB2 Web Query was designed as a solution for customers running their business on the System i, with a majority of their data in DB2 for i5/OS, and their IT infrastructure built around that platform. And unlike those other solutions, DB2 Web Query allows you to query your data in DB2 for i5/OS, enabling you to leverage the advantages provided by a database that is tightly integrated with the operating system: database optimization, security, backup, change management, and other procedures and policies. This greatly simplifies the delivery of information to users without a requirement for server farms, data replication, and a completely new set of administration processes and procedures.

It is designed as a Web-based product, minimizing client configuration, licensing, and administration. All components of DB2 Web Query reside in a single installation of i5/OS, including the Web application server, the reporting server, the database engine, the metadata files, and the report definitions.

Other available BI solutions also claim to run natively on System i. A key advantage that DB2 Web Query has over some of these is its ability to generate SQL for data access, thereby leveraging the new SQL Query Engine (SQE). Other solutions rely exclusively on data access techniques such as OPNQRYF and the QQQQRY APIs. These technologies are still supported and do a fine job of data retrieval and manipulation, but all of these non-SQL requests are processed by the older CQE technology. The advantages of SQE over CQE are many and are well-documented in other publications (see the IBM Redbook Preparing for and Tuning the SQL Query Engine on DB2 for i5/OS, SG24-6598), but to summarize, here are the primary SQE benefits:

• Better access methods and smarter techniques to retrieve, sort, join, and group data, which result in better overall performance
• Ability to use more features: Only SQE can utilize new database technologies such as Autonomic Indexing, Materialized Query Tables (MQTs), and the ability to cache query results.
• Better optimization and performance tools: Only queries processed by SQE will have their plans stored in the SQE plan cache, an internal matrix-like repository that stores plans and statements processed by the new engine. Tooling available in the System i Navigator product provides an interface to easily access this valuable source of information.

Base Product 5733QU2

The base option for DB2 Web Query includes the development tools and runtime necessary to create a standard, Web-based query and reporting environment. With the base option, you can create reports and graphs that far surpass anything you could do with Query/400.

The product follows a pricing model that is based on the server's processor tier. This means, for example, that a company with a P30 tier will pay more for the product than one that installs it on a P10 tier. Each tier comes with a number of entitled user licenses. Every i5/OS user profile that needs to develop or run reports requires a user license and must be registered using the license manager component of the product. If you need more named user licenses than the tier provides, you can purchase more (at a flat fee per license). However, another option exists to support users through a single or a minimum number of user licenses, and that is the Run Time User Enablement option. More information on that option is provided later in this article.

When you order product id 5733QU2 base option, the following components are included:

• Three data adapters to interface with the database
• Three Web-based report authoring tools
• DB2 Web Query Reporting Server and meta data storage
• IBM's Integrated Web Application Server

Three Data Adapters

DB2 Web Query uses three data adapter technologies to access the data in DB2 for i5/OS. Each adapter is used to not only access data, but also create and manage the product's metadata layer (also known as synonyms). Synonyms provide an abstraction layer that describes to DB2 Web Query the overall structure of the data source as well as the specific data elements (field names, field types, etc.). Before you create reports or graphs against any data source, you must first create a synonym for that data source.

Depending on your requirements, you will use one or more of the following data adapters:

• DB2 CLI Adapter: When a synonym is created using the DB2 CLI adapter, the adapter will convert the report specifications (sort columns, display columns, selection parameters, etc.) into one or more SQL statements. This SQL statement is ultimately submitted to the DB2 for i5/OS engine from the DB2 Reporting Server using Call Level Interface (CLI). The use of SQL as the database access method is quite important because when that SQL statement is executed, the database will try to use SQE to process the request. If no SQE inhibitors prevent this from happening, the report will (in most cases) run faster than a request in which the underlying synonym is based on one of the two other adapters. In other words, if you want to take full advantage of SQE, your objective should be to use this adapter for all synonyms. However, this is not always possible, so two other adapters are provided.
• Query/400 Adapter: This adapter is all about investment protection. It allows shops with many *QRYDFN objects to leverage their significant investment by importing these objects into DB2 Web Query. This will allow most Query/400 objects to be brought into DB2 Web Query unchanged and run through DB2 Web Query. The output can be "Webified" or imported into other modern output formats such as PDF, Active Reports, and Excel spreadsheets. However, as mentioned previously, database access for Query/400 requests is handled by CQE, not SQE. During benchmark testing done in Rochester, we observed some categories of queries that ran more than 10 times faster when the reports were developed based on DB2 CLI adapter synonyms.
• DB Heritage Adapter: Do your legacy applications use multi-format logical files or multi-member physical files? Did you know that SQL does not support these file types? Fear not, as the DB Heritage adapter allows DB2 Web Query to access and report against these files. This adapter uses the OPNQRYF command as the mechanism to access the files. While, like Query/400, OPNQRYF does not use SQE, this adapter does allow you to protect significant investments made in the multi-format and multi-member file types.

Three Web-Based Report Authoring Tools

The base option of 5733QU2 comes with three tools that developers can use to create modern reports and graphs:

Report Assistant is a report development tool that provides an easy-to-use, graphical environment for report creation and maintenance. Its intuitive interface eliminates the need for report authors to know database syntax and terminology, allowing them to focus on business logic to quickly create meaningful reports. In addition, its features accommodate several of the BI requirements listed previously. Those features include the following:

• Drill-down capabilities
• Conditional styling/traffic lighting
• Twelve output formats (including HTML, PDF, Excel spreadsheet, XML)
• On-demand paging to allow users to jump to a specific page in a large report
• Parameter-driven reporting for ad hoc capabilities

Graph Assistant is development tool for creating and editing graphs. It has the same general look and feel as Report Assistant, and in fact offers the same drill-down, conditional styling, output formats, and ad hoc features that Report Assistant does. But unlike Report Assistant, it includes several additional interfaces for selecting the types of graphs you want, as well as ways to customize the pie slices, bars, and x-y axis properties. If you're looking for a specific kind of graph to impress your management, Graph Assistant has over 120 different kinds of charts and graphs to choose from.

Power Painter is a report and graph creation utility. It can also be used as a layout tool for creating composite reports (reports that show multiple reports and graphs from one interface). Although its primary purpose is for creating reports in the PDF format, it can direct the output to HTML, Active Reports, and Excel spreadsheets. It is based on AJAX and Web 2.0 technologies, which means that there is an extensive amount of activity and communication between the client application and the server. As such, it does consume and require more memory on both the client side and on the server. It may appear that there is some overlap between Graph Assistant and Power Painter. This is somewhat true, but there are some differences. A common question is, "When would you use Power Painter instead of Graph Assistant?" Power Painter should be used when you want do the following:

• Create composite reports to spreadsheets or PDFs
• Create graphs that require advanced graphing features that Graph Assistant does not support. Examples of this include customized minimum/maximum values shown in a graph and ability to customize the increments (grid steps) of the graph.
• Use a development tool that is more of a "What You See is What You Get" (WYSIWYG) type of experience. The Power Painter interface is generally considered more modern and provides this type of experience to the developer.

DB2 Web Query Reporting Server

The DB2 Web Query Reporting Server is the "workhorse" that resides between the DB2 Web Query users and the database. It employs the data adapters to translate user queries and report requests into SQL statements, performs additional tasks to properly format the results, and delivers the content to the specified output format, ensuring ease of use and efficient requests for the database.

i5/OS Integrated Web Application Server

In V5R4, IBM made available a Web application server that is integrated in the operating system. DB2 Web Query uses this application server because it is more lightweight and requires a smaller memory footprint than the WebSphere Application Server. It is also easier to administer, start, and stop. In fact, the i5/OS commands provided by DB2 Web Query handle the starting and stopping of both the reporting server and the integrated application server. The implementation used by DB2 Web Query also uses IBM Technology for Java, the 32-bit Java Virtual Machine (JVM), which further reduces the system resources needed and generally results in better overall application performance.

Additional Components

The DB2 Web Query base product can essentially be thought of as a more modernized version of Query/400. While it is much more robust and feature-rich than Query/400, its capabilities are primarily for querying and reporting, just as the product it strategically replaces.

To take this product to the "next level" requires the implementation of several additional LPP options available from IBM. The powerful features included with these options are what transform DB2 Web Query from a modern reporting tool into a solution that provides true BI capabilities. For example, if you want the ability to enhance the metadata layer and hide the complexity of the database, provide visually rich dashboards to your management, give your business analysts the ability to slice and dice data to solve complex business problems, give your mobile employees access to powerful reports in a disconnected environment, and provide true ad hoc reporting capabilities to empower your end users, you will want take a serious look at these additional options.

Most of these options are server-based, chargeable features that, like the base component, follow a processor tier pricing model. The exception to this is 5733QU2 option 3 (Developer Workbench), which is a PC client application with a flat-fee pricing model. You purchase a license for each PC that the tool is installed on. Let's have a look at each of these options.

Active Reports (5733QU2 Option 1)

Active Reports is special report output format (specified in the Report Assistant tool) in which the data and the reporting controls are all stored within a single HTML file. These reporting controls provide the user with a variety of built-in data analysis and manipulation features. Here are some examples:

• Sort: The report can be sorted by any column in ascending or descending order.
• Filter/highlight: Any column or combination of columns that match specified values can be filtered or highlighted.
• Calculate: Math functions sum, minimum, maximum, count, and percentage of total can be applied to selected columns.
• Chart: Line, bar, and pie graphs can be displayed in a pop-up window.
• Rollup: Values in a selected column can be aggregated by another selected column and results displayed in a pop-up window.
• Pivot: Column values can be aggregated and results displayed in a cross tab grouped by two or more columns.
• Visualize: Bar charts of selected columns can be displayed for a visual comparison of column values.
• Export: Reports and charts can be extracted and immediately exported into other applications.

All these features are built-in and ready to use via a Web browser; no client application is required. In addition, since all the information is encapsulated in a single HTML file, you can do all this in a disconnected mode. The end user works with the data contained in the HTML file and does not require a network connection to the DB2 for i5/OS server.

This also means that users do not have to be registered named users. Only the user who originally ran and populated the report needs to be a licensed user (or member of a Run Time User Enablement group). The report can be distributed (emailed) to anyone, and those recipients simply open the HTML file in a Web browser and have the same information and same data manipulation capabilities as the user who originally created the report.

OLAP (5733QU2 Option 2)

Online Analytical Processing (OLAP) is a component of BI software that provides an interactive experience of working with the data. The user typically runs an initial report to analyze data or examine trends. The report may yield information that leads to more questions, or perhaps it may uncover a problem area within the corporation. To help the user gain more insight on the information, an OLAP application enables the user to extract and view the data at different points of view. This could mean dragging and dropping new fields into the report, filtering the data, re-sorting by a specific column, drilling down to the next level of detail, or pivoting specific columns in the report.

To perform this type of function, OLAP tools structure data in a hierarchical design. As such, OLAP is often associated with data that is multi-dimensional (often extracted from cubes or a data warehouse that was designed based on star schema or snowflake schema principles). However, this is not a requirement for DB2 Web Query. Databases that are relational can also be analyzed using the OLAP module in the Report Assistant tool. Once your OLAP dimensions and hierarchies are defined using the Developer Workbench application, report developers can simply "switch on" OLAP to enable these types of capabilities. This feature empowers your users, giving them many ways to dissect and manipulate the data to gain a better understanding of information that is buried within it.

Developer Workbench (5733QU2 Option 3)

Developer Workbench is a Windows client-based application that provides additional and advanced features for DB2 Web Query development activities. While synonym creation, as well as report and graph development tasks, can be accomplished by using the other tools, the numerous features provided in this application make it well worth looking into. Some of the highlights include the following:

• Synonym Editor: This advanced editor includes features to perform impact analysis, data profiling (statistics, patterns, values, and outliers), date decomposition, creation of synonym-level joins, creation of virtual (computed) columns, and the configuration of the dimensions and hierarchies needed for OLAP processing. Developers also have the option of viewing the synonym structure in a tree view, modeling view, or simple text editor.
• HTML Layout Painter: This component provides an interface from which developers can create layouts by inserting text, images, existing reports and graphs, and input selection parameters (using design elements such as radio buttons and drop-down lists). The tool then generates an HTML file so that this layout can be used as the framework for a Web page. Dashboards, composite reports, Key Performance Indicators (KPIs), and the front-end page for parameterized reports are developed using this very powerful accessory.
• SQL Report Wizard: If you prefer to create reports based on your own SQL statements or if you have SQL SELECT statements stored in files in the IFS, the SQL Report Wizard will be a useful tool in your environment.

Runtime User Enablement (5733QU2 Option 4; Available Only with V1R1M1)

In March of 2008, the Run Time User Enablement option was added to the product offering. While it provides no additional features, this option gives customers more licensing flexibility when the majority of the users are run-time only.

With the base licensing option of 5733QU2, all users (whether they are report developers or report users) must be explicitly registered as named users using the product's license manager component. Each of these named users consumes a user license. Consequently, a company with many users would be forced to create an environment that is both expensive and difficult to administer. This option was made in response to feedback received from the initial product release based on such environments. With Runtime User Enablement, a company in which the majority of users are run-time only users (perform no report development activities) can license those users under i5/OS group profiles. Each of these group profiles secures a unique set of reports. This means that runtime licensing is based on the number of reports sets being made available rather than the number of uses that access the reports. This can result in significant savings and reduced administration.

The Future

In March of 2008, IBM and Information Builders announced a joint product roadmap for DB2 Web Query. This roadmap provides a glimpse of what is to come in 2008 for this product. Some of the planned product enhancements and offerings included in this announced were:

• A report scheduling and distribution application that supports the generation and automated email distribution of reports
• A Software Development Kit (SDK) that will provide the capability for application developers to embed DB2 Web Query content into any application as well as drive report execution and delivery through the previously described report distribution application
• An Excel plug-in to provide users the ability to execute DB2 Web Query reports from within Excel spreadsheets and pull the report results directly into the spreadsheet

Summary

Query/400 was a very popular product and had a great run. Indeed, many thousands of licenses have been sold since 1988, and customers continue to use it today. But as useful as it was, its time as a primary query and reporting tool has passed. A changing of the guard has occurred, and with it comes a wealth of new features to deliver the information that your users need, in the format that they want, and with additional features that allow them to utilize true Business Intelligence capabilities. All this from a product that is designed to leverage all the good things about i5/OS: security, backup and recovery, and use of the integrated database...and all from a single i5/OS server. This means easier installation, maintenance, problem determination, licensing, and queries that are optimized for DB2 for i5/OS.

If you would like more information about this exciting new product, I'd suggest starting with the DB2 Web Query for System i home page.

Also take a look at the IBM Redbook Getting Started with DB2 Web Query for System i, which accompanied the product launch. It contains much more information about the product, tutorials to help you easily get started using the product, and performance tips and techniques to maximize your usage of the latest database technologies.

Gene Cobb is a DB2 for i5/OS Technology Specialist in IBM's ISV Business Strategy & Enablement for System i group. He has worked on IBM midrange systems since 1988, including over 10 years in the IBM Client Technology Center (now known as IBM Systems and Technology Group Lab Services). While in Lab Services, he assisted customers with application design and development using RPG, DB2 for i5/OS, CallPath/400, and Lotus Domino. His current responsibilities include providing consulting services to System i developers, with a special emphasis in application and database modernization. He can be reached at This email address is being protected from spambots. You need JavaScript enabled to view it..

Also Read

TechTip: Fine-Tuning Db2 Symmetric Multiprocessing (SMP) to Improve Performance for Long-Running Queries

Celebrating 40 Years of Db2: Running the World’s Mission Critical Workloads

TechTip: Data Source Base Project, Part 3 - SSIS Code Structure