02
Sat, Nov
2 New Articles

IBM i Devs Get Some REST with OData!

Programming - Other
Typography
  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times

 

Adopt the OASIS Open Data Protocol to consistently publish and consume REST services.

 

In the article "Implement a Service-Oriented Architecture with REST APIs," I covered some basics about implementing a service-oriented architecture using REST APIs. In general, because the REST methodology doesn't have many standards regarding how to pass parameters, how to format the result data, etc., the implementation of these APIs can vary widely from vendor to vendor and developer to developer.

 

Originally started by Microsoft, the Open Data Protocol (OData) was devised in part as a means of having a standard input and output so that REST APIs based on OData would be consistent in their implementation. This standardization offers many benefits, including the ability to have a generic tool set connect to an OData source and consume the data. Just as ODBC paved the way as a standard for database access, OData is striving to be a popular standard for REST. OData's slogan is "The Protocol for REST APIs."

ODBC was great because it allowed client/server developers a standard way to access various database servers (such as DB2, SQL Server, Oracle, etc.) using the same coding practices. Because it was standardized and could access database metadata, various tools could interrogate the database server and build a list of available tables, columns, and other database objects to present to the user. The drawbacks with ODBC are that it's primarily Windows desktop- and server-based (with ports available on Linux), each database manufacturer needs to provide a driver, and the driver needs to be installed on each machine requiring database access. That's quite a bit of deployment maintenance.

With REST, ODBC's problems go away in that no special "middleware" providers/drivers are required and that use of Internet standards makes REST platform- and OS-independent. Building on REST's and ODBC's benefits, OData gives developers a standard way to "program" against an API, which increases productivity.

To illustrate some OData concepts, I'll be using the sample test services conveniently offered on OData.org to demonstrate some OData highlights (as I perceive they'd be of interest to the IBM i community). This article will merely be an introduction to OData, but interested parties can pursue much more at OData.org.

 

Excel and Other Tools

To start, I'll defer some boring technical explanations in favor of showing how OData can be used by Excel. Microsoft Query (built into Excel) was one of the early tools that allowed users to access database data with ODBC. Excel 2013 allows users to directly query data from an OData service with similar benefits.

To access data from an OData API in Excel 2013, there are a few simple steps to follow: choose the Data ribbon > "From Other Sources" > From "OData Data Feed" as shown in Figure 1.

 

011415SansoterraFigure1

Figure 1: In Excel 2013, select an OData Data Feed as a source to import data directly from an OData API.

When prompted for the location of the OData data feed, simply paste in the URI to the sample Northwind Summary of Sales by Years test service:


http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years

This sample OData resource returns a list of orders, their ship dates, and their subtotalssimilar to a database query against an ERP orders table.

Excel gives the option to put in credentials, but the test service does not require them, so simply hit Next. You will be prompted to check the checkbox next to the Summary_of_Sales_by_Years table and then click Finish. The last Excel prompt allows you to place the OData feed as a table, pivot table, etc. at a specified location in the worksheet. Shown in Figure 2 is a partial result set with the OData feed formatted as a table (the Excel default):

ShippedDate

OrderID

Subtotal

11/27/1996

10358

429.4

8/13/1997

10593

1994.4

1/21/1998

10836

4705.5

9/18/1997

10662

125

7/30/1996

10261

448

12/3/1997

10751

1631.48

3/3/1997

10455

2684

1/30/1998

10850

629

Figure 2: This abridged Excel data table was created directly from the Summary_of_Sales_by_Years OData feed.

This simple demonstration illustrates that OData APIs can offer data and logic to be consumed by a client similar to how ODBC clients access data and logic from a database server. But unlike ODBC, a driver installation is not required, knowledge of a particular database's data types and conventions is not required, and it can theoretically be accessed by any modern device or platform.

Incidentally, Excel 2010 and 2013 can also access OData and many other data feed types via the Power Query add-on.

In addition to Excel, many other tools and products can produce and/or consume an OData feed, including these:

  • SharePoint 2010 and later
  • Microsoft Azure
  • IBM WebSphere
  • SQL Server Reporting Services (reading data via the XML data source wrapped with a special query, producing an AtomPub feed with SQL Server 2008 R2 and later)
  • SQL Server 2012 and later Integration Services (using an OData source)
  • SAP NetWeaver Gateway
  • Microsoft Dynamics NAV ERP
  • eBay

As for OData-capable development environments:

  • Publish using .NET applications using WCF
  • Publish or consume using .NET applications created with LightSwitch
  • Publish or consume using Java applications using OData4j (a work in progress) or Apache Olingo.
  • Any applications that can take advantage of ODBC, OLE DB, or JDBC using a tool from RSSBUS that puts an ODBC (or OLE DB using ODBC)/JDBC wrapper around an OData feed
  • Others listed on the OData website, including options for iOS and Python

As applications become "service-oriented," it's easy to see the advantage of being able to share data among multiple products and programming languages with a standard protocol.

Keep in mind there are several versions of OData (with v4 as the latest) and you'll need to check with your tool/application provider to find out what version(s) are supported. As noted, Microsoft is the big supporter of OData but other major companies have contributed to the standard too.

The Ins and Outs of an OData Service

Back to the technical explanations, I'll describe here a little bit about how OData works.

URI Structure for Accessing an OData Service

The OData standard has laid out a specific URI syntax to support many features, as shown in Figure 3.

011415SansoterraFigure3

Figure 3: This is the anatomy of an OData URI (from odata.org).

This URI structure should be familiar to web developers, particularly those working with HTTP-based services. The Query Options portion of the URI is where OData's features shine. Shown below are several examples of useful things that can be done with the OData URI, although this list certainly is not exhaustive. And the good news is you can paste these URIs in your browser to see how they work. Oddly enough, I'd stay away from Internet Explorer in these examples in favor of Chrome or Opera, where it's easier to see the raw data returned from the service.

URI Examples

Starting with the same base URI used in the Excel example above, pasting this in a browser will return a summary of all orders in the OData test system (with results returned as XML):

http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years

Building on the base URI, the OData specification offers several options for using the URI to filter, sort, compute, return a count, etc. To get a feel for OData's features, paste the next several URIs into your browser as well.

This next URI uses a filter query option to obtain all sales with a subtotal equal to a specific value of 654.06 (the M is used to indicate that the number is a decimal literal):

http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years?$filter=Subtotal eq 654.06M

Of course, specifying the text $filter=Subtotal eq 654.06M is responsible for creating the filter. For another feature example, this URI selects only the top 10 sales entries, sorted by subtotal in descending order:

http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years?$orderby=Subtotal desc&$top=10

When the host OData service supports it, client-side paging can even be implemented with the use of $skip and $top. (It is important to note that not every OData API is required to implement all of these featurers.)

In this next example, the first 50 rows of the result set are skipped and only 10 rows are returned, while $orderby ensures consistency in the rows returned.

http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years?$skip=50&$top=10&$orderby=Subtotal desc

If you want to know how many rows will be returned by a service (which is useful when the client-side UI pages data), use the $count system query option:

http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years/$count

One other notable feature is the $metadata operation that can be used to retrieve the metadata provided by an OData service root's resources:

http://services.odata.org/V3/Northwind/Northwind.svc/$metadata

Consider an excerpt from the metadata associated with the Summary_of_Sales_By_Quarter resource:

<EntityType Name="Summary_of_Sales_by_Quarter">

<Key>

<PropertyRef Name="OrderID"/>

</Key>

<Property Name="ShippedDate" Type="Edm.DateTime"/>

<Property Name="OrderID" Type="Edm.Int32" Nullable="false"/>

<Property Name="Subtotal" Type="Edm.Decimal" Precision="19" Scale="4"/>

</EntityType>

It's easy to discern from this XML that three pieces of information are returned by the resource, including ShippedDate (DateTime), OrderId (Int32), and Subtotal (Dec(19,4)). Also the PropertyRef named OrderId indicates that a specific piece of information related to a single order id can be obtained by passing the order id in the URI as follows (using order id 10250 as an example):

http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years(10250)

This URI will return the sales only for the requested order id. If a resource's unique identifier is a string (such as an item number), it can be requested using a string value delimited by single quote. Each OData service is expected to have a unique identifier.

Further, the value of a specific property can be requested (in this case Subtotal). When the path segment contains $value, the raw value is returned (without XML or JSON formatting). This URI returns the plain value 1552.6000, the subtotal value of order id 10250:

http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years(10250)/Subtotal/$value

One important thing to note is that the names in these queries are case-sensitive (Subtotal is not the same as SUBTOTAL). These subsequent URI examples can be modified to use the other resources available by this OData test service, such as querying the products that have a unit of measure ending with ml (milliliter):

http://services.odata.org/V3/Northwind/Northwind.svc/Products?$filter=endswith(QuantityPerUnit,'ml')

Or query suppliers with a postal code starting with 99:

http://services.odata.org/V3/Northwind/Northwind.svc/Suppliers?$filter=startswith(PostalCode,'99')

For more information on OData URIs, including how to do computations, see the following links:

http://msdn.microsoft.com/en-us/library/ff478141.aspx

http://www.odata.org/documentation/odata-version-2-0/uri-conventions/

OData Responses: AtomPub (XML) and JSON

OData supports returning a response as the XML-based Atom Publishing Protocol (aka AtomPub) or JSON. Both of these formats are popular today and can be produced or consumed on the IBM i as discussed here. I'll show a few examples of the output formats in the next section(s). If you're unfamiliar with AtomPub, it’s the same XML you’ve seen show up in your browser if you've been reviewing the URI examples.

OData and CRUD Operations

OData is also geared for CRUD (CREATE, READ, UPDATE, DELETE) operations using HTTP verbs like DELETE (delete), POST (insert), PUT/PATCH (update), etc. This standardization gives clients a consistent way of updating data via an OData service.

An MSDN Blog titled oData and JSON payload examples shows a few examples of using JSON to perform CRUD operations using the Northwind sample OData service. Although you can't easily run these in a browser, the blog shows that it's quite easy to create the appropriate HTTP request programmatically.

This is an HTTP DELETE example from the blog that requests Product Id 200 be removed.

DELETE http://services.odata.org/V3/(S(ettihtez1pypsghekhjamb1u))/OData/OData.svc/Products(200) HTTP/1.1

DataServiceVersion: 1.0;NetFx

MaxDataServiceVersion: 3.0;NetFx

Accept: application/json;odata=minimalmetadata

Host: services.odata.org

The random portion of the URL (after the V3) is temporarily assigned for the test service. Likewise, another JSON example is reproduced here using an HTTP POST to create a new product (with data supplied as JSON):

POST http://services.odata.org/v3/(S(ettihtez1pypsghekhjamb1u))/odata/odata.svc/Products HTTP/1.1

DataServiceVersion: 3.0;NetFx

MaxDataServiceVersion: 3.0;NetFx

Content-Type: application/json;odata=minimalmetadata

Accept: application/json;odata=minimalmetadata

Accept-Charset: UTF-8

Host: services.odata.org

Content-Length: 180

Expect: 100-continue

{

   "odata.type":"ODataDemo.Product",

   "Description":null,

   "DiscontinuedDate":null,

   "ID":200,

   "Name":"My new product from leo",

   "Price":"5.6",

   "Rating":0,

   "ReleaseDate":"0001-01-01T00:00:00"

}

Of course, these data modification operations are possible only when the OData API supports it. (Otherwise, the danger of linking updateable ERP tables within a Microsoft Access database will be lived all over again!)

In summary, when supported by the service provider, OData queries come with the same benefits of database queries (available metadata; versatility with filtering, ordering, updating; and strongly typed results) with the advantage that OData, because it uses standards, doesn't require database-specific middleware.

DB2 Query to "Manually" Parse Atompub

Finally, let's not forget that good ol' DB2 can consume an OData source that returns AtomPub, which is, after all, XML. The following DB2 for i example query uses the new HTTP function HTTPGETBLOB to retrieve and parse the AtomPub data from the Summary_Of_Sales_By_Years test OData source (the same URI used in the Excel example).

Notice that the second parameter of HTTPGETBLOB is a request header containing an Accept key/value pair. This is required in order for the server to return a response. Note: DB2 HTTP functions require IBM i 7.1 with database group PTF level 23.  Finally, the XMLTABLE table function is used to shred the AtomPub XML returned from the service.

WITH XML_DATA AS (

Select XMLPARSE(DOCUMENT SYSTOOLS.HTTPGETBLOB('http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years',

'<httpHeader>

<header name="Accept" value="text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" />

<header name="Accept-Encoding" value="gzip" />

</httpHeader>')) AS ODataSummary

From SYSIBM.SYSDUMMY1

)

SELECT Shred.*

FROM XML_DATA,XMLTABLE(

XMLNAMESPACES(

'http://schemas.microsoft.com/ado/2007/08/dataservices' AS "d",

'http://schemas.microsoft.com/ado/2007/08/dataservices/metadata' AS "m",

'http://www.w3.org/2005/Atom' AS "a",

'http://services.odata.org/V3/Northwind/Northwind.svc' AS "base"),

'$doc/a:feed/a:entry/a:content/m:properties' PASSING XML_DATA.ODataSummary AS "doc"

COLUMNS

/* On i 7.2 you can use TIMESTAMP(0) */

ShippedDate TIMESTAMP     PATH 'd:ShippedDate',

OrderId     INT          PATH 'd:OrderID',

SubTotal   DECIMAL(19,4) PATH 'd:Subtotal'

) AS Shred

;

The results from this query are the same as the results shown from Excel in Figure 2. Basically, OData-aware tools examine the metadata and perform the above type of transformation automatically so the developer doesn't have to do the parsing!

The SQL query can be modified so that the service returns JSON instead of AtomPub XML. This can be done by simply changing the request header's "Accept" value:

WITH JSON_DATA AS (

Select SYSTOOLS.HTTPGETCLOB('http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years',

'<httpHeader>

<header name="Accept" value="application/json" />

<header name="Accept-Encoding" value="gzip" />

</httpHeader>') AS ODataSummary

From SYSIBM.SYSDUMMY1

)

SELECT *

FROM JSON_DATA

The abridged JSON result looks like this:

{"odata.metadata":

"http://services.odata.org/V3/Northwind/Northwind.svc/$metadata#Summary_of_Sales_by_Years",

"value":

[{"ShippedDate":"1996-07-16T00:00:00","OrderID":10248,"Subtotal":"440.0000"},

{"ShippedDate":"1996-07-10T00:00:00","OrderID":10249,"Subtotal":"1863.4000"},

{"ShippedDate":"1996-07-12T00:00:00","OrderID":10250,"Subtotal":"1552.6000"},

{"ShippedDate":"1996-07-15T00:00:00","OrderID":10251,"Subtotal":"654.0600"},

{"ShippedDate":"1996-07-11T00:00:00","OrderID":10252,"Subtotal":"3597.9000"},

{"ShippedDate":"1996-07-16T00:00:00","OrderID":10253,"Subtotal":"1444.8000"},

{"ShippedDate":"1996-07-23T00:00:00","OrderID":10254,"Subtotal":"556.6200"},

{"ShippedDate":"1996-07-15T00:00:00","OrderID":10255,"Subtotal":"2490.5000"},

{"ShippedDate":"1996-07-17T00:00:00","OrderID":10256,"Subtotal":"517.8000"}

etc.

Although probably not extremely useful with traditional IBM i languages such as RPG or COBOL, JSON is a frequently used format for client-side (i.e., browser) scripting in web-based applications. This is because the above result can be transformed into a JavaScript object that can be bound to controls on the web page, among other things.

Conclusion

OData provides a standard feature-rich protocol for REST APIs. These features include complex filtering, querying, sorting, paging, CRUD, and metadata retrievalto name a few. IBM i and other shops can benefit by adopting the OData protocol when creating REST APIs and consuming them.

References

Implement a Service-Oriented Architecture with REST APIs

JSON and XML Conversion in DB2 for i

Rev Up "i" Reporting with SQL Server 2008 Reporting Services

Michael Sansoterra is a DBA for Broadway Systems in Grand Rapids, Michigan. He can be contacted at This email address is being protected from spambots. You need JavaScript enabled to view it..


BLOG COMMENTS POWERED BY DISQUS

LATEST COMMENTS

Support MC Press Online

$

Book Reviews

Resource Center

  • SB Profound WC 5536 Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. You can find Part 1 here. In Part 2 of our free Node.js Webinar Series, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Brian will briefly discuss the different tools available, and demonstrate his preferred setup for Node development on IBM i or any platform. Attend this webinar to learn:

  • SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.

  • SB HelpSystems ROBOT Generic IBM announced the E1080 servers using the latest Power10 processor in September 2021. The most powerful processor from IBM to date, Power10 is designed to handle the demands of doing business in today’s high-tech atmosphere, including running cloud applications, supporting big data, and managing AI workloads. But what does Power10 mean for your data center? In this recorded webinar, IBMers Dan Sundt and Dylan Boday join IBM Power Champion Tom Huntington for a discussion on why Power10 technology is the right strategic investment if you run IBM i, AIX, or Linux. In this action-packed hour, Tom will share trends from the IBM i and AIX user communities while Dan and Dylan dive into the tech specs for key hardware, including:

  • Magic MarkTRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now!  Request Now.

  • SB HelpSystems ROBOT GenericForms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.

  • SB HelpSystems ROBOT GenericIT security is a top priority for businesses around the world, but most IBM i pros don’t know where to begin—and most cybersecurity experts don’t know IBM i. In this session, Robin Tatam explores the business impact of lax IBM i security, the top vulnerabilities putting IBM i at risk, and the steps you can take to protect your organization. If you’re looking to avoid unexpected downtime or corrupted data, you don’t want to miss this session.

  • SB HelpSystems ROBOT GenericCan you trust all of your users all of the time? A typical end user receives 16 malicious emails each month, but only 17 percent of these phishing campaigns are reported to IT. Once an attack is underway, most organizations won’t discover the breach until six months later. A staggering amount of damage can occur in that time. Despite these risks, 93 percent of organizations are leaving their IBM i systems vulnerable to cybercrime. In this on-demand webinar, IBM i security experts Robin Tatam and Sandi Moore will reveal:

  • FORTRA Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:

  • FORTRAManaging messages on your IBM i can be more than a full-time job if you have to do it manually. Messages need a response and resources must be monitored—often over multiple systems and across platforms. How can you be sure you won’t miss important system events? Automate your message center with the Robot Message Management Solution. Key features include:

  • FORTRAThe thought of printing, distributing, and storing iSeries reports manually may reduce you to tears. Paper and labor costs associated with report generation can spiral out of control. Mountains of paper threaten to swamp your files. Robot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing. Manage your reports with the Robot Report Management Solution. Key features include:

  • FORTRAFor over 30 years, Robot has been a leader in systems management for IBM i. With batch job creation and scheduling at its core, the Robot Job Scheduling Solution reduces the opportunity for human error and helps you maintain service levels, automating even the biggest, most complex runbooks. Manage your job schedule with the Robot Job Scheduling Solution. Key features include:

  • LANSA Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.

  • LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed. Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

  • LANSASupply Chain is becoming increasingly complex and unpredictable. From raw materials for manufacturing to food supply chains, the journey from source to production to delivery to consumers is marred with inefficiencies, manual processes, shortages, recalls, counterfeits, and scandals. In this webinar, we discuss how:

  • The MC Resource Centers bring you the widest selection of white papers, trial software, and on-demand webcasts for you to choose from. >> Review the list of White Papers, Trial Software or On-Demand Webcast at the MC Press Resource Center. >> Add the items to yru Cart and complet he checkout process and submit

  • Profound Logic Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.

  • SB Profound WC 5536Join us for this hour-long webcast that will explore:

  • Fortra IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn: