Clustered storage has a role to play in improving IBM i availability under some scenarios, but there are network implications. This article examines these important issues.
An increasing number of organizations are at least considering the use of clustered data storage, a technology that has network implications when used in a multi-site configuration. The network implications are often not fully understood and may lead to failed implementations or unexpected costs. The purpose of this article is to help build that understanding.
Like many terms, "clustered storage" means different things to different people. Consequently, a definition is needed in order to put a stake in the ground before proceeding. In the context of this article, clustered storage refers to the redundant storage of data (both business and system data) at two or more nodes in a cluster for the purpose of achieving higher availability or disaster recovery. Under this definition, the replication of data occurs at the storage subsystem level rather than the database and object level.
Over the last few years, high availability options for IBM i (for "IBM i" read also AS/400, iSeries, System i, OS/400, and i5/OS) have broadened to include clustered storage replication technologies. With the announcement and release of Power Systems and IBM i 6.1, these technologies have been renamed and promoted.
Clustered Storage Primer
Before discussing the related network considerations in detail, a brief overview of clustered storage and its use is in order.
Traditional logical replication, which has been the core technology underlying IBM i high availability for many years, operates at the database and object level. It creates and maintains real-time or near real-time replicas of constructs such as libraries, files, spool files, records, profiles, and so on.
In contrast, clustered storage replication takes place below the database and object level. It works at the disk subsystem level to replicate pages (blocks) stored on disk. At this level, there is no knowledge of what the bytes in a page represent. For replication at this level, when the disk subsystem gets write requests to update pages, it replicates the entire page that has been changed to a remote disk subsystem. This form of replication is simple in concept, but it requires a complex implementation underneath the covers. A part of that complexity is imposed by the network.
Clustered storage replication concepts are generally OS-independent, and the technology is more prevalent in non-IBM i environments that lack some of the unique logical replication capabilities provided by IBM i.
The above is not intended to be a detailed comparison of storage-level versus logical replication, which is the subject of an article scheduled for publication in MC iTechnology Manager on June 23. Each form of replication has its purpose and, in some cases, the two are complementary.
IBM i Storage-Based Replication
IBM i groups its various clustered storage replication technologies under an umbrella brand of Cross-Site Mirroring (XSM), which includes three distinct capabilities with different characteristics: Geographic Mirroring, Metro Mirroring, and Global Mirroring.
•· Geographic Mirroring is an IBM i operating system capability that works with either internal storage or SAN-based storage. It replicates independent ASPs (IASPs) but cannot replicate data on system ASPs. In general, replication operations are synchronous, meaning that the data written to the source system storage is transported to the target system before the operation requesting the disk write transfers control back to the application. Contrary though it may seem, this is true for both the "synchronous" and "asynchronous" configuration options for Geographic Mirroring! The only difference between "synchronous" and "asynchronous" Geographic Mirroring in IBM terminology is that with asynchronous mirroring, less processing is required on the target system before the acknowledgement is sent back to the source system, but that acknowledgement must still be received on the source before the application can continue. From a purely technical standpoint, Geographic Mirroring does not limit the separation of the source and target systems. However, because network access is required for the operation to complete, and total network travel time, including the time required to send back an acknowledgment, depends in part on how far the data must travel, there are very real practical limits to the distance between the source and target.
•· Metro Mirroring is an IBM storage capability that works only with IBM SAN storage, but it can replicate both IASP and system ASP. Like Geographic Mirroring, Metro Mirroring is synchronous in that the target SAN must receive the data before control is returned to the application. Thus, Metro Mirroring has network distance limitations similar to those of Geographic Mirroring.
•· Global Mirroring is similar to Metro Mirroring in that it works only with IBM SAN storage but can replicate both IASP and system ASP. The difference is that Global Mirroring operates asynchronously, which reduces the network requirements needed for replication. Specifically, there is no distance limitation, and bandwidth requirements are less critical to application performance.
Clustered Storage Network Considerations
Network considerations fall into two categories: Distance (i.e., the maximum practical separation between the source and target) and bandwidth (i.e., the additional network pipe size required to accommodate replication traffic), but it is worth noting that these are real issues only when you intend to run XSM replication outside the local data center.
Distance
Even if bandwidth is unlimited, the response time required to send data across a network can be significant, and this latency increases as the distance between the source and target systems increases. Consequently, because of the synchronous nature of replication under Geographic or Metro Mirroring, increasing distances has a negative impact on application performance and throughput when using either of these technologies.
Both Geographic and Metro Mirroring work best (i.e., they impose minimal application impact) when the separation between systems is limited to metro area distances: Fifty miles or less would be a reasonable practical guideline.
Global Mirroring has a distinct advantage when it comes to network distance considerations. Because it replicates data asynchronously, network traffic is normally irrelevant when determining how long it will take for control to be returned to a user application. Communication of page changes is normally completed within a few seconds, but whether the time required is sub-second or several seconds has no impact on application response times.
The tradeoff that you must accept when using Global Mirroring is that, in the event of a source system failure, some page updates may not make it to the target, and, if not, they will be lost. Consequently, you cannot use Global Mirroring if you have a Recovery Point Objective (RPO) of zero. (RPO is a measure of how much data you are willing to lose in the event of a disaster or other system failure.) In contrast, because they perform synchronous replication, Metro and Geographic Mirroring do not risk any data loss in the event of a failure and, thus, can support an RPO of zero.
Bandwidth
To prevent significant application slowdowns, when using Geographic and Metro Mirroring, care must be taken to evaluate the peak bandwidth needed to replicate all pages immediately as they change. Make sure your network pipe is larger that the peak requirement. In addition, either ensure that the network bandwidth will be dedicated to this purpose as required or implement an appropriate QoS (quality of service) configuration that will prioritize this traffic as critical.
When using Global Mirroring, temporary spikes in bandwidth needs are less relevant. Nonetheless, you should look for any sustained periods of high usage and make sure your bandwidth can accommodate the most demanding of these periods with minimal replication latency. The tradeoff you must consider is that the increased cost required to acquire more bandwidth buys improved sustained RPO at times when network demand would otherwise exceed the available bandwidth.
Application Implications
Clustered storage replication also has implications for application environments. When using any of the mirroring technologies described above, it is important, and an absolute requirement for Geographic Mirroring, that the application be adapted to an IASP environment in order to ensure high availability. Despite it being possible to protect non-IASP data with Metro and Global Mirroring, recovery times after a disaster or other failure are long when using ASP storage because failing over requires an abnormal IPL. An abnormal IPL will result in significantly increased recovery time. Thus, if you are using one of the mirroring technologies to protect non-IASP data, the result is more of a disaster recovery solution than a high availability solution.
Bandwidth requirements for mirroring are highly dependent on the nature of the applications using the mirrored storage. The key factor is how write-intensive the applications are. When estimating network bandwidth requirements and costs and/or when working to minimize those requirements and costs, there are three significant areas to consider:
• Write activity: Best practices for application performance design and implementation avoid writing data to storage unnecessarily, although the definition of "unnecessary" might be subject to some debate during the application design phase. Superfluous writes might not be a serious issue if you don't use XSM because the resulting local performance degradation may be tolerable. However, it may become a problem when the changed pages must be replicated to a remote machine before the application can continue. Thus, before implementing, XSM applications should be tuned to use in-memory constructs where possible.
• IASP use: Design or tune applications such that temporary (non-critical) objects and files are placed outside of the replicated environment (i.e., a non-replicated IASP or, in the case of Geographic Mirroring, the system ASP).
• Database indexes: Reduce or eliminate unnecessary indexes on files. These can have a major impact on bandwidth use. Inserting, updating, or deleting a record in a database file results in one or more pages needing to be replicated. In addition, at least one other page will likely be hit for each index on the file. While index use will improve application performance locally, the negative impact on bandwidth and performance in a clustered storage environment may outweigh the benefits. This is an important issue, but further discussion of it is beyond the scope of this article. There are books and tools available that can help you to optimize index usage.
Estimating Bandwidth Requirements
How should you estimate the bandwidth requirements for clustered data storage? The best approach is to set up an environment that simulates your production environment and use IBM performance tools and/or network bandwidth tools to measure the results.
It might be feasible in your environment to set up a local Geographic Mirroring test between partitions using a virtual LAN. If so, you can simulate application activity using data generators or a logical replication product that replicates data changes from your production journal to your test IASP environment. By using a dedicated virtual LAN connection, it is easy to use IBM i's performance collection tools to measure the traffic over the virtual LAN (ETMBTR field from QAPMETH). This provides a meaningful bandwidth estimate for any of the XSM technologies because all methods replicate based on IBM i page size (4K). This technique does not help to estimate peak bandwidth or application performance implications as it is performed on a local, unconstrained network.
Other techniques are possible, but they might not be very accurate. It is especially difficult to forecast real-world loads when replicating system ASPs. It is easier to estimate a database-driven load, which is often the most significant contributor to bandwidth requirements when using IASP replication. Some of the factors that must be considered when making these estimates include the following:
•· Journal receivers: If your applications are journaled, you can record the size of all journal receivers generated over a period of time. This provides a good estimate of the bandwidth required for traditional logical replication environments. This value then has to be adjusted because measurements indicate that the bandwidth requirements for XSM replication are two to three times higher than logical replication requirements. Remember, unlike logical replication, which only needs to send data changes (estimated by journal receiver size), all pages that are changed as a result of both the object update and the journal update must be sent in an XSM environment. Note that the above assumes complete journal entries are deposited in the journal (MINENTDTA(*NONE)). If the journal uses minimized entry data, the estimate for XSM bandwidth requirements that you obtain by looking at the journal receiver size will be low and, therefore, even more inaccurate. Note: IBM highly recommends that customers adopting XSM-based solutions also use journaling to improve the recoverability of their applications following an unplanned system outage.
•· Database statistics: DB2 provides statistics via the SYSTABLESTATS table, including counts of insert, update, and delete operations. Transaction volumes can be measured by capturing these values before and after a specific time period and comparing the two.
•· Indexes: Estimate the number of indexes hit per update to forecast the amount of bandwidth that will be required as a result of indexing activity. A couple of indexes per file probably won't make a big difference, but if the average number of indexes is more than that, increase your estimate at least 25 percent. Each hit to an index may cause an index page or a journal page to be written, depending on whether user-controlled or implicit (SMAPP) access path journaling is done.
•· Peak bandwidth for synchronous environments: If bandwidth is constrained, an accurate estimate of peak bandwidth requirements is critical if application performance is to be maintained within acceptable bounds at all times. Methods for accurately forecasting this value are beyond the scope of this article, but it is highly recommend that you simulate your environment using the simulation suggested above or using the IBM benchmark center, to test the potential application impacts.
Unless you can afford to set up an accurate simulated production environment, guesswork will play a part in forecasting clustered storage replication bandwidth requirements, but with the techniques mentioned above, it can be well-informed guesswork.
Consider Clustered Storage, But...
Clustered storage has a role to play in ensuring high availability, with the greatest benefits coming in IASP application environments. However, when using clustered storage to implement a multiple-site high availability solution, the associated networking considerations--including bandwidth, performance, recovery times, recovery points, and costs--are non-trivial. Therefore, before implementing clustered storage in such an environment, take the time necessary to ensure that you understand and address the networking requirements.
More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.
TRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now! Request Now.
Forms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.
Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:
Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.
IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn:
LATEST COMMENTS
MC Press Online