It's time for a review of the basic concepts of MI locks. These examples will help.

In computer science, a lock is a synchronization mechanism for enforcing limits on access to a resource in an environment where there are many threads of execution. In IBM i, locks are implemented in such a complex and sophisticated manner that they cannot be compared directly with their counterparts in common platforms, such as semaphores, critical sections, read/write locks, and mutexes.

The design of IBM i locks allows OS applications and user programs to achieve fine-grained control of resources and escape some common problems with lock implementations. At the MI level, some of the complex and versatile locking support is arranged and exposed to user programs in the form of a bunch of lock management MI instructions. This article reviews the basic concepts of MI locks by looking at some easy-to-follow examples and experiments.

In this article, I refer to an IBM i job (or an MI process at the MI level) simply as process; the term lock requester means a process or a thread; and the term lock target means an MI object lock or a space location to lock.

More examples of using MI lock instructions written in ILE RPG can be found here.

Types of MI Locks

IBM allows user programs to acquire or release the following types of MI locks:

• MI object locks—MI instructions to acquire or release an MI object lock are Lock Object (LOCK) and Unlock Object (UNLOCK), respectively.
• Space location locks—MI instructions to acquire or release an MI space location lock are Lock Space Location (LOCKSL) and Unlock Space Location (UNLOCKSL), respectively.

The most significant difference between these two types of MI locks is that object locks are at the granularity of an MI object, while space location locks are at the granularity of each single byte in the associated space of an MI object.

(Note: The Lock Teraspace Storage Location (LOCKTSL) and Unlock Teraspace Storage Location (UNLCKTSL) can be used to lock/unlock either a Teraspace location or a Single Level Store (SLS) space location. For reasons of simplicity, these instructions and Teraspace location locks are not covered in this article.)

Here's an explanation of the common lock allocation procedure of MI locks:

• A single lock-acquisition instruction (LOCK or LOCKSL) can request the allocation of one or more lock states on one or more lock targets (MI objects or space locations). Locks are allocated sequentially until all locks requested are allocated.
• When two or more threads are competing for a conflicting lock allocation on a lock target, the machine attempts to first satisfy the lock allocation request of the thread with the highest priority. Within that priority, the machine attempts to satisfy the request that has been waiting longest.
• If any exception is identified during the instruction's execution, any locks already granted by the instruction are released, and the lock request is canceled.
• For each object lock, counts are kept by lock state and by thread, process, or transaction control structure. When a lock request is granted, the appropriate lock count of each lock state specified is incremented by 1. For each space location lock, counts are kept by lock state and by thread. If a transfer of an object lock from another thread causes a previously unsatisfied lock request to become satisfied, the lock request and the transfer lock are treated independently relative to lock accounting. The appropriate lock counts are incremented for both the lock request and the transfer lock function.

Lock States

Unlike common lock mechanisms, such as mutexes or read/write locks in which lock states are very limited, MI locks can request one or more of all the following five lock states on the lock targets:

• LSRD—The lock target can be shared with another lock requester if the user does not request exclusive use of the lock target. That is, another user can request a LSRO, LSUP, LEAR, or LSRD lock state.
• LSRO—The lock target can be shared with another lock requester if the lock requester requests either a LSRO lock state or a LSRD lock state. This lock state is appropriate when a user does not intend to change a lock target but wants to ensure that no other user changes the lock target.
• LSUP—The lock target can be shared either for update or read with another lock requester. That is, another user can request either a LSRD lock state or a LSUP lock state for the same lock target. This lock state is appropriate when a user intends to change a lock target but wants to allow other users to read or change the same lock target.
• LEAR—The lock target is allocated to the lock requester that requests it, but other lock requesters can read the lock target. This lock is appropriate when a user wants to prevent other users from performing any operation other than a read.
• LENR—The lock target is reserved for the exclusive use of the requesting lock requester; no other lock requesters can use the lock target. However, if the lock target is already allocated to another lock requester, you cannot get exclusive use of the lock target. This lock state is appropriate when a user does not want any other user to have access to the lock target until the function being performed is complete.

The following table shows the valid lock state combinations for an object.

The design of the multiple lock states is important. First, this design can help lower the performance overhead due to lock contention, which is a common problem with usual lock mechanisms. For example, if a lock requester requests only LSRD on a lock target, no lock contention will occur between it and other lock requesters who request a LSRO, LSUP, or LEAR lock on the same lock target. Second, this design can help user programs and OS applications avoid deadlocks, which are more common in other platforms, where most lock mechanisms available are mutually exclusive locks.

It's interesting that, except for the Lock Management MI instructions, the only interfaces IBM provides to user programs to acquire or release an MI lock are CL commands: the Allocate Object (ALCOBJ) command and the Deallocate Object (DLCOBJ) command. Users can use them to acquire or release object locks conveniently either interactively or in programs. These commands use a set of special values to indicate different lock states. For convenience, I list the mapping of MI lock state names and lock state special values used by ALCOBJ and DLCOBJ in the table below.

To obtain the mapping table shown above, you can follow a simple experiment like this:

1. Select an MI object as your lock target (say, a program object called *LIBL/MAPMAP).

2. Obtain the 8-byte SLS virtual address of MAPMAP by dumping it (via the Dump Object (DMPOBJ) command or the Dump System Object (DMPSYSOBJ)) or find it in the SST via object name and MI object type/subtype code. Let's say the address of MAPMAP is hex 060A6CEBB4000000.

3. Acquire a specific lock state on MAPMAP using the ALCOBJ command. For example, acquire a LENR lock by ALCOBJ ((MAPMAP *PGM *EXCL)).

4. Examine the lock record via the advanced analysis macro LOCKINFO of the System Service Tool (SST)—for example, LOCKINFO 060A6CEBB4000000. Check the MI lock state name in the output of macro LOCKINFO, and you will discover that *EXCL means MI lock state LENR. An example output is the following:

5. Release the acquired lock on MAPMAP via the DLCOBJ command. For example, DLCOBJ ((MAPMAP *PGM *EXCL)).

6. Repeat steps 1 through step 5 for each of other lock states.

Lock Request Types

With a single LOCK or LOCKSL instruction, a lock requester can request one or more lock states on one or more MI objects or space locations. An MI lock request can be one of the following types:

• Immediate request—If all locks cannot be immediately granted, a Lock Request Not Grantable (hex 1A02) exception (aka MCH2602) will be raised.
• Synchronous request—Wait until all locks can be granted for a specified time interval or infinitely.
• Asynchronous request (available only for MI object locks)—Allow processing to continue and signal event when the object is available.

For an asynchronous object lock request, if the asynchronous lock request is satisfied, then the Object Locked (hex 000A,01,01) event is signaled to the requesting thread. If the request is not satisfied in the specified time interval, the Asynchronous Lock Wait Timeout (hex 000A,04,01) event is signaled to the requesting thread. No locks are granted, and the lock request is canceled. If an object is destroyed while a thread has a pending request to lock the object, the Object Destroyed (hex 000A,02,01) event is signaled to the waiting thread. These events are signaled to the requesting thread, regardless of the scope of the requested lock. If the invocation issuing the asynchronous object lock request is terminated, the lock request remains active.

Let's experiment with different lock request tests using the following example program, lock01.rpgle.

For example, you may start two interactive jobs: JOB#A and JOB#B. Type the following commands in JOB#A and JOB#B.

The Work with Object Locks (WRKOBJLCK) command is helpful when checking current lock status of a given object (including acquired lock states or lock states being requested synchronously or asynchronously). For example, when JOB#B requests an asynchronous LENR lock on *PGM LOCK01, the output of command WRKOBJLCK LOCK01 *PGM might like this:

Lock Scopes

In common platforms, locks are usually scoped to thread, while MI locks may be scoped to three different lock scopes. Space location locks acquired by LOCKSL can only be scoped to thread; object locks and space location locks acquired by LOCKTSL can be scoped to a thread, a process, or a transaction control structure. Locks scoped to a thread can never conflict with a lock scoped to its containing process, but may conflict with a lock scoped to a different process, a transaction control structure, or any other thread (depending on the lock states involved). Locks scoped to a transaction control structure attached to the current thread may conflict with a lock scoped to a different transaction control structure, a process, or any other thread (depending on the lock states involved).

What's a Transaction Control Structure (TCS) object? It's an MI object type whose MI object type code is hex 23. You can find out the sub-object types of TCS via the lsobjtypes QShell utility provided by the i5/OS Programmer's Toolkit.

Note: Common job-level or activation-group level commitment definitions are also represented by the 23A1 TCS object since the introduction of the TCS MI object.

For detailed information about XA support provided by IBM i, please refer to documentation on XA APIs in the IBM i Information Center.

A nice article written by Jarek Miszczyk in 2007 is also available at MC Press Online: "Bridge the Legacy-to-Java Transition with DB2 for i5/OS Distributed Transactions."

Allocated process scope locks are released when the process terminates. Allocated thread scope locks are released when the thread terminates. If a thread requested a process scope lock, the process will continue to hold that lock after termination of the requesting thread. If a thread requested a transaction control structure scope lock, the transaction control structure will continue to hold that lock after the termination of the requesting thread. If a thread is terminated while waiting for a lock with a lock request type of either synchronous or asynchronous, the lock request is canceled regardless of the scope of the requested lock.

Authority Required to Acquire MI Object Locks

To acquire a space location lock via a valid space pointer, the lock requester need not have any authority to the MI object that the space pointer addresses. But to acquire an object lock, the condition is somewhat different:

• First, the lock requester needs Execute authority to the contexts (libraries) referenced for address resolution.
• Second, the lock requester needs some authority to the target object.

The document of the ALCOBJ command says: The user issuing the command must have object operational (*OBJOPR) authority to the object.

The document of the LOCK instruction in the information center says: Authorization required to the object to be locked: Some authority or ownership.

When an MI object is created via CL commands or APIs, full authorities to the object are always granted to its owner user profile. Then, what does some authority mean? If you are really curious about it, try a simple experiment like the following:

• Say program object I#PGM is owned by user USER#A, and its public authority is set to *EXCLUDE.
• Say another user, USER#B, has no source of authority to obtain any authority to program I#PGM (e.g., via authorization lists or primary group (PGP)).
• Grant any one of the object authorities or data authorities to USER#B on program I#PGM by the Grant Object Authority (GRTOBJAUT) command or the Edit Object Authority (EDTOBJAUT) command. Object authorities to an object are *OBJEXIST, *OBJMGT, *OBJOPR, *OBJALTER, *OBJREF, and *AUTL; data authorities to an object are *READ, *ADD, *DLT, *UPD, and *EXECUTE. For example, GRTOBJAUT OBJ(*LIBL/I#PGM) OBJTYPE(*PGM) USER(USER#B) AUT(*READ).
• Finally, try to lock I#PGM in an interactive job under user profile USER#B. For example, ALCOBJ ((*LIBL/I#PGM *PGM *EXCL)).

Now, you should have the answer!

Advisory Locks or Mandatory Locks?

In common platforms, most locks are advisory locks, where each thread cooperates by acquiring the lock before accessing the corresponding data. However, some platforms also implement mandatory locks, where attempts of unauthorized access to a locked resource will force an exception in the entity attempting to make the access. So, are MI locks advisory locks or mandatory locks?

Space location locks are advisory locks. No matter what states of lock are being allocated by a thread on a space location, another thread can do whatever it wants to the space location: changing data content at the space location, extending or truncating the space object, and even destroying the space object.

Unlike space location locks, in a sense, MI object locks are mandatory locks. Locks are not required in order for an object to be referred to (or modified by) an instruction. However, a process or thread is not allowed to use an object if any other process or other thread in the same process holds a conflicting lock. This is referred to as Lock Enforcement Rules in MI documentation. For example, when a LSRO lock on the owner user profile of a permanent space object or the context the space object resides in is held by another thread, a thread issuing a Destroy Space (DESS) MI instruction on the space object will cause an Invalid Lock State (hex 1A01) exception (aka MCH2601), which means one or more lock enforcement rules are violated when an attempt is made to access an MI object. Another example occurs in the previous section of this article, Lock Request Types, where although JOB#A holds a LENR lock on program object LOCK01, JOB#B can still call it as usual; this is because no lock enforcement rule is enforced on the Call External (CALLX) instruction of the called program object.

For lock enforcement rules enforced on an individual MI instruction, please refer to the Lock Enforcement section in the MI instruction's documentation.

Note that it's a good convention to acquire proper lock states on a shared object before actually accessing it so that the lock enforcement rules can help you to guarantee the integrity of your data and application systems. APIs and CL commands that access objects are a good example of this. For example, when a HLL program opens a physical file for reading, the QSYS/QDMCOPEN API will acquire a LSRO lock on the *FILE object (with MI object type code/sub-type code hex 1901), and the QSYS/QDBOPEN API will acquire a LSRO lock on the data space object (*QDDS, with MI object type code/subtype code hex 0B90) of the physical file.

Object Lock Transferring

As mentioned above, when multiple threads are competing for a conflicting object lock, the system satisfies one of the multiple lock requests according to thread priorities and waiting time of the threads. Besides the system coordinating mechanism, IBM i provides an alternative method to allow a process to transfer an object lock it holds to another process. After the transferring, the latter process becomes the new holder of the object lock. The MI instruction we can use to achieve object lock transferring is Transfer Object Lock (XFRLOCK). The first operand of XFRLOCK is a system pointer to the receiving process's Process Control Space (PCS) object.

Here, I'd like to demonstrate the use of object lock transferring with a simple and interesting experiment. Imagine we have a User Space (*USRSPC) object called BALL, and we want BALL available to only one of a couple of jobs (JOB#A and JOB#B) at any given time. We'll let one of the two jobs, say JOB#A, hold a LENR lock on BALL and then transfer the LENR lock on BALL to JOB#B when JOB#B needs to use BALL. Thus, the LENR lock on BALL is transferred between JOB#A and JOB#B repeatedly, and no other job can have the opportunity to acquire any kind of lock on BALL. Objects involved in this experiment are the following:

• *USRSPC BALL
• *USRSPC PCSPTR—The PCS pointer of the job to receive the LENR lock on BALL is stored at the beginning of PCSPTR's associated space.
• *PGM LOCK03—LOCK03 writes the PCS pointer of the current process to the beginning of the associated space of PCSPTR.
• *PGM LOCK02—LOCK02 transfers the LENR lock on BALL to the process identified by the PCS pointer stored in the associated space of PCSPTR.

Here's the source of OPM MI program LOCK03, lock03.emi:

Here's the source of ILE RPG program LOCK02, lock02.rpgle:

Now, let's start our ball-passing experiment. Use the Work with Object Lock (WRKOBJLCK) command to check which job is currently holding the lock on BALL.

Deadlocks

Deadlocks are a well-known problem with most lock mechanisms, especially where locks are always mutually exclusive locks. Software failures due to deadlocks are hard to detect or reproduce. The design of MI lock states is helpful in avoiding this problem, but the possibility of deadlocks still remains. The following is a simple example. Imagine we have two CL programs: PGM#A and PGM#B.

PGM#A

PGM#B

Submit two jobs like the following:

The result is that either of the two submitted jobs enters lock-waiting status. The object lock statuses of *USRSPC SPCA and SPCB are the following.

SPCA

SPCB

JOB#A is waiting for a LSUP lock on SPCB, which conflicts with the LSRO lock on SPCB held by JOB#B. And JOB#B is waiting for a LSUP lock on SPCA, which conflicts with the LSRO lock on SPCA held by JOB#A. This is a deadlock condition, since neither of the lock requests can be satisfied and neither of the two jobs will have the opportunity to recover from the deadlock.

So, as you might have already guessed, it's a good idea to avoid acquiring unnecessarily strict locks on lock targets (for example, a LENR lock when what you need is only to change the data content of your lock target). Stricter locks are more likely to cause blocking or deadlocks.

Additionally, to avoid deadlocks on a group of logically related lock targets, it will be helpful to standardize the lock acquisition sequences so that locks on the lock targets are acquired and released in the same order for each thread that uses this group of lock targets.

Junlei Li is a programmer from Tianjin, China, with 10 years of experience in software design and programming. Junlei Li began programming under i5/OS (formerly known as AS/400, iSeries) in late 2005. He is familiar with most programming languages available on i5/OS—from special-purpose languages such as OPM/ILE RPG to CL to general-purpose languages such as C, C++, Java; from strong-typed languages to script languages such as QShell and REXX. One of his favorite programming languages on i5/OS is machine interface (MI) instructions, through which one can discover some of the internal behaviors of i5/OS and some of the highlights of i5/OS in terms of operating system design.

 

Junlei Li's Web site is http://i5toolkit.sourceforge.net/, where his open-source project i5/OS Programmer's Toolkit (https://sourceforge.net/projects/i5toolkit/) is documented.

Also Read

Programming in ILE RPG - Defining Data Structures, Part I

H and F Position-less Specs

Practical RPG: AI and IBM i