Even if the AI may provide insightful suggestions, it still generates wrong recommendations. It lacks the domain-specific knowledge that is necessary to safely interface with and modify business-critical legacy systems that may be decades old.
By Joydip Kanjilal
So, it is not advisable to completely reject or fully trust AI. In this article, we will look closely at which situations AI can be helpful, which situations it won’t be, and how application developers on the IBM i can effectively make use of AI.
Figure 1 below illustrates where AI code assistants help, where they struggle, and the governance practices that keep both safe on IBM i.
Figure 1: Where AI code assistants help, where they struggle, and what keeps them safe
AI Code Assistants in Enterprise Development
AI code assistants are tools that help enhance the efficiency of software developers by helping them generate and analyze source code. In recent times, the use of AI code assistants is on the rise. According to Gartner, 75% of software engineers working in enterprise applications will use AI Code Assistants by the year 2028.
Until now, general-purpose programming assistants, such as GitHub Copilot, ChatGPT, and Claude, haven’t been successful — they have had trouble working with RPG, since RPG is specific to IBM i and has hardly any publicly available source code for the model to learn from.
Additionally, RPG’s fixed-format programming structure is column-based, which differs from the coding structures the AI coding assistants were trained on, making it harder for the AI models to learn to code in RPG.
IBM Project Bob: A Purpose-Built Example
IBM Bob is worth mentioning here because it serves as a valuable real-world example for many concepts discussed in this article. It was created specifically for the IBM i platform, which includes the IBM i stack—RPG, CL, COBOL, and Db2 for i— and comes with support for agentic workflows, built-in security, and enterprise-grade deployment flexibility. Essentially, IBM Bob is an AI coding agent that can enhance productivity, accelerate modernization, and automate testing in cloud, on-premises, or hybrid environments.
Where AI Performs Well
In this section, we will look at how AI code assistants can provide value to IBM i applications.
Code Documentation and Explanation
AI assistants can help with code documentation and code explanations, especially for the IBM i platform. If you give your AI-powered code assistant a legacy RPG program that doesn’t have any comments, it can provide you with a fairly accurate first draft describing how the program works, what files are accessed, and what subroutines are invoked by the program, thereby saving you a significant amount of time.
Test Scaffolding
AI-powered code assistants are also good at generating the boilerplate code you need when creating an RPG unit test, and generating sample input records or even a list of assertions that you will likely want to validate. Although AI-based code assistants cannot determine which business rules need to be tested, they can help you write the code for your test, thereby saving a significant amount of your time.
Where AI Struggles
In this section, we will look at how AI code assistants struggle to add value.
Business Logic Not Written Down
Typically, IBM i programs contain many lines of code spread across isolated modules, with little to no proper documentation. Now, since the business logic is not well documented, it often becomes difficult for your AI-based code to determine a specific business rule when it reads the source code.
Consider the following example. Suppose we have a rule in a process stating that whenever a customer is on a credit hold, they cannot receive any discounts. However, this rule may not be well documented, and you'll only see a CHAIN and an IF statement without any comments.
Legacy Fixed-Format RPG
AI-based models trained on large, unstructured collections of text often fail to accurately determine column positions in a structured programming language such as RPG. Fixed-format RPG is often underrepresented in the data used to train the AI models. In general, fixed-format RPG uses a column-oriented syntax, in contrast to the approaches most models are trained on, which use free-format syntax. The greatest problem is not the use of column-position syntax-related errors, because the compiler catches them as a syntax error. Even though the compiler can report that the use of the indicator is correct from a syntax viewpoint, it cannot determine whether the indicator was used under the proper conditions. Thus, we can create an AI application that can compile well but doesn't apply the indicator correctly.
Domain-specific tools can change this perspective. Project Bob was specifically developed with RPG, CL, COBOL, and Db2 for i, using the workflow context of IBM i, such as library lists, program call chains, and Db2 for i semantics. It is not that it is difficult to integrate AI with RPG; instead, it is more appropriate to state that general-purpose AI tools will always have severe limitations in dealing with undocumented business logic and legacy applications.
Regulated Environments
To implement a code change in a regulated environment such as a bank, there should be a ticket, a reviewer, and an audit trail. Hence, your code change process must have proper approval and audit procedures in place to ensure no gaps in your change management. However, the AI-based code assistant does not have any built-in concept of your SOX, PCI DSS, or HIPAA regulatory obligations.
Understanding the Risks
Hallucinations
The fact that an AI-based tool can write source code compliant with a programming language's guidelines does not mean the final product will yield favorable results in practice after compilation.
To counteract the hallucinations that AI code generation can produce, there are two important ways to reduce the likelihood of hallucinations: implementing retrieval-augmented generation (RAG) and Static Analysis.
Refactoring Issues
Refactoring code may appear to function properly when compiled and run. Still, it may be changing some internal steps, such as rounding, null value processing, and lock bounds, which may not be immediately apparent.
Security Issues
In addition to the advantages that the use of AI brings, it also introduces multiple security risks, especially in information security. If you use corrupted or biased data to train your model, it will produce false-positive identifications or output errors.
Therefore, the model must use only data that has not been tampered with or altered in any way. Another critical security consideration is confidentiality. You must have procedures in place to ensure that only the customer has access to their personal data.
Governance Models for Safe AI-Assisted Development
No matter where the code came from, the governance model shouldn’t change. Hence, the governance model will remain the same — you will need to apply these same processes (i.e., code review, regression testing, change-management process, audit trail, and compliance) irrespective of whether it came from IBM Project Bob, a general-purpose assistant, or a human contractor.
To implement AI governance, IBM i shops need to create the following:
- AI Governance model: By creating shared contextual files and guidelines for AI governance, teams can ensure that AI-generated output aligns with how the team works and creates products.
- AI Prompt Standards: The creation of a common set of standard prompts for completing documentation, explaining SQL statements, and refactoring requests will help reduce instances of developers using different types of prompts, thereby reducing the potential for intent drift among developers.
- Code Review Criteria: Ensure that the team's code review criteria, before it goes into production, include architectural verification to determine whether the written code meets all security and data flow requirements defined for IBM i systems.
- Code Health Metrics: Ensure that code health metrics include monitoring of duplicate code, the frequency of refactoring, and module churn; do not measure only the speed at which code is merged.
A Practical Adoption Checklist for IBM i Teams
Before unleashing an AI code assistant on RPG or SQL, work through this list:
- First off, you should confirm to what extent the AI code assistant may access data and source code.
- You should also verify that the AI code assistant can be deployed in a manner that meets your data storage and compliance requirements.
- Establish trust in the AI code assistant before its use for generating or refactoring code; therefore, provide documentation and explain how the documentation relates to the task before going ahead and generating or refactoring any code.
- You’ll need to review any AI-recommended code changes manually before deploying them to production.
- When reviewing the SQL that was generated, do not assume that any SQL statement that looks like standard SQL is valid in an IBM Db2 for i environment; verify that each statement complies with Db2 for i requirements.
- Verify that any commitment control, error handling, or restart logic is retained after an AI-assisted refactoring using a test case created through regression testing rather than visually reviewing the code and results.
- For any programs that you intend to modify using an AI-assisted refactor, keep or build regression tests so that the regression tests — rather than visual inspection of the refactored code — will identify any semantic drift.
- You should also log all AI-assisted code changes in addition to the non-AI-assisted code changes for later reference during audit and incident investigations.
- Lastly, you should periodically review the checklist itself since AI-assisted or AI-powered tools are continuously improving by the day.
A Practical Example: AI-based Code Refactoring
In this section, we'll examine a short example of an AI-generated SQL refactoring code example that can replace a legacy SQL code listing. Consider the following piece of code:
exec sql DECLARE prod_cursor CURSOR FOR
SELECT product_no, qty_on_hand, reorder_pt
FROM product WHERE prod_status = 'A';
exec sql OPEN prod_cursor;
exec sql FETCH prod_cursor
INTO :product_no, :qty_on_hand, :reorder_pt;
dow sqlcode = 0;
if qty_on_hand <= reorder_pt;
exec sql UPDATE product SET reorder_flag = 'Y'
WHERE product_no = :product_no;
endif;
exec sql FETCH prod_cursor
INTO :product_no, :qty_on_hand, :reorder_pt;
enddo;
exec sql CLOSE prod_cursor;
In the preceding code snippet, we open a cursor over active products and check whether the quantity on hand is below the reorder level for each product.
The following code snippet shows the AI-optimized version.
UPDATE product
SET reorder_flag = 'Y'
WHERE prod_status = 'A'
AND qty_on_hand <= reorder_pt;
In the previous code example, optimizing the codebase and using an AI-enabled code assistant have greatly reduced the amount of code required, replacing a separate update statement for each qualifying product row with a single update statement. However, AI optimization also changes how row locks are held and how lock timeouts are handled when using the two approaches to update products.
For example, in the original approach, each row that is updated is locked for the duration of the execution of the individual update statement. In contrast, in the AI-optimized approach, a single update statement would use a single row-level lock across all updated rows during execution.
Additionally, if the original loop encounters a lock timeout while it executes, any rows it has already updated will remain updated. However, if a lock timeout occurs during the AI-optimized single-update statement, the entire transaction will roll back, and no rows will remain updated.
However, the use of AI did not correct a previously identified issue in the original code, where the reorder flag column is set to 'Y' for each eligible product and never reverts to 'N' if the product is replenished above the reorder level.
Additional IBM i Concerns
Here’s a quick look at how the two versions of the SQL code differ:
Commitment control
As the loop commits row by row, this ensures the durability of all flags from the beginning of the job, regardless of the job's commitment level. The rewrite in the AI-optimized version carries out its process as one single unit of work since any changes made would not have any durable effect.
Journaling
Both versions of the code sample produce the same journal entries, but the former generates a journal entry for each row changed, while the latter creates all entries in bulk.
Transaction scope
Each transaction scope in the loop runs one transaction at a time, while the transaction scope in the AI-optimized version is carried out for all transactions that meet the conditions specified.
Lock duration
In the first version, locking and releasing one transaction is done immediately, but the AI-optimized version needs to hold locks through the entire set of transactions.
Isolation level
Because the loop works with one transaction, all it needs to ensure, as far as isolation is concerned, is a simple isolation level.
In the AI-optimized version, however, many rows get read-locked for the duration of the entire SQL statement, even if no changes occur, thus requiring a stricter isolation level.
The Future
IBM has been investing in Project Bob as a dedicated assistant for the platform, and that investment itself is a marker of where this thing is going. The intention was never to replace RPG developers — the platform has no surplus of them, and the business logic coded into decades worth of production code is not something we can truly expect a model to simply figure out for us.
In the future, experienced IBM i professionals will be able to deliver faster with the help of a tool that reads and writes alongside them, but the judgment of what a program is actually supposed to do — and responsibility for shipping it — remains precisely where it always was: with the person who understands the platform.
Key Takeaways
- AI-based code assistants can read and comprehend, and even explain, the business logic of your legacy RPG programs quickly.
- The best and most useful ways to exploit AI today are documentation, code explanation, and test scaffolding.
- Complex business logic embedded in fixed-format RPG source code, combined with government-regulated environments, presents some of the biggest challenges to the use of AI-assisted tools.
- The most significant risk of using AI-assisted code is that, although it may look correct, it could contain major issues in the business logic.
- You can reduce the risks associated with using AI-powered tools through appropriate and established governance policies that include scoped access, manual review, and regression testing.
- The use of AI will not eliminate the need for proper RPG programming skills training.
Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.
IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn:
LATEST COMMENTS
MC Press Online