I recently had the opportunity to do an interview with John Ghrist about the “RPG Programmer Dilemma.” The article from that interview was posted here last month.

Today, I’m going to follow up with some thoughts and information about our investment in the upcoming RPG Code Assistant tool, which represents the latest investment IBM is making to help address the difficulty many IBM i clients face in finding the skills to maintain their existing applications.

In May of 2024, I stood on stage at the COMMON POWERUp conference and discussed the need for a code assistant that specializes in the RPG programming language. Of course, the IT industry is well-acquainted with AI-driven code assistants, which can help programmers develop code much more quickly than traditional software development tools. However, in the case of the IBM i community, one of the key values that a code assistant can deliver is transferring knowledge from developers who wrote code a long time ago to those tasked with maintaining, debugging, and enhancing it years later. That knowledge, however, is buried in code, which can be very difficult for new developers to understand, particularly if it's written in a programming language or a version of that language that is unknown to them.

At that conference and in numerous webcasts, articles, interviews, and podcasts since then, I have explained what the RPG code assistant project aims to deliver. Today, I’ll explain what we’re doing, give some insight into how it’s being done, and shine a light on what’s next.

“Explain” is First

Code assistants can have many different functions. For almost a year before POWERUp 2024, I had spent time speaking with our advisory councils and several IBM Champions whose expertise lies in RPG programming and code modernization. Though many of the most flashy capabilities of code assistance are the first things people look for -- such as being able to generate a piece of code given just a few words of description in a natural language -- the more I talked to our clients and advisors, the more apparent it was that the most significant requirement from our customer base is assisting developers who need to take ownership of existing code without the benefit of the knowledge of the programmers who created that code.

It is for this reason that the initial focus of the RPG code assistant is the ability to explain the meaning and function of existing RPG code. As a programmer looks at a piece of code that they didn't develop or that they developed years ago, the code assistant should have the ability to generate an explanation of what that code is doing. This explanation should be in a natural language that the programmer can understand, such as English.

That capability to explain RPG in English may sound simple, but many technical hurdles need to be jumped to reach that finish line. First, as I said at the conference and then in articles that followed, the RPG programming language doesn't have nearly as much code available in the open as other programming languages for which existing assistants are commonly used. Having access to a large training set is critical to any large language model (LLM), no matter what questions you expect the LLM to answer or what functions you expect the LLM to perform. For this reason, we began our project by asking our community to submit code that we could use to train a large language model to understand RPG in the way that IBM i customers are using it.

We have had an amazing response to this request. As I write this, we have leveraged over 100,000 lines of contributed RPG code to train our Granite-based LLM. This supplement material includes our IBM RPG manuals, our IBM RPG-related Redbooks, multiple MC Press Books, such as Programming in ILE-RPG by Jim Buck, and texts written by some of the preeminent authors on RPG and other IBM i concepts. And we are not done. Customers continue to submit new material for us to use in training our large language model.

Again, saying that our code assistant will be able to explain RPG code might sound simple, but we have worked with many RPG programmers in our community to collaborate on the kinds of explanations that they would most value while they are working with a piece of RPG code. Certainly, everyone will expect to be able to highlight a piece of code in their IDE and ask the code assistant to explain it. But what other kinds of explanations would be useful to a client?

Programmers have told us that sometimes they just need a summary of a subroutine. Other times, they would like an explanation of how a particular program or procedure gets used – how is it called? The explanations they want to see could be most beneficial if they are summaries at times, and at other times, if they are written as if an experienced programmer is explaining the details of the code to a junior programmer. Finally, many existing older RPG programs were written with insufficient comments. An explanation might be a new comment inserted to explain to the existing programmer and future programmers what the code is doing.

We can already see that several flavors of explanation would be beneficial. But programmers aren't the only people who can gain value from the explanation of code. We’ve also received input from people who manage programmers or who manage IT shops. Many of them want the ability to have very high-level descriptions of the business rules that their applications implement. So, another form of explanation that a code assistant might deliver is a summary of business rules. Will our assistant be able to do that? In part, it depends on how much training material we get from businesses, but so far, we are hopeful. Our RPG code assistant is attempting to address all of those core needs, and we are certain we will be asked for even more flavors of explanation as we get more developers using the tool.

Context & Prompt Engineering - Gathering the right information for the LLM

When you're asking any large language model to do something, the quality of the response is greatly affected by the amount of information that you give the LLM when you make the request. This is extremely important for an RPG code assistant because if you're going to ask for an explanation of a piece of code, that explanation can be made better if you send a great deal of information about the code to the LLM. This “information about the code” is known in LLM circles as “context.” And asking the LLM to do something, such as providing a code explanation, is done by building what's called a “prompt.” In this case, a simple version of a prompt might look like “Explain the following RPG code.” The explanation will be more accurate if, in addition to that prompt and the code you want explained, you could send a significant amount of context – that is, information about the surrounding software, its data structures, its data definition through SQL, and so on.

Some of the most demanding work being done as we develop the RPG code assistant involves determining which context information should be sent and how to phrase the prompt that we send to the LLM. This has been an exciting part of developing a code assistant! Our developers have never had to learn how to prompt before, and this is the first time they’ve had to figure out what surrounding information makes the most effective context. Essentially, what they're doing is figuring out how their own minds operate as they, as humans, work to understand a piece of code, then send that information to the LLM and see if it learns from that same information.

In the world of artificial intelligence and LLMs, the term for what the developers are learning to do is “prompt engineering.” We continue to learn the best practices for prompt engineering from talking to people who have done it for other code assistants, as well as through trial and error. You see, we need to determine which aspects of RPG programs are required for context and prompting when those pieces of information might not have existed, or might not have been as important for other programming languages.

Just the beginning

Our first delivery of the RPG code assistant will be exceptionally valuable. This has been validated many times by discussions, interviews, and demonstrations with our early advisors.

Clearly, however, there are more functions that our customers would like to see from an RPG code assistant. For example, at POWERUp in May of 2024, I expressed our desire to enable our tool to generate unit test software for existing RPG code. That is clearly one of the items that we have on our road map. Similarly, a code assistant is expected to assist a programmer as they write code. And so our LLM and our tool must be trained to generate RPG as a developer is working to create new code. Certainly, it will be interesting to be able to turn English words into RPG. For example, it will be fun as a programmer to be able to type “Please create a subroutine written in free-format ILE RPG that generates a summary report for the employees in the table represented by this SQL” and see the resulting RPG and SQL get written by the assistant.

However, one of the clear pieces of advice our developer experts have given us is that a more commonly used generation would be for the assistant to anticipate what a programmer might want to code next and suggest that code, so the developer has a framework that they can use to complete whatever task they're coding. Think of it as an “autocomplete suggestion” in a text editor or e-mail client, but for code.

All three of the examples I've just given are forms of the general function “generate.” Our road map contains generate capabilities, and it is likely that the follow-on functions, which will be delivered soonest, will be generation capabilities.

Clearly, the capability to explain, followed by the capability to generate, will help many of our clients address the needs they have to be able to equip the developers they hire with the tools necessary to work with and enhance existing RPG programs.

Our goal, however, is not only to address the skills gap but also to help our clients modernize their existing RPG code, adding new business capabilities that address business requirements with modern technology. The future of modernizing IBM i applications, utilizing AI and LLM capabilities, begins with a code assistant, but there are opportunities that extend beyond that. Who knows what we may accomplish next?

For those of you who can join me at the 2025 COMMON POWERUp conference or the 2025 COMMON Europe Congress, you will have the opportunity to learn more about this project. We are on the cusp of some exciting announcements! And if you happen to be at the IBM THINK conference this week, come find me! Between the IBM i 7.6 announcement and this topic, we’ll have lots to discuss!

This is an exciting time to be working in the IT industry, in general, and in the IBM i community, specifically.

Steve Will is the Chief Technology Officer and Chief Architect for IBM i at IBM, as well as an IBM Distinguished Engineer. Since joining IBM in 1985, he has played a pivotal role in evolving the IBM i platform, leading its modernization through technologies like cloud, AI, and VS Code integration. Appointed Chief Architect in 2007, he succeeded Frank Soltis and has been a key advocate for the platform’s future. Known for his leadership, technical vision, and community engagement, Will actively shares insights through blogs, conferences, and social media.