29
Fri, Nov
0 New Articles

Using Bit Fields as Data

General
Typography
  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times

Benefits that go beyond saving disk storage.

by Craig Pelkie

For most system designers, deciding how to store settings for program options is very simple: allocate a byte for each option. This method is simple to define and use, and the design proceeds with no further concern for that data element. But there are times when a more complicated definition would provide benefits beyond the mere storage of data. This more flexible alternative is to store the option as a bit within an option byte.

A project that I am currently working on involved this choice between two methods. I naturally chose the simple method first, in my haste to get on with the work, but was finally forced to reexamine and recode the data. The data that I needed to encode was a series of yes/no responses, representing both system options and user preferences.

Things Change...

The problem with system design (well, mine at least) is that it is impossible to analyze and predict all of the options that might be required or desirable. I started the project with the intention that the "meaningful" decisions in the system could be controlled by the client, rather than hard- coding in my own preferences. Although my clients were pleased that many options were available, they were quick to point out that other parts of the system should also be under their control. Also, each client had his or her own ideas about options that they wanted to control, so the additions were growing with each new client.

Traditionally, these types of requests could be easily satisfied by using part of the "filler" space that is usually included in a file. For the System/36 version of the product, this was no real problem, since I had included filler in the files to begin with. For the S/38 and AS/400 versions, the solution was a bit more complicated, since I don't include fillers, but add to the file as needed. This necessitated recreating the file, recompiling all of the programs using the files, and remembering to update the client version of the files. After this happened several times, I desperately began to wish for an easier way to accommodate the requests for additional options.

The solution that I found was to encode the options as bit fields. This involves more programming than the method of encoding one byte per option, and there are some other tradeoffs that are made. Overall, the method has worked well and lets me freely add options without any of the problems I described. I'll tell you the mechanics of the technique and review some of the concerns with you.

A Single Character

A "bit field" is really nothing more than a one-byte character field. The difference is that programs using this one-byte field treat it not as a byte, but more like a miniature array of eight bits. Each bit within the field can have a value of 0 or 1, and you can assign whatever meaning you want to those values. Usually, you would think of those as being "yes" or "no" responses. For example, you might want to allow your users the option of answering a confirm message before updating a file. I also use the values for other "binary" situations: for example, 0 indicating a monochrome monitor, 1 indicating color (which I suppose is really another type of yes/no setting).

The point is, a single bit can represent two settings. In some cases, you might need more than two settings. You can still use bit settings. For example, using two bits allows you to represent four choices (with settings of 00, 01, 10 and 11). Three bits allow up to eight settings; the number of choices equals 2 to the Nth power, where N is the number of bits you're using. On the other hand, there are instances where it is easier to simply define a separate option field and store the setting as a separate byte. I don't try to force everything into a bit field, especially when it's obvious that it's not the best solution.

Getting Started

The first decision you'll have to make when using a bit field is what the purposes of the assignments that you're making are. Each bit field contains eight bits, numbered (from left to right) 0 to 7. Why the numbering within a high level language starts at 0 is beyond me, but it's important to know that the compiler expects us to abide by that convention. You must decide on the meaning of each bit. For example, you might decide that bit 0 means that the user wants to have the cursor set to blinking or no blink, and bit 1 means they want the alarm to sound or remain silent when they get an error message. The point is, you decide this yes/no stuff before doing anything else, and you'd better write it down in an obvious place, since you'll be referring to it frequently. On the AS/400, I favor including the bit settings as comments within the DDS. For the S/36, I have all of my file definitions in /COPY modules, and include the comments there as well. I think it makes sense to include the comments within the file definition, since the file can be referenced from many programs. If you put the comments in a program, then you must always remember which program includes the comments.

Once you've decided on the meaning for a particular bit setting, commented it, and resolved to never use that position for anything else, you're almost ready to start using it. Naturally, you'll have to first define the bit field itself in your files. This is as simple as defining a one-position character field. In my project, I have more than eight options that I'm setting, so I defined multiple bit fields. In fact, I have even included a few "filler" bit fields that I'm not using yet, so I can avoid the problem of having to redefine the file later. I have included 10 bit fields in my file definition, which allows me to define up to 80 options. Although I don't feel that saving space is as big an issue now as in the past, you can see that I've saved quite a bit of space by using bit fields.

One of the problems that you must be aware of concerns the default initialization of the bit fields. That is, when your program writes your new record to the file, the character fields are set to blanks unless you set them otherwise. "Blank" is x'40', not x'00', meaning that bit 1 of your field is set ON. You should be careful to initialize bit fields, either to the settings that you want or to x'00', before using them.

Setting and Testing

Longtime readers of Midrange Computing are probably familiar at least with the techniques of setting and testing bit fields, since there have been many articles dealing with this subject. The RPG operation codes are not particularly difficult to understand, and there are equivalent operations in the other high-level languages (although I can't think of any technique to use bit-level operations conveniently in CL programs). The RPG opcodes you'll use are BITOF, BITON and TESTB. For this article, I'll point out only what I've used to accomplish my purposes, and avoid discussing some of the exotica of the opcodes, particularly TESTB.

1 shows an example of how you can initialize a bit field and set options within the field. The bit field SYSOPT is first set to x'00'. Various user preferences are then set; for example, cursor blinking is set in bit 0, alarm sounding in bit 1, and confirm messages in bit 2. If you have additional options, you can continue them in bits 3 through 7, or (this was the important part for me) leave bits 3 through 7 "reserved for future use." For me, that was the reason to convert from the "one byte per option" method to using bit fields. In the future, when a client requests another option, I already have five bits available. I can choose any one of those five bits to store the option setting, without altering anything else in the file. Since the byte is already in the file, I don't need to be concerned with adding another option byte to the file, with the attendant headaches of changing the record structure.

Figure 1 shows an example of how you can initialize a bit field and set options within the field. The bit field SYSOPT is first set to x'00'. Various user preferences are then set; for example, cursor blinking is set in bit 0, alarm sounding in bit 1, and confirm messages in bit 2. If you have additional options, you can continue them in bits 3 through 7, or (this was the important part for me) leave bits 3 through 7 "reserved for future use." For me, that was the reason to convert from the "one byte per option" method to using bit fields. In the future, when a client requests another option, I already have five bits available. I can choose any one of those five bits to store the option setting, without altering anything else in the file. Since the byte is already in the file, I don't need to be concerned with adding another option byte to the file, with the attendant headaches of changing the record structure.

Setting options into bit fields isn't much use unless you can later decipher the settings, and 2 shows you the coding you would use to determine an option setting. In this example, we want to determine whether the alarm should sound when we display a message. Since this option was originally stored in bit 1 of the option byte, we test that bit to determine whether it is "on". Using the TESTB opcode with an indicator in the "equals" position sets the indicator on if the bit is on, off if the bit is off.

Setting options into bit fields isn't much use unless you can later decipher the settings, and Figure 2 shows you the coding you would use to determine an option setting. In this example, we want to determine whether the alarm should sound when we display a message. Since this option was originally stored in bit 1 of the option byte, we test that bit to determine whether it is "on". Using the TESTB opcode with an indicator in the "equals" position sets the indicator on if the bit is on, off if the bit is off.

You can review the RPG manual for each of these opcodes, and learn about the additional combinations of settings and indicators that you can use. I have only shown here the essentials of setting and testing bit fields used for options. If this is your first experience with bit fields, I suggest that you stick with simple usages like those shown until you become more familiar with the techniques. On the other hand, this is about as complicated as it gets, so there's really not a lot more to learn at this point.

Pro and Con

Deciding whether or not to use bit fields is not really a point of debate, say, along the order of not using GOTO/TAG in programs. It is more a matter of planning, deliberation and weighing the consequences.

On the "pro" side, I have found that setting up my 10-bit fields in the file has let me define all of my current options, and has left me plenty of room for future needs. I've tended to group similar types of settings into the same byte field; for example, terminal control options into one byte, options for a particular program into another, and so on. This means that each bit field usually has one or two unassigned bit settings, which is exactly what I want. I want it that way so that in the future, when another option needs to be set, I can set it in the byte that makes the most sense.

By using bit fields, I am able to assign a great number of options in a small amount of file space. Although I'm not usually frantic about saving a few bytes in a control or options file, I find it a comfort to be able to have a few option bytes available, already assigned, knowing that I can get eight times the work out of each byte. By making each byte pull more weight, I don't have to worry about going back into the file definition, adding another byte field, then recompiling and updating my clients' files again.

On the other hand, you have probably already seen several potential problems with bit fields. First, you must quite carefully map out the bit assignments. In a single byte option field, this would not be such a concern, but you don't want to mix up your assignments among bits. Second, you should probably take care to initialize all of the bits in the bit field. This usually amounts to setting all of the bits OFF, as shown in 1, then setting bits ON as needed.

On the other hand, you have probably already seen several potential problems with bit fields. First, you must quite carefully map out the bit assignments. In a single byte option field, this would not be such a concern, but you don't want to mix up your assignments among bits. Second, you should probably take care to initialize all of the bits in the bit field. This usually amounts to setting all of the bits OFF, as shown in Figure 1, then setting bits ON as needed.

The programming required to set and test bit fields is also more involved than using single byte fields. For example, rather than just move a "yes" or "no" value into the setting for the "blink cursor" option, you must now use the less obvious BITON opcode. This also can be a problem when you use the setting; for example, without proper documentation, it would be difficult to discern the meaning of the bit test in 2.

The programming required to set and test bit fields is also more involved than using single byte fields. For example, rather than just move a "yes" or "no" value into the setting for the "blink cursor" option, you must now use the less obvious BITON opcode. This also can be a problem when you use the setting; for example, without proper documentation, it would be difficult to discern the meaning of the bit test in Figure 2.

Another problem that you must think about before using bit fields is retrieving the settings, other than in programs. Will you need to test the setting of an option for a sort or a logical file create? As an example, you might think that you could code a gender setting of male/female as a bit setting. But before you do, you should consider whether you would ever use that value outside a program, where there are opcodes to test the setting. Chances are, something like a male/female option should probably be encoded as a separate byte.

So what can, or should, you encode as a bit field? It's hard to say, but in my experience, I have found that many system options and user preferences can be encoded. It is unlikely that those settings will ever need to be referenced outside of the programs that use the settings, so I don't have the sort/logical file concern. It's tough to say that experience will teach you, but I've occasionally had to move a bit setting back to a byte field. However, at this point, the pure bit settings are more obvious, and when I recognize that an option must be a byte field, I don't try to force it into a bit field.

If you haven't seen or used this concept before, you may wish to consider it when you start your next project. I doubt if the technique would lend itself to "retrofitting" into existing code, since the programming involved, as shown in the two figures, can become quite a bother. But for new work, you might decide that the effort required for additional coding is offset by the advantage of a more stable file definition. Readers with experience or additional thoughts on this technique are welcome to share their opinions with us, so that this little known technique might become more widely known.


Using Bit Fields as Data

Figure 1 Initial bit settings

 Figure 1: Initial Bit Settings .20....+...30....+...40....+...50 BITOF'01234567'SYSOPT BLINK IFEQ 'Y' BITON'0' SYSOPT END ALARM IFEQ 'Y' BITON'1' SYSOPT END CONFRM IFEQ 'Y' BITON'2' SYSOPT END 
Using Bit Fields as Data

Figure 2 Testing a bit setting

 Figure 2: Testing a Bit Setting .20....+...30....+...40....+...50....+...60 TESTB'1' SYSOPT 90 
BLOG COMMENTS POWERED BY DISQUS

LATEST COMMENTS

Support MC Press Online

$

Book Reviews

Resource Center

  • SB Profound WC 5536 Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. You can find Part 1 here. In Part 2 of our free Node.js Webinar Series, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Brian will briefly discuss the different tools available, and demonstrate his preferred setup for Node development on IBM i or any platform. Attend this webinar to learn:

  • SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.

  • SB HelpSystems ROBOT Generic IBM announced the E1080 servers using the latest Power10 processor in September 2021. The most powerful processor from IBM to date, Power10 is designed to handle the demands of doing business in today’s high-tech atmosphere, including running cloud applications, supporting big data, and managing AI workloads. But what does Power10 mean for your data center? In this recorded webinar, IBMers Dan Sundt and Dylan Boday join IBM Power Champion Tom Huntington for a discussion on why Power10 technology is the right strategic investment if you run IBM i, AIX, or Linux. In this action-packed hour, Tom will share trends from the IBM i and AIX user communities while Dan and Dylan dive into the tech specs for key hardware, including:

  • Magic MarkTRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now!  Request Now.

  • SB HelpSystems ROBOT GenericForms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.

  • SB HelpSystems ROBOT GenericIT security is a top priority for businesses around the world, but most IBM i pros don’t know where to begin—and most cybersecurity experts don’t know IBM i. In this session, Robin Tatam explores the business impact of lax IBM i security, the top vulnerabilities putting IBM i at risk, and the steps you can take to protect your organization. If you’re looking to avoid unexpected downtime or corrupted data, you don’t want to miss this session.

  • SB HelpSystems ROBOT GenericCan you trust all of your users all of the time? A typical end user receives 16 malicious emails each month, but only 17 percent of these phishing campaigns are reported to IT. Once an attack is underway, most organizations won’t discover the breach until six months later. A staggering amount of damage can occur in that time. Despite these risks, 93 percent of organizations are leaving their IBM i systems vulnerable to cybercrime. In this on-demand webinar, IBM i security experts Robin Tatam and Sandi Moore will reveal:

  • FORTRA Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:

  • FORTRAManaging messages on your IBM i can be more than a full-time job if you have to do it manually. Messages need a response and resources must be monitored—often over multiple systems and across platforms. How can you be sure you won’t miss important system events? Automate your message center with the Robot Message Management Solution. Key features include:

  • FORTRAThe thought of printing, distributing, and storing iSeries reports manually may reduce you to tears. Paper and labor costs associated with report generation can spiral out of control. Mountains of paper threaten to swamp your files. Robot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing. Manage your reports with the Robot Report Management Solution. Key features include:

  • FORTRAFor over 30 years, Robot has been a leader in systems management for IBM i. With batch job creation and scheduling at its core, the Robot Job Scheduling Solution reduces the opportunity for human error and helps you maintain service levels, automating even the biggest, most complex runbooks. Manage your job schedule with the Robot Job Scheduling Solution. Key features include:

  • LANSA Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.

  • LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed. Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

  • LANSASupply Chain is becoming increasingly complex and unpredictable. From raw materials for manufacturing to food supply chains, the journey from source to production to delivery to consumers is marred with inefficiencies, manual processes, shortages, recalls, counterfeits, and scandals. In this webinar, we discuss how:

  • The MC Resource Centers bring you the widest selection of white papers, trial software, and on-demand webcasts for you to choose from. >> Review the list of White Papers, Trial Software or On-Demand Webcast at the MC Press Resource Center. >> Add the items to yru Cart and complet he checkout process and submit

  • Profound Logic Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.

  • SB Profound WC 5536Join us for this hour-long webcast that will explore:

  • Fortra IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn: