AI will never reach its potential unless fundamental security problems are addressed more effectively by AI app producers and AI system users.

By John Ghrist

While there’s been much written and said about the need for "guardrails" for AI systems, most concern currently centers on the possibility of AI systems becoming independent bad actors. This concern is justified, but to an extent, it is crowding out from some people’s awareness the fact that AI systems already have serious downsides when it comes to simply guaranteeing the safety and confidentiality of AI apps and their data. Data security for the integrity of AI apps and databases is necessary to protect sensitive enterprise data, prevent the misuse of AI systems to launch malware attacks on other systems, ensure compliance with regulations in regulated industries, and avoid damage to AI models.

Dangers for a Whole New Software Species

AI systems face environmental threats in any publicly accessible network space that are analogous to those faced by nearly any life form in our planet's history that has mutated enough to be called a new species. AI is vulnerable to some security problems that have long interfered with computer systems. Still, it also has its own set of unique concerns that older non-AI apps running on servers may never encounter. There are numerous examples.

"Poisoning" is a prime one. Corrupted data has been the bane of many IT managers since the vacuum-tube days, but AI systems have two kinds of corruption unique to them: poisoned data and poisoned models. Poisoned data occurs when an intruder corrupts an AI training dataset with incorrect entries, creating false data points that skew an AI's learning process. Poisoned models are damaged when an intruder alters their parameters, causing the AI to draw false conclusions — sometimes only under specific conditions, which makes the poisoning hard to spot. Alternatively, poisoned models can be rigged to pass along confidential data points from a training dataset.

Data tampering of other types is also possible. If an intruder can access training data inputs, they can manipulate them in ways that bypass current security efforts to detect tampering while degrading the AI system's ability to reach valid conclusions. Another type of data interference can occur when an attacker analyzes patterns in AI outputs to infer confidential data indirectly. Most antivirus programs today are busy looking for external software that infects the system rather than inspecting data inputs, so this tampering can go undetected more easily.

Computer security sleuths are already familiar with API attacks that try to gain access to systems for intruders. AI systems have their own types of API attacks that can give outsiders access to input data and model content. Hardware can cause communication problems in any network where software systems are sending data to each other, but sometimes, in handling communications between AI systems as well, specific hardware designs can bypass intended software protections.

Sometimes, AIs are trained using prefabricated models. Adversarial data can create a fault in the base model that can introduce a bias — or even a backdoor for later tampering. For example, in a Membership Interference attack, outsiders can dissect the output of an AI to understand some of the training data. This can tell the intruder if a specific piece of data was used to train the AI, which could constitute a data breach if that data was confidential.

AIs also seem particularly useful for enhancing social engineering attacks, which manipulate human psychology to motivate computer users to grant unauthorized access to systems, apps, and sensitive data. False messages and AI-driven videos can sometimes persuade workers to reveal inside information to outside intruders.

Taken together, these and other kinds of attacks are referred to as "adversarial AI," a blanket term for techniques that compromise or manipulate AI systems. These are already an ongoing and likely permanent threat to enterprises using or planning to use AI for legitimate business purposes.

How Bad It Is Already

The European Union Agency for Cybersecurity (ENISA) released a report in November 2025 analyzing overall cybersecurity threats. Phishing (at 60 percent) and vulnerability exploitation (at 21.3 percent) were identified as the two most common intrusion access points. While not all of these intrusions were directed at AI systems specifically, the exploitation of vulnerabilities that subtly interfere with models and the data inputs to training sets is becoming a growth industry.

The report also points out how Large Language Models (LLMs) are being used by intruders to optimize phishing and social engineering attacks. In fact, the study found that more than 80 percent of the social engineering instances reviewed as background for the report were AI-driven. This trend is dangerous enough already and seems likely to expand as the use of AI systems becomes more common.

Because of their nature, attacks on AI systems don’t simply cause the common damage types to enterprises, such as loss of profit and customer trust; they also represent possible financial and regulatory compliance failures, particularly violations of data-protection laws such as the EU’s General Data Protection Regulation (GDPR) or the US’s Health Insurance Portability and Accountability Act (HIPAA).

ATLAS Isn’t Shrugging Off AI Threats

Because adversarial AI is keyed to intrinsic Machine Learning (ML) algorithm limitations, security personnel can’t use many standard security remedies that exploit standard hardware or software vulnerabilities. The Adversarial Threat Landscape for Artificial-Intelligence Systems (ATLAS) is a framework and knowledge base developed jointly by MITRE and Microsoft as a response to this hydra-headed threat. Developed as a collaborative effort involving academic and industry research groups, including IBM, ATLAS offers ATT@CK>, an interactive matrix of case studies that illustrate types and methods of adversarial attacks to help security experts address some of the threats mentioned earlier, as well as others.

The publicly available framework highlights the various types of adversarial attacks to which AI systems are vulnerable and is continually updated as new techniques are discovered. ATLAS is also at the center of a collaborative effort to bring together experts from academia, industry, and government to build a shared strategy to defend against adversarial attacks and develop a set of standardized AI security methods.

As of last February, IBM is offering the 2026 X-Force Threat Intelligence Index, an intelligence-sharing service about security threats and a collaboration platform for IT and security teams to consult experts and peers about effective countermeasures. The service’s inaugural announcement noted a 49 percent increase in ransomware attacks in 2025, a quadrupling since 2020 in compromising attacks on large supply software systems and software-as-a-service (SaaS) environments, and that vulnerability exploitation in 2025 caused 40 percent of reported incidents. Also, unlike the ENISA report, which focused solely on Europe, X-Force observed that North America was the global region suffering the most attacks.

Threats are Greater to Agentic and Production AI

Agentic AI and chatbots are both threatened by attacks, but AIs that rely on agents face a greater risk, if for no other reason than they are more versatile and feature-laden. Chatbots can answer simple questions by relying on scripted responses or rules that help them produce relatively simple responses to unscripted questions. Agentic AI, on the other hand, can perform multistep tasks, reason on its own, learn from user behavior, and is designed to personalize responses more than chatbots can. There are more openings to exploit and features to compromise with agentic AIs.

Similarly, production AI apps are at more risk than experimental AIs, largely because of their basic nature. Experimental AIs are more likely to be used to test the feasibility of solving a particular problem with AI and, therefore, tend to reside in a closed environment. Malicious tampering with an experimental AI is harder to accomplish and less productive because it touches fewer resources. Agentic AIs, when deployed, usually reside in production environments that demand stability, have access to production resources such as databases, and are expected to perform reliably despite interference from heavy communication traffic in the wider environment, unpredictable inputs, and changing business priorities over time. That also means they may have — or be given — power to pollute data and interfere with real-world business activities.

AI shouldn’t be allowed to touch production data except under strict controls. For example, on IBM Power Systems, according to the Programmers.io blog, *OBJMGT and *OBJEXIST authorities should be limited to a data object’s owner to reduce the risk of the authority being misused or accidentally deleted. *OBJALTER authorizes the user to “add, clear, reorganize, and initialize database file members and to alter the attributes of database files.” It also enables changes to the attributes of tables, files, and SQL packages, which no AI intruder should be able to make. In addition, *OBJREF lets a user specify a database file or table as the parent file in a referential constraint. When users shouldn’t have that ability, *OBJMGT, *OBJALTER, and *OBJREF authority must all be restricted. *OBJMGT and *OBJEXIST authorities should be limited to the object’s owner so the authority can’t be deleted or used improperly. Finally, the *EXECUTE authority will only allow access to an object if the user also has authority over both the object’s resident library and directory.

A Word About AI Slop

“AI Slop” is a term for AI-produced verbiage that looks coherent but either doesn’t contain much real content or may also contain outright errors and significant omissions. While the value of such is clearly limited in nearly any informational context, from an AI security perspective, it also risks introducing flawed or incomplete information into what should be trusted data.  Faulty data that’s used to support conclusions that justify business actions, regardless of where it came from, is basically doing an intruder’s job without requiring an actual intruder. Spreading such information downstream increases unreliability of data pipelines, threatens organizational reputation, and degrades models, among other risks.

Enterprises should avoid these by not allowing workers to use Generative AI tools without governance or an understanding of those tools' limitations, requiring validation of all synthetic datasets before using them in production, and insisting that AI-produced code or documentation validate code outputs or specify the documentation's usage.

If workers are to use AI to generate documentation, for example, be as specific as possible about the nature of the documentation the AI is being asked to produce, what prompts it should present to users, necessary goals and details to be included, and never use generated text without some human oversight.

AI Security Safety is an Attitude

Protecting the security of AI systems is easiest to achieve when the enterprise makes security awareness part of its culture, uses effective technical means to address various security weaknesses, and trains employees to use software tools thoughtfully rather than treating them as an end in themselves. Current security awareness programs need to be established if they're not already in place, and if they are, they must be expanded to address the growing body of threats to AI systems.

John Ghrist has been a journalist, programmer, and systems manager in the computer industry since 1982. He has covered the market for IBM i servers and their predecessor platforms for more than a quarter century and has attended more than 25 COMMON conferences. A former editor-in-chief with Defense Computing and a senior editor with SystemiNEWS, John has written and edited hundreds of articles and blogs for more than a dozen print and electronic publications. You can reach him at This email address is being protected from spambots. You need JavaScript enabled to view it..