Agentic Artificial Intelligence: The Challenges of Autonomy and Security in Digital Systems

Artificial intelligence is undergoing a significant transformation in the way it interacts with digital systems. During the early years of widespread language model adoption, most applications focused on producing text, interpreting documents, analyzing information, and generating responses. However, the development of agentic artificial intelligence expands these possibilities by allowing computing systems not only to respond to requests, but also to coordinate tools, execute operations, and monitor the results of their own actions. This evolution offers an opportunity to transform organizational processes, but it also introduces challenges related to security, operational autonomy, and human control.

The fundamental difference lies in the shift from generating information to carrying out activities. A conventional assistant can explain how to perform an operation in a particular system. An artificial intelligence agent, when integrated with the necessary tools and properly authorized, can perform that operation, verify its outcome, and determine what additional steps are needed to complete the task. This capability changes the relationship between the user and the technology, as some operational decisions may be made automatically, without requiring a new human instruction at every stage.

Agentic artificial intelligence, also known as agentic AI, does not necessarily represent a new category of artificial intelligence model. In many implementations, it is an architecture that combines language models, planning mechanisms, digital tools, memory capabilities, and orchestration components. The model interprets the objective it receives and the available information, while the other components enable its decisions to be translated into concrete actions. Depending on the architecture, the agent may query databases, use programming interfaces, process documents, execute functions, and interact with external applications.

Imagine an organization that needs to analyze administrative documents, identify inconsistencies, and prepare reports. In a conventional application, different stages may require a professional to review files, conduct searches, organize information, and consolidate results. In an agentic solution, some of these operations can be coordinated automatically. An agent can locate authorized documents, extract data, compare information, and prepare a preliminary report for human validation. If it finds an inconsistency, it may consult another permitted source or refer the situation for specialized analysis.

The potential of this approach lies in its ability to integrate different activities into a single process. However, its use requires a distinction between conventional automation and agentic autonomy. In traditional systems, the execution sequence is usually determined by predefined rules and workflows. In agentic systems, some actions may be selected dynamically, taking into account the objective received, the available context, and intermediate results. This flexibility makes it possible to handle less predictable situations, but it also requires mechanisms capable of controlling decisions that have not been specified in advance in every detail.

It is important to clarify that operational autonomy does not mean absolute independence, consciousness, or free will. An agent operates within a technological architecture, using models, instructions, tools, and resources defined during its design and configuration. Its autonomy refers to its ability to select and perform certain actions without continuous human intervention. The extent of that autonomy depends on the permissions granted, the tools available, and the controls implemented. A system may have advanced planning capabilities while still being restricted to retrieving information, without authorization to modify it.

This distinction is essential to understanding the security challenges. In an application designed exclusively to generate responses, an error may produce incorrect information, inadequate analysis, or imprecise recommendations. When an agent has access to operational systems, databases, or external services, the potential effects are different. An inappropriate decision may lead to changes in records, the transmission of information, the execution of unplanned operations, or access to resources that should not be part of the authorized activity.

The problem does not necessarily lie in a flaw in the artificial intelligence model. In many cases, the risks arise from the way the model is integrated with other components. A tool may expose an excessively broad range of operations. A programming interface may have insufficient authorization controls. An execution mechanism may allow decisions to be acted upon without appropriate validation. The security of an agentic solution therefore depends on the architecture as a whole, rather than solely on the quality or capabilities of the model used.

One of the fundamental principles of this architecture is to clearly separate the functional objective from operational permissions. The objective describes the expected outcome. Permissions establish which resources may be used to achieve it. Business rules define the conditions that must be respected, while oversight mechanisms determine when an operation may proceed automatically and when it requires human intervention.

This separation prevents the technical ability to perform a particular action from being confused with authorization to do so. An agent tasked with retrieving financial information may have access to specific reports and records, but not necessarily permission to modify amounts or make payments. Similarly, a system designed to prepare administrative documents may be authorized to generate proposals and complete forms without having permission to approve or submit them in final form.

This principle is related to an established concept in information security: least privilege. Under this approach, users, processes, and applications should receive only the permissions necessary to carry out their responsibilities. Applied to agentic artificial intelligence, it means that each agent must operate within a defined set of capabilities. File access, program execution, database queries, and administrative operations must be authorized according to the specific purpose of each activity.

Implementing this principle requires particular attention to identity management. In digital systems, authentication and authorization are distinct mechanisms. Authentication seeks to verify the identity of whoever or whatever is accessing a particular resource. Authorization establishes which operations that identity may perform. Depending on the solution, an agent may use its own credentials or receive permissions delegated by a user. In either case, the system must ensure that the delegation does not exceed the rights granted and that operations can be associated with an identifiable identity.

This concern becomes even more important when agents act on behalf of people. Authorizing a system to perform a particular task does not mean granting it unrestricted freedom to interact with all available information. Delegation must be tied to an objective, a set of resources, and execution conditions. Ideally, permissions should also take into account the duration of use, the sensitivity of the data, and the need for approval for operations with greater impact.

Application programming interfaces, commonly known as APIs, play a central role in this process. An API allows different software components to exchange information and request operations. Artificial intelligence agents can use these interfaces to retrieve records, create documents, update information, or invoke services. However, every request must be subject to the authentication, authorization, and validation controls established by the responsible application.

The fact that an operation is technically accessible does not mean that it is authorized under every circumstance. A service may allow a particular user to retrieve their own records, but it must prevent access to information belonging to third parties without appropriate authorization. This verification must take place in the components responsible for processing requests. Restrictions present only in the visual interface do not provide sufficient protection, as different applications and agents can interact directly with the available services.

Another important challenge concerns the way language models interpret external information. Agentic systems frequently process documents, pages, messages, API responses, and content from different sources. These materials may contain data needed to perform the task, but they may also include inappropriate instructions or manipulated information. The agent must distinguish content used as information from instructions it is actually authorized to follow.

This situation is related to the problem known as prompt injection, or the injection of instructions. It occurs when less trustworthy content attempts to improperly influence the behavior of an artificial intelligence system. A document consulted by an agent, for example, may contain instructions intended to change its behavior, disregard rules, or send information to an unauthorized destination. The fact that these instructions appear in material processed by the model does not make them legitimate.

Protection against this problem should not depend exclusively on instructions written in the prompt. Security instructions are important, but they need to be supplemented by technical controls that are independent of the model. Restricted permissions, validation of operations, separation between data and instructions, verification of communication destinations, and approval mechanisms help limit the effects of inappropriate content. The objective is to prevent information received during a task from improperly changing the execution conditions established by the organization.

Data security also requires a specific approach. An agent may access personal information, confidential documents, or corporate records while performing its activities. This information must be protected according to its classification and intended use. The system must limit access to necessary data, control its sharing, and prevent sensitive content from being improperly included in execution logs, messages, or responses intended for other services.

In more complex architectures, protection must also take into account temporary information storage and memory mechanisms. Some agents retain intermediate results, operational preferences, or information related to previous tasks. These capabilities can improve continuity, but they require care regarding data retention, updating, isolation, and deletion. Incorrect, outdated, or manipulated information may influence subsequent decisions if it is reused without appropriate validation.

Another relevant measure is the isolation of execution environments, often referred to as sandboxing. A sandbox is a controlled environment in which programs or operations can run under specific restrictions. In agentic solutions, this mechanism can limit access to the operating system, the network, files, and other resources. Its purpose is to reduce the consequences of unexpected operations and prevent experimental activities from having improper effects on sensitive environments.

However, isolation should not be regarded as an absolute guarantee of security. Protection depends on its configuration, the permissions granted, and the interactions permitted with external components. A robust architecture therefore needs to combine different defense mechanisms. Network controls, identity management, credential protection, environment segmentation, and monitoring must work together, reducing reliance on a single technological barrier.

Complexity increases when different agents collaborate within the same process. In multi-agent architectures, a system may distribute activities among specialized components. One agent may research information, another may analyze results, and a third may prepare an operation. This structure allows responsibilities to be separated and more elaborate processes to be developed, but it also requires clear mechanisms for communication, coordination, and control.

Collaboration between agents should not imply unrestricted sharing of permissions. Each component must have an operational identity and a set of authorizations appropriate to its responsibility. Information received from other agents must be handled with regard to its origin, reliability, and classification. An incorrect decision at one stage should not automatically propagate to the other components without appropriate validation mechanisms.

The distribution of activities also requires attention to traceability. When several agents participate in the execution of a process, it becomes necessary to identify which components performed particular operations, what information was used, and what results were produced. Without these records, investigating problems can become complex, particularly when decisions are made in sequence and involve different systems.

Traceability is therefore an important requirement of agentic architecture. An organization must be able to reconstruct the main stages of an execution, identify the tools used, verify the changes made, and understand when human interventions occurred. These records must comply with privacy and information protection requirements, while providing sufficient evidence for audits, compliance assessments, and incident investigations.

However, monitoring cannot rely exclusively on reviewing records after the fact. Depending on its configuration, an agent may perform several operations within a short period. If an inappropriate activity is identified only after it has been completed, its consequences may already have occurred. For this reason, solutions that perform sensitive operations need to consider mechanisms for monitoring execution as it happens, operational limits, and capabilities to suspend actions when certain conditions are detected.

In this context, human oversight continues to play a fundamental role. The concept of human-in-the-loop describes an approach in which certain decisions or operations require human validation. This does not mean that every stage of a process must be carried out manually. Oversight can be proportionate to risk, allowing greater autonomy for low-impact activities while requiring approval for operations that may have significant consequences.

An application responsible for organizing documents, for example, may classify files and prepare preliminary reports automatically. However, deleting important records, changing official information, or carrying out financial operations may require specific confirmation. These conditions should be defined with regard to business rules, the sensitivity of the information, and the consequences of possible errors.

This approach must also distinguish between preparing an action and executing it in final form. An agent may have permission to draft a proposal, but not necessarily to approve it. It may identify an inconsistency without being authorized to modify the corresponding record. It may recommend an operation without receiving permission to carry it out. This separation makes it possible to benefit from artificial intelligence without indiscriminately transferring decision-making responsibilities to automated systems.

The ability to stop a task is another essential element. An agent should be evaluated not only on its ability to complete activities, but also on its ability to recognize situations in which it cannot appropriately proceed. Insufficient information, missing permissions, inconsistent results, or the need to access unauthorized resources may justify a controlled interruption. In these circumstances, the expected behavior may be to request guidance, record the difficulty, or present alternatives that are compatible with the established rules.

The possibility of reversing operations must also be considered. In digital systems, certain actions can modify records, trigger communications, or have consequences for other processes. Not all of them can be undone through a simple opposite operation. The architecture must therefore identify reversible activities, operations that require confirmation, and situations in which recovery or compensation mechanisms are needed.

These precautions demonstrate that the security of artificial intelligence agents cannot be treated merely as a question of model configuration. It depends on decisions related to functional architecture, integration architecture, software development, data protection, and organizational governance. Each of these disciplines helps establish boundaries and responsibilities that must work together as an integrated whole.

Functional architecture plays a particularly important role in this context. Before determining which tools the agent will use, it is necessary to understand the process to be automated. This involves identifying its objectives, actors, information, business rules, exceptions, critical decisions, and completion conditions. Without this understanding, there is a risk of building a solution that is technically capable of performing activities but insufficiently aligned with the organization's needs and responsibilities.

Technical and integration architecture translates these definitions into operational mechanisms. It establishes how the agent communicates with applications, how it uses tools, which services may be accessed, and how restrictions will be enforced. Information security, in turn, helps define controls for identity, authorization, credential protection, isolation, and monitoring. Governance establishes responsibilities, acceptance criteria, and oversight mechanisms throughout the solution's life cycle.

The responsible adoption of agentic artificial intelligence also requires evaluations before deployment to production. Traditional functional tests remain necessary, but they are not sufficient to examine every characteristic of these systems. It is important to observe how the agent responds to incomplete information, unexpected results, unavailable tools, ambiguous requests, and attempts to induce unauthorized behavior. Evaluations must consider both the quality of the results and the compliance of the actions performed.

A prudent strategy is to begin using agents in clearly defined processes, with restricted tools and permissions, and gradually expand autonomy as the results are evaluated. This approach makes it possible to understand the system's behavior, identify difficulties, and improve controls before extending its scope. Greater autonomy should be supported by evidence of performance, security, and the ability to maintain oversight, rather than merely by demonstrations of new functionality.

Oversight must continue even after deployment. Changes to integrated applications, new tools, model updates, and changes to business rules may alter the behavior of an agentic solution. A system initially suited to a particular context may require new controls when its scope is expanded. Change management, operational monitoring, and periodic permission reviews therefore become essential parts of its operation.

There is also an important dimension related to accountability. Using artificial intelligence agents does not eliminate the need to identify who is responsible for designing, authorizing, deploying, and overseeing the systems. The specific distribution of these responsibilities depends on the organization, contractual relationships, and applicable law. From a technology governance perspective, however, it is necessary to establish clearly who defines the objectives, approves permissions, monitors behavior, and determines the conditions for interrupting or reviewing the solution.

This responsibility must be considered from the beginning of development. Security should not be merely a final check performed before the system is made available. It should guide the definition of requirements, the selection of tools, the design of workflows, exception handling, and oversight mechanisms. In agentic systems, architectural decisions directly determine the extent of autonomy and the possible consequences of the actions performed.

It is important to recognize that many of these challenges did not originate with artificial intelligence. Issues related to authentication, authorization, systems integration, data protection, and traceability have been part of software engineering for decades. What distinguishes agentic AI is the combination of these issues with systems capable of dynamically selecting actions and interacting with different tools to achieve objectives. This characteristic increases the flexibility of automation, but it also requires a corresponding evolution in control mechanisms.

Security in this context should not be understood as an obstacle to innovation. A well-defined architecture allows agents to operate more predictably, respect responsibilities, and be used more reliably. Appropriate controls also make it easier to expand solutions gradually, as they put organizations in a better position to understand and manage the risks associated with operational autonomy.

Agentic artificial intelligence thus represents a significant change in the way we design digital systems. The focus is no longer exclusively on the ability to interpret information and generate responses; it also encompasses the coordinated execution of activities. This evolution may increase productivity, integrate processes, and enable new forms of interaction among people, applications, and services. However, its responsible development depends on the ability to establish clear and verifiable boundaries for automated actions.

The real challenge is not simply to build agents capable of performing increasingly complex tasks. It is to ensure that these systems understand the scope of activities in operational terms, use only authorized resources, respect business rules, and remain subject to effective oversight mechanisms. The ability to achieve an outcome should not be confused with the legitimacy of the means used to obtain it.

The future of agentic artificial intelligence will depend not only on the evolution of models and their execution capabilities, but also on the maturity of the architectures that support their use. The greater the autonomy granted to systems, the more rigorous the definition of permissions, responsibilities, and control mechanisms must be. The true measure of an intelligent solution lies not only in what it can do, but also in the security, reliability, and accountability with which it performs its actions.