Why Most Agentic Architectures Will Fail
Agentic artificial intelligence is expected to have a major impact because it can execute complex tasks autonomously. For now, the hype is outstripping successful implementations, and there are a lot of reasons for that. “In 2024, AI agents have become a marketing buzzword for many vendors. However, for user organizations, agents have been an area of early curiosity and experimentation, with actual implementations being far and few,” says Leslie Joseph, principal analyst at Forrester. “We expect this to change in 2025 as the technology and the ecosystem mature. However, our prediction offers a cautionary note.” Joseph says organizations attempting to build AI agents are failing for three main reasons: a poorly scoped vision for agentic workflows, a poor technical solution, and a lack of focus on change management. “A poorly scoped vision for agentic workflows results in either a too broad or narrow bounding box for agent functionality,” says Joseph. “Too narrow a scope may render the problem as solvable by a deterministic workflow, while too broad a problem might introduce too much variability. Agent builders should ask themselves how best to define the business problem they are trying to solve, and where an AI agent fits into this scope.” Related:AI’s on Duty, But I’m the One Staying Late Second, it’s early days. Agents are still very early-stage applications, and the ecosystem, including agentic tooling, is less evolved than one might expect. “While many vendors message around the ease-of-use and drag-drop nature of their agent builder platforms, the fact is that there is still a lot of engineering needed under the hood to deliver a robust enterprise solution, which requires strong technical skills,” says Joseph. Finally, a lack of focus on change management isn’t helping. Organizations need to understand how the agentic workflow fits into or enhances existing processes and being proactive about managing change. “The invention of LLMs was like the discovery of the brick,” says Joseph. “With agents, we are now figuring out how to put these bricks together to construct homes and cities and skyscrapers. Every enterprise will need to identify what their desired level of autonomy is, and how to build towards that using AI agents.” Leslie Joseph, Forrester He expects the short-term benefits to be process improvement and productivity, but over the longer term, enterprises should be ready for agents to create disruptions across the tech stack. For now, companies should embrace AI agents and agentic workflows, given its disruptive potential. Related:How Do Companies Know if They Overspend on AI and Then Recover? “Start investing in experiments and allocating budgets towards proofs-of-concept. Ensure that your teams learn along the way rather than outsourcing everything to an ISV or tech vendor, because these learnings will be crucial down the road,” says Joseph. Multi-Agent Workflows Are Challenging When establishing a multi-agent workflow, there are three primary challenges businesses face, according to Murali Swaminathan, CTO at software company Freshworks. First, it’s incredibly difficult to make workflows predictable in a world that is unstructured and conversational. Second, even complex reasoning in workflows can be prescriptive and hard to achieve reliably. Third, continuous evaluation of these workflows is necessary to measure, and ultimately realize efficacy. “[E]nterprises must establish clear approaches on what workflows or problems they want the agentic systems to solve,” says Swaminathan. “Additionally, it’s critical that they develop a clear plan on how they will gauge success. This approach will ensure that expectations are measured, and that a strategy of ‘progress over perfection’ is employed.” Over the short term, enterprises will most likely achieve task-based goals related to the employee and agent. Over the long term, business benefits should follow, along with insights about what the business should and should not do. Related:Addressing the Security Risks of AI in the Cloud “[C]reate a clear game plan on how to implement, utilize, and measure the success of agentic architectures,” says Swaminathan. “Failing to plan is planning to fail.” Insufficient Infrastructure and Data Governance When it comes to agentic architectures, infrastructure and data governance matter greatly. “Without the right infrastructure and data governance in place, agentic architectures struggle to handle the complexity, scale, and interoperability needed for successful implementation,” says Doug Gilbert, CIO and chief digital officer at experience-driven digital transformation partner Sutherland Global. “Companies should focus on building a strong digital core that can handle the high demands of AI, from data processing to seamless integration with hybrid or multi-cloud environments. This not only allows organizations to scale AI capabilities efficiently but also ensures the flexibility to adapt as systems evolve.” Equally important is a well-defined data strategy. Whether leveraging a hybrid, private, or multi-cloud approach, secure and accessible data is essential for building robust AI solutions, ensuring compliance and security across the board. Interconnectivity Matters Interacting with other systems designed for humans is much harder for agentic AI to do than it seems. “Making RPA [Robotic Process Automation] nearly 100% reliable took 12-plus years. And that’s carefully hard coded to interact with human operated systems across the web and Windows. So, we see these people suggesting that they can get an LLM to do the same and it turns out [to be] quite unreliable,” says Kevin Surace, chairman and CTO at autonomous testing platform Appvance. “People will be disappointed when the agent thinks it did everything right, but you later find that payment never went out.” Despite the fact humans don’t get everything right, people expect agentic AI outcomes to be 100% accurate. As an accuracy benchmark, Surace suggests setting the accuracy goal as high as RPA or well-trained humans. “Anyone can demo a simple action a few times,” says Surace. “But doing complex tasks with variability a thousand times without failure — then you have a product people want.” Orchestration Can Be Tricky Orchestration involves end-to-end harmonization of outputs from multiple agents, delivering a unified and comprehensive resolution to the user’s query. “A key of the agentic AI architecture is its capability to organize agents logically by functional domains such as IT, HR, engineering, and
Why Most Agentic Architectures Will Fail Read More »