As AI models become more capable and organizations increase their investments in agentic AI, the bottleneck for ROI is shifting from pilot quality to a need for enterprise-wide data readiness. A new report from autonomous knowledge platform Teradata that surveyed 1,000 senior technology and data leaders shows that although 90% of respondents expect to increase agentic AI investments over the next 12 months, only 37% report strong positive returns to date.
The report reveals that data fragmentation is a primary roadblock for organizations trying to move from agentic AI pilots to enterprise-wide autonomous knowledge that delivers measurable returns. But beyond connecting data alone, businesses must add specific context layers to allow agentic AI to reason and take effective action.
Why is data fragmentation such a roadblock?
Every enterprise has mountains of data. Some do a better job than others at organizing that data. But even in the best cases, that data is typically built for humans, with the assumption that they will be the ones to make sense of its context and meaning in order to effectively act on it.
Now, enterprises need to contextualize that data for AI agents. The report reveals significant roadblocks for organizations making this shift:
- 77% of executives report that 20% or less of their enterprise data is sufficiently described and contextualized for agents to use
- 78% say it’s challenging to unify data and knowledge across business functions so agents can reason across the full enterprise
- Among specific barriers, leaders rank data that’s lacking the necessary metadata, context, and relationships (43%) nearly equal to the data itself being fragmented across systems (42%)
The roadmap to agentic AI maturity
The report uses an Agentic AI Maturity Index to identify how much progress organizations have made with agentic AI, which roadblocks appear when, and the impact each roadblock has on delivering enterprise-wide returns.
Companies in the Experimenting stage (28%) are still exploring localized pilot projects. These pilots, even when successful, typically deliver personal productivity gains only.
In the Developing stage, where 40% of enterprises currently sit, organizations have found success with some models and automations but are limited by data connectivity across teams. These organizations are working to shift from personal to organizational gains and encountering interoperability challenges across their data sets.
Those in the Building stage (25%) have gotten to the point where they have basic structured governance and workflows but have yet to solve the knowledge fragmentation problem. They have more connected data but still lack critical context layers to enable autonomous decision-making at the enterprise level.
Only the 7% of enterprises in the Operationalizing stage have dynamic governance and safety rules alongside the data foundations that are necessary to scale autonomous decision-making across the entire organization. These are the companies starting to see measurable business impact from agentic AI that can execute multistep workflows.
Connected and contextualized data unlocks agentic AI
Businesses in the Operationalizing stage have made the shift to what the report calls autonomous knowledge: data that carries enough context, governance, and meaning for AI agents to act on it reliably and repeatedly. But you don’t have to boil the ocean to make meaningful progress, says Teradata CIO Josh Fecteau, in the report: “The goal of contextualizing your entire data estate is likely the wrong goalpost, and chasing it is part of why organizations stall. Data estates are always changing: new types, new sources, multimodal inputs. You’ll never finish.”
“Instead,” Fecteau suggests, “identify the highest-value portion of your data—structured or unstructured—and focus on getting that portion fully described, governed, and agent-ready. If most of the data is unusable, the answer isn’t to fix all of it at once. It’s to be ruthlessly selective about which 20% to 50% you start with.”
Where to start: Overcoming decision paralysis
Figuring out where to start is its own challenge. The report finds that 60% of leaders experience some decision paralysis on technology decisions due to a desire for long-term flexibility. Specifically, 30% of leaders cite vendor lock-in as an active worry.
The solution to this problem is embedding the necessary context and connecting disparate data layers within the foundational data architecture, to enable reliable autonomous decision-making. Building data into reusable, governed assets that any model can make sense of removes specific software and tooling dependencies. The foundation itself becomes portable, which is what enables a shift to autonomous knowledge. This shift enables organizations to realize enterprise-level ROI from agentic AI, while preserving flexibility in the software layer.
The report summarizes the steps to reach the Operationalizing stage of Agentic AI Maturity:
- Audit and contextualize the data estate, starting with the highest-value datasets. This means applying consistent definitions, lineage, and meaning for data points across the organization, regardless of team or function.
- Embed dynamic governance in the foundational data architecture itself, not the software or application layer. Embedded context allows any model or agent to reason the way a human would and ensures that guardrails transfer and decisions are traceable throughout multistep processes.
- Build for portability. Following the steps above preserves flexibility as both organizations and models evolve. When you connect data and embed context across organizational datasets, enterprises are free to change tooling without disrupting the underlying governance or context layers that enable autonomous agentic decision-making.