How to Measure AI Readiness and Progress

A closer look at the Trusted AI Index  

Recent research by Teradata and NewtonX, a leading business-to-business market research firm, revealed that 70% of C-suite execs say their AI strategy is not fully aligned with their business strategy. 

Other survey highlights: 

• 89% of executives believe AI is critical for competitiveness 
• Only 56% report having a clear AI strategy 
• 40% do not believe their company’s data is ready to achieve accurate outcomes 

In my last article, I emphasized the importance of trust in AI models, and defined Trusted AI and its core principles. 

But is it possible for an organization to measure how ready they are for AI—or how much progress they’ve made? As a data and analytics company, we decided to reverse engineer the research results and find out. 

Trusted AI framework: Measuring AI readiness 

Before measuring AI readiness, it helps to define what AI success looks like. Teradata’s three principles of Trusted AI provide a useful framework. 

By combining the Trusted AI framework with the research results, we developed a formula to help organizations measure their AI readiness, establish a starting point, identify gaps, and determine what they need to do to improve. 

Principle #1: People 

People are at the core of building trust in AI. The human element helps enhance accountability and security while preventing harmful biases. Sustainability, often overlooked in the generative AI revolution that started in late 2022, is crucial.  

OpenAI used 10,000 GPUs to develop ChatGPT. Now we’re seeing 100,000, 350,000, and 1 million GPU farms being built. These data centers require massive energy for running and cooling, making sustainability a critical pillar for building trust in AI. 

Principle #2: Transparency 

For many organizations, transparency is where the dream stops. Statistics show how difficult it is to move AI models from proof of concept (POC) into production. Progress on AI initiatives depends on transparency, which is the leading factor for not getting AI capabilities into production. Common challenges include governance and security concerns, change management, and lack of stakeholder trust and buy-in. 

Principle #3: Value creation 

AI capabilities must create value for organizations and their customers. This means net benefits—cost reductions, revenue increases, efficiency enhancements, and more—must exceed the cost of using AI.  

Teradata’s Trusted AI core principles and sub-principles are comprehensive yet elegantly simple, with three core principles and three sub-principles each. 

Teradata’s core principles of Trusted AI

Trusted AI Index: Measuring transparency and explainability 

We distilled these principles into the Trusted AI Index, a 9-question quiz that helps organizations identify where they are in their AI innovation journey, so they can accelerate time-to-value and trust for AI initiatives. 

The quiz asks one question for each sub-principle and uses a detailed rubric to score an organization’s position.  

For example: 

“How transparent and explainable are the decisions made by your organization's AI models?” 

The rubric shows what the different levels of transparency look like. 

• Minimal transparency/Explainability: AI/ML models lack transparency and provide little to no insight into how decisions are made. Limited documentation or communication about the processes used to make AI/ML models more understandable. No model versioning capabilities. 
• Limited transparency/Explainability: Some efforts to improve transparency, but they’re insufficient. Basic documentation or explanations regarding model decisions, but not comprehensive. Techniques for enhancing explainability may be implemented inconsistently or inadequately. Little to no model versioning capabilities. 
• Moderate transparency/Explainability: Moderate efforts toward improving transparency and explainability. Clear attempts to document model decisions and provide some level of insight into the decision-making process. Techniques for enhancing explainability are implemented to a certain extent, but there is room for improvement. Limited model versioning capabilities. 
• High transparency/Explainability: Robust practices to ensure transparency and explainability of AI/ML models. Comprehensive documentation and communication regarding model decisions and the reasoning behind them. Advanced techniques and tools to enhance explainability, even for complex or 'black box' models. Moderate model versioning capabilities. 
• Exceptional transparency/Explainability: Transparency and explainability are top priorities. The company goes above and beyond industry standards to ensure AI/ML models are transparent and understandable. Active contribution to research and development efforts aimed at improving transparency and explainability across the field. Moderate model versioning capabilities. 

At the end of the quiz, the organization receives a blended score with a relative position, a results summary, and next steps to move towards becoming a leading, AI-driven organization. 

Curious about your organization’s AI readiness? Take our 9-question quiz to see your score. 

Related resource link:

• Trusted AI in the C-Suite: Highlights From the Global Executive Survey 
• Enterprise 2030: Building the AI-powered company of the future 
• Leadership and decision-making in the era of generative AI