When AI works in the lab but struggles at scale

In 2021, Zillow made a decision that became one of the most visible cautionary tales of enterprise machine learning.

The company shut down Zillow Offers, its technology-driven home-buying business, after its forecasting models struggled to reliably predict future home prices in a rapidly changing housing market. The company reported significant losses, and researchers studying the case have since pointed to a combination of model limitations and operational scaling decisions that contributed to the outcome.

But there is another lesson for technology leaders hidden beneath the headlines.

Years before Zillow Offers was discontinued, Zillow's own engineering team had identified an infrastructure challenge: machine learning models placed dramatically different demands on its existing serving architecture. Some models might be called once during an offer request, while others could be called thousands of times per minute. Balancing availability, cost, and the ability to handle sudden increases in demand required the company to rethink how those models were deployed.

The lesson isn't that Zillow failed because it lacked cloud infrastructure. It didn't.

The lesson is more important:

An AI model does not operate independently from the technology and operating environment around it.

‍A model can perform successfully in development. A proof of concept can deliver impressive results. But when an organization attempts to scale AI across departments, locations, applications, and business processes, the demands change.

Suddenly, AI requires reliable access to distributed data, integration with enterprise applications, scalable computing resources, resilient connectivity, and security and governance across an increasingly complex environment.

What looked like an AI initiative becomes an infrastructure challenge.

For CIOs and other technology leaders, the question is no longer simply:

Which AI technologies should we adopt?

It is: Can our existing infrastructure support AI securely, reliably, and at enterprise scale?

Why does cloud infrastructure matter for enterprise AI?

Enterprise AI depends on cloud infrastructure that can securely connect applications, data, users, models, and computing resources across increasingly distributed environments.

AI applications rarely operate within a single system.

An AI agent designed to automate a business process, for example, may need to retrieve customer data, access an ERP or CRM platform, interact with APIs, authenticate users, apply security policies, and initiate actions across several systems.

If those systems cannot communicate reliably—or if the data feeding the AI is fragmented, inaccessible, or poorly governed—the value of the AI application quickly diminishes.

This is why AI readiness cannot be evaluated independently from infrastructure readiness.

For IT executives, the infrastructure supporting AI increasingly needs to provide five foundational capabilities:

• Reliable connectivity across cloud, data center, branch, and edge environments
• Secure and governed access to enterprise data
• Scalable computing and storage resources
• Visibility across distributed applications and infrastructure
• Consistent security across increasingly interconnected environments

Cloud modernization provides an opportunity to strengthen these capabilities before infrastructure limitations become barriers to AI adoption.

Moving to the cloud is not the same as modernizing

Over the past decade, many organizations have measured cloud progress by migration: how many applications moved, how much infrastructure was retired, or how many workloads shifted to public cloud platforms.

Those metrics no longer tell the entire story.

An organization can have significant cloud adoption and still operate with fragmented infrastructure, inconsistent security controls, disconnected data, limited visibility, and inefficient workloads.

Cloud modernization is not simply the movement of workloads from one environment to another. It is the process of creating an infrastructure architecture capable of supporting how the business needs to operate next.

For an AI-enabled enterprise, that architecture is increasingly hybrid.

Some applications may remain on-premises because of performance, regulatory, security, or technical requirements. Others may operate in public cloud environments. Data may reside across multiple platforms. AI processing may occur in centralized cloud infrastructure, private environments, or at the edge.

The objective, therefore, should not necessarily be to move everything into a single cloud.

It should be to ensure that the entire technology environment can function as a connected, secure, resilient, and manageable enterprise architecture.

How does legacy infrastructure limit AI adoption?

Legacy infrastructure can limit AI by creating data silos, connectivity bottlenecks, integration challenges, security gaps, and scalability constraints that make enterprise-wide deployment difficult.

These limitations frequently become more apparent as AI initiatives expand.

A controlled proof of concept may rely on a limited dataset and only a few integrations. Scaling the same solution across the enterprise can require access to dozens of data sources, legacy applications, cloud platforms, APIs, locations, and business processes.

Legacy applications may lack modern integration capabilities. Network performance can vary between locations. Security teams may struggle to maintain consistent policies across environments. Infrastructure teams may lack visibility into where performance problems originate.

The technology may work.

The architecture around it may not.

Before expanding AI initiatives, technology leaders need to identify where infrastructure constraints could become transformation constraints.

AI changes the demands placed on enterprise connectivity

Cloud adoption already transformed enterprise networking. AI is accelerating that transformation.

Traditional networks were largely designed around predictable traffic patterns: employees accessing applications, branches connecting to data centers, and users connecting to centralized resources.

Modern environments are far more distributed.

Applications run across multiple clouds. Employees connect from anywhere. IoT devices continuously generate information. Edge environments process data closer to operations. AI applications may transfer significant volumes of data while interacting with models and services hosted across different environments.

Connectivity is no longer simply the mechanism that allows users to reach applications.

It is becoming part of the AI architecture itself.

Poor network performance can slow AI applications, interrupt automated workflows, limit access to real-time information, and undermine user experience. A lack of redundancy can also introduce operational risk as organizations begin relying on AI-enabled processes for business-critical functions.

This makes network resilience, application performance, intelligent routing, and secure cloud connectivity increasingly important components of AI readiness.

Technology leaders should therefore consider not only whether their networks support today's operations, but whether they can support the applications, data movement, and distributed processing their AI strategies will require in the years ahead.

AI needs connected data, not just more data

Most enterprises do not have a shortage of data.

They have a shortage of accessible, trusted, connected data.

Information may exist across ERP platforms, CRM systems, operational technology, SaaS applications, databases, cloud environments, and legacy systems. Different business units may also maintain their own datasets, definitions, and governance practices.

AI exposes these inconsistencies quickly.

Generative AI and retrieval-augmented generation applications depend on reliable access to relevant enterprise information. Predictive models depend on accurate historical data. AI agents require trustworthy information to make decisions and execute actions.

When the underlying data environment is fragmented, organizations may encounter inaccurate outputs, incomplete insights, duplicated information, or systems that cannot reliably access the data they need.

This creates a different question for technology leaders.

It isn't whether the organization has enough data for AI.

It is whether the organization can securely deliver the right data to the right AI system at the right time.

Security and governance become infrastructure requirements

As AI moves deeper into enterprise operations, the security conversation changes.

An employee using a standalone generative AI tool presents one type of risk. An AI agent with permission to access enterprise data, interact with applications, initiate workflows, or influence business decisions presents another.

The more connected AI becomes, the more important identity, access controls, data governance, cybersecurity, and infrastructure visibility become.

This is especially challenging in hybrid environments where applications and data span public clouds, private infrastructure, SaaS platforms, data centers, and edge locations.

AI raises the stakes further because AI-enabled systems and agents can potentially interact with enterprise resources at machine speed.

Security and governance therefore cannot be added after deployment.

They need to be designed into the architecture supporting AI from the beginning.

For technology leaders, AI governance is becoming increasingly inseparable from infrastructure governance.

Hybrid cloud isn't the problem. Disconnected architecture is.

Hybrid infrastructure is sometimes described as a transitional state—something organizations operate while moving from legacy environments toward the cloud.

For many enterprises, that view is outdated.

Hybrid is increasingly the operating model.

Sensitive workloads may remain within private environments. Applications will continue running in public clouds. SaaS adoption will expand. Operational data may need to remain close to factories, distribution centers, healthcare facilities, or other edge environments. Regulatory and data-sovereignty requirements may also determine where information resides.

AI does not eliminate this complexity.

It makes orchestrating it more important.

The goal should not be to eliminate hybrid infrastructure, but to make hybrid infrastructure operate as a connected architecture.

That requires organizations to think beyond individual platforms and consider how workloads, applications, networks, data, identity, security, and operations work together.

For CIOs, this represents an important shift in cloud strategy.

The next phase of modernization may be less about where workloads live and more about how effectively the enterprise can manage, connect, secure, and adapt them.

What does an AI-ready infrastructure look like?

AI-ready infrastructure provides scalable compute, connected and governed data, resilient connectivity, integrated security, and the flexibility to support changing AI workloads across hybrid environments.

There is no single architecture that will make every organization AI ready. Business requirements, existing technology investments, regulatory obligations, workloads, data, and risk tolerance will shape the right approach.

However, technology leaders can begin by evaluating five foundational areas:

1. Connectivity and resilience

Can applications, users, data, cloud environments, and edge locations communicate reliably? Are critical connections redundant? Can the network accommodate changes in traffic and application demand?

2. Cloud and workload architecture

Are workloads running in environments that make sense based on performance, security, cost, latency, and data requirements? Can infrastructure scale as AI demand changes?

3. Data accessibility and integration

Can AI systems securely access the information they need across enterprise platforms? Are legacy systems creating data silos or integration barriers? Is the underlying data sufficiently governed and trusted?

4. Cybersecurity and governance

Can security policies, identity controls, monitoring, and governance be applied consistently across environments? Does the organization understand what AI applications and agents can access—and what actions they can take?

5. Operational visibility

Can IT teams see how infrastructure, networks, applications, and workloads are performing across the environment? When something fails, can they identify the source quickly enough to prevent a technology problem from becoming a business disruption?

These questions move the AI conversation beyond models and applications.

They help determine whether the enterprise beneath the AI is prepared for what happens when experimentation becomes production.

Modernization doesn't require replacing everything

One of the biggest misconceptions about infrastructure modernization is that organizations must replace their existing technology estate before they can move forward with AI.

For most enterprises, that would be unrealistic—and unnecessary.

The objective should be intentional modernization.

Some legacy systems may need to be replaced. Others may need APIs or integration layers that make their data accessible to modern applications. Certain workloads may benefit from cloud migration, while others may perform better on-premises or at the edge.

Networks may require greater resilience. Security architectures may need to evolve. Data may need to become more accessible and better governed.

The right question is not:

How quickly can we move everything to the cloud?

It is:

Which infrastructure constraints will prevent us from achieving our business and AI objectives?

That question creates a very different modernization roadmap.

Instead of pursuing transformation as a series of isolated technology upgrades, organizations can prioritize investments based on the capabilities and business outcomes they need to enable.

The CIO's AI question should start with the foundation

AI has made it remarkably easy to demonstrate what is possible.

The harder challenge is creating an enterprise capable of doing it repeatedly, securely, reliably, and at scale.

That is where infrastructure becomes strategic.

Before approving the next major AI initiative, technology leaders should understand whether the environment beneath it can support the organization's ambition.

Where is the data? Can applications access it? Can the network move it reliably? Can security teams protect it? Can IT maintain visibility across the environment? Can infrastructure scale when an AI pilot becomes a business-critical application?

And can all of this happen without creating another layer of technology complexity?

These are not simply infrastructure questions anymore.

They are digital transformation questions.

Organizations that modernize cloud, connectivity, data, and security as part of a connected strategy create a stronger foundation not only for AI, but for whatever comes next.

Build the foundation for scalable AI

AI readiness begins before an organization selects a model or launches its next proof of concept. It begins with understanding whether the technology environment can support the business outcomes AI is expected to deliver.

Claro helps organizations assess and modernize the infrastructure behind digital transformation—connecting cloud environments, networks, cybersecurity, data, and emerging technologies into a more resilient and scalable foundation.

Because the goal isn't simply to deploy AI. It's to build an enterprise capable of scaling it.

FAQ

What is AI-ready infrastructure?

AI-ready infrastructure is a technology environment designed to support AI workloads with scalable compute, reliable connectivity, accessible and governed data, integrated cybersecurity, operational visibility, and flexibility across cloud, on-premises, and edge environments.

How does cloud modernization support AI?

Cloud modernization connects applications, data, infrastructure, and computing resources while improving scalability, security, integration, and visibility. These capabilities help organizations move AI from isolated experiments to enterprise-scale deployments.

Does an organization need to move everything to the cloud for AI?

No. Enterprise AI can operate across public cloud, private cloud, on-premises, and edge environments. The priority is creating a secure, connected architecture that allows workloads and data to operate effectively across them.

Why do AI initiatives struggle to scale?

AI initiatives can struggle to scale because of fragmented data, legacy applications, inadequate connectivity, infrastructure limitations, inconsistent security controls, integration challenges, and insufficient governance or operational visibility.

How can CIOs assess AI infrastructure readiness?

CIOs should evaluate connectivity and resilience, workload architecture, data accessibility, cybersecurity and governance, integration capabilities, scalability, and operational visibility before expanding AI across the enterprise.