VeloDB’s Nitin Waghulde on why AI agents break down after the demo

AI agents tend to look good in a pilot. The scope is narrow, the data is controlled, and the questions are predictable. Production is different, and that gap is where many projects stall.

Gartner predicts that over 40% of agentic AI projects will be cancelled by the end of 2027, due to escalating costs, unclear business value, or inadequate risk controls. For Nitin Waghulde, Regional Director, APAC at VeloDB, a large part of the problem sits below the model, in the data layer.

In a keynote titled ‘AI agents at scale: Surviving beyond the demo’, Nitin explained how AI agents use data differently from people, what they need to work reliably at scale, and how companies are redesigning their data infrastructure to support them.

Nitin started his session with an example. In a traditional system, a business manager who wanted last year’s sales figure would run a ready-made report and get the number in one lookup.

Ask an AI agent why sales dropped over the last three quarters, and the work changes. The agent pulls order records from the ERP and CRM systems, scans support tickets and chats to check whether major customers have escalated problems, and reads sales team reports on territory dynamics. If its reasoning is incomplete, it goes back for more.

“What used to be completed in one simple lookup or one simple query, now you’re taking hundreds of lookups for the same thing,” he said.

AI is now a customer of your data

In the past, people and reports were the main consumers of business data, and systems were built around fixed reports with clear boundaries. 

Now, Nitin said, AI has become a new customer of that data, and it behaves very differently.

Agents run around the clock, ask many questions at once, and work in an iterative loop: ask, think, search, and retry until they can answer. Their questions are also unscripted. That puts a heavy, unpredictable load on the data layer.

The challenge grows because business data lives in several places at once: transactional records, chat memory, documents, dashboards and pre-built reports, and vector stores for meaning-based search. An agent has to search all of them and combine the results before it can answer.

Each extra trip to the data adds cost. If the data is stale, the answer is wrong. Nitin compared it to driving with an inaccurate traffic map. And when data comes from many sources, it becomes harder to know where an answer came from, which creates a trust problem.

He likened it to sending someone to five filing cabinets to collect pieces of paper. By the time the person returns from the fifth cabinet, the information from the first may already be out of date.

What production agents need

Nitin outlined four requirements for agents to move from pilot to production.

• Access to every data type: Agents need numbers from structured records, text from documents, and meaning from unstructured sources.
• Scale: A single query may now need to scan petabytes of data, not the few terabytes that systems were once designed for. Also, the platform should support concurrency at scale to support iterative querying for data-hungry agents
• Freshness: Agents can only act correctly on current information. If an agent processing a refund cannot see the latest transaction record, it has to wait. Every change, including updates and deletions, needs to be visible to the agent immediately, without waiting for a nightly batch job.
• A record of every interaction: Every AI interaction leaves a trail. These logs are growing faster than business data itself, and they feed back into decision-making.

VeloDB’s answer is a single engine that handles all of these needs. The company was founded in 2023 by the original creators of Apache Doris, an open-source real-time analytics database that became a top-level Apache Software Foundation project in June 2022. According to VeloDB, Apache Doris has more than 14,000 GitHub stars and 700 contributors, and is used by over 5,000 enterprises worldwide.

To cut down the back-and-forth trips that slow agents down, VeloDB supports hybrid queries. A single query can combine SQL, vector search, and text search, running deterministic and probabilistic searches in parallel. The result is one unified source of truth instead of scattered data.

What it looks like at scale

Nitin shared two examples of companies running this architecture in production.

The first was one of the largest crypto exchanges, with more than 120 million users, which he did not name. The exchange makes transaction data available to AI agents that retail investors can query, for example to see what positions large traders are taking on an asset. He said its agents handle about 33,000 conversations with a tail latency of around 3 milliseconds, running round the clock.

The second was MiniMax, a Chinese AI company that listed on the Hong Kong Stock Exchange in January 2026 and says it serves 212 million users in more than 200 countries. MiniMax rebuilt its logging system on Apache Doris after moving away from Grafana Loki, because logs had become a valuable source of intelligence worth keeping and querying. 

Nitin said MiniMax ingests data at 10 Gbps and can query about 10 petabytes in under two seconds, at 70% lower cost. VeloDB has separately reported that the Doris-based system now supports logs across all of MiniMax’s business lines at petabyte scale with over 99.9% availability.

The same challenges apply to early-stage startups building agents, Nitin added. VeloDB works with fintech startups that face the same scaling problems as MiniMax or the crypto exchange, just at an earlier stage.

His closing point was that agents will always ask hundreds of questions, so the infrastructure has to make each one cheap, fast, and current. 

For Nitin, scaling AI agents is less about finding a better model and more about giving agents real-time, trustworthy data from a single source. Companies that get the data layer right, he argued, can make decisions faster, reduce costs, and move their agents from pilot to production with confidence.