When code is cheap, trust is not: Inside the AI-era engineering roundtable

AI has made writing code faster than ever, but the harder question for engineering leaders is what happens after the code is written. At a roundtable on operating engineering in the AI era, leaders from logistics, food tech, healthcare, telecom, fintech, ecommerce and higher education compared notes on scaling with AI without losing reliability, security or control. The shared view was that speed is no longer the hard part. Governing it is.

The roundtable, moderated by Ipshita Basu, brought together Manjunath Nakshathri, Head of Engineering, Mojro; Shobhit Suman, CPO, HungerBox; Pavan Kumar A T, CTO, Digii (DigiiCampus); Pranjal Nadhani, Co-founder and CTO, Cautio; Kenil Paul, VP, Technology and Solutions, 6D Technologies; Rimjhim Agrawal, Co-founder and CTO, BrainSightAI; Dhruv Rastogi, Chief AI Officer, MediAssist; Chandrakesh Pandey, Senior Vice President of Technology, Niyo Solutions Inc.; Achal Sharma, CTO, TechnoSport; Sunil Kumar, Director Engineering, Meesho; and Anshul Sharma, Director Engineering, Scaler.

The session was hosted by groundcover and Data Signals in partnership with YourStory, and was led on the host side by Paul Trebe, Head of Sales and Partnership, groundcover, and Satish Dange, Founder, Data Signals.

Where the bottleneck has moved

Participants agreed that AI has compressed build time, but the pressure has shifted downstream. When agents generate large volumes of code, the questions become who reviews it, who tests it and who monitors dozens of deployments a day. Several leaders said their teams now spend more time on design, context and governance than on writing components. Agents are only as effective as the context they receive, and few organizations yet have repository knowledge, documentation and templates in a form an agent can reliably use.

Drawing the line on autonomy

The room converged on a progressive approach. Reversible, read-only work, such as fetching data or generating reports, can go to agents with little friction. Irreversible actions, such as deleting data or running database updates, stay behind human review. Blast radius matters as much as reversibility. If a wrong call could affect end users or production systems, a person makes it.

One of the participants described a ‘four eyes’ model in which AI does the maker work of reading and analyzing, deterministic rules deliver clear yes or no outcomes, and a human signs off. Another speaker described risk scoring based on severity and occurrence, which focuses human review on medium and high criticality items. Others drew the line by specialization, letting engineers use agents freely in their own domain while production ownership stays with people who can debug at 2 AM.

That scenario also surfaced in a hopeful form. Instead of waking an engineer, an agent can investigate an incident and hand over a report by morning, leaving the decision with a human. Trust, participants said, will build gradually, much as it did with online payments, and the line will keep moving.

Accountability and the skills question

Accountability drew firm agreement: it cannot be delegated to a model. Leaders remain answerable for what agents do. AI works as an intelligence multiplier, one participant noted, which means mistakes are amplified as quickly as good decisions.

Skill erosion sparked livelier debate. Some saw a shift in skills, much as calculators changed arithmetic. Others, particularly in healthcare, worried about losing the expertise needed to judge AI output at all. One practical answer was to decide deliberately where AI drafts and where humans do.

Data control as a design decision

For telecom and healthcare leaders, data control is settled at design time. Models cannot simply run in a public cloud when regulators, tenancy separation, and residency rules apply, and one mistake can bring heavy penalties. Others stressed governance from the first stage, covering what data is ingested, what is used for training versus inference, and how it is masked or de-identified. Concerns about intellectual property and bias also led to a call to vet AI tools as rigorously as new hires.

Observability for an agentic world

Observability ran through every topic. Teams described tracing agent inputs, decisions, and handoffs so they can see why an agent acted as it did and correct it. One ecommerce team learned the cost of underinvesting: a ranking change left every dashboard green while business impact surfaced days later, and the root cause analysis took more than a week. It has since built a change ledger, lineage, a system graph, and a debugging agent that proposes ranked hypotheses.

The observability view was that it can no longer sit apart from development. Paul Trebe put it plainly: “You can no longer decouple observability from development.” Agents need context-rich data to act well, he noted, and the agents themselves must be monitored, since AI traces can run to 500,000 lines that only other agents can realistically parse.

What comes next

No one claimed a finished playbook. Open questions remain on measuring agent reliability and return on investment, protecting data and IP, and keeping engineers sharp enough to challenge what AI produces.

Still, the conversation pointed to a common direction. Give developers room to move fast, but build the guardrails first. Keep humans in charge of irreversible and high-impact calls, and widen automation as trust is earned. Several participants argued that standing still is not an option, since competitors will not wait. The real test of the AI era is less about how quickly teams can build and more about how confidently they can run what they have built.