From Data Engineer to Data Leader: Building High-Impact Careers in the AI Era

0 min read

A Fireside Conversation on Growth, Impact and AI

This Engineers' Day, HCLTech hosted a fireside conversation on a question a lot of technology professionals quietly ask themselves: as AI reshapes data engineering, what does it take to grow from doing the work to leading it? The session was moderated by Ramya Hariharan, HCLTech's Global Talent Acquisition leader for the US and LatAm markets, joined by Nagaraj Shastri, who leads HCLTech's Data and AI service line, and Matt, from talent acquisition at HCLTech focused on US hiring for technology roles.

How Data Engineering Is Evolving

"Gone are the days when you had to code... from a data engineering perspective," said Shastri, describing how coding on tools like DataStage, Informatica, or Databricks is being reshaped by agentic ways of working. Within HCLTech, a user requirement now feeds into an agent-driven process that identifies data sources and builds a "source to target mapping," work that used to be manually scoped.

Engineers across Shastri's teams, which run to several thousand people, are trained and certified on agentic AI through an internal program called AICM (AI Capability Methodology), which covers how agents work, communicate, and operate within a "control plane." About 60% are already certified, with a goal of 80% within 12 months. Shastri tied this to a founding question of the session, whether data is ready, reliable, and governed enough for AI to build on, which is why "the role of a data engineer is changing so rapidly."

Skills That Matter Today

Asked what three skills he'd prioritize for someone starting out, Shastri named base technical skills (Python, Spark, Java), the ability to learn, and the ability to adapt, given HCLTech's 223,000 people across 60 countries.

Matt's hiring lens depended on seniority: junior roles value certifications, senior roles need hands-on breadth, business impact, and end-to-end trade-off thinking. Both speakers flagged communication as its own skill, the ability to "articulate or communicate effectively," and to distill technical complexity for business stakeholders. Domain understanding rounded out the list: as Shastri noted, AI agents themselves now need that same grounding, built through ontologies and knowledge graphs.

What Makes a High-Impact Engineer

Matt said the most common gap is a lack of end-to-end ownership, candidates who've done one part of a project well but haven't influenced architecture decisions or seen the business picture. Shastri connected this to how HCLTech spots emerging leads: not by seniority, but by whether someone takes on the "extra load... in terms of responsibility" and is willing to be accountable for it.

On confidentiality, Ramya Hariharan advised framing contributions by industry and role rather than naming clients, then quantifying the outcome.

"Protect the client's confidentiality. Absolutely. But don't let confidentiality prevent you from telling the story of your own impact." (Ramya Hariharan)

From Engineer to Leader

HCLTech grows leaders through "cohorts," specialization-based groups led by capability managers who connect regional teams so solutions reuse across geographies. Informal leads emerging within these cohorts have even formed their own cross-regional leadership cohort.

Failure isn't disqualifying. "We always realize that a person who is elevated may fail. We are okay with it. We don't blame the person," Shastri said, pointing to capability managers and senior architects as support. He also described a "forward deployment engineer" track, professionals dropped into messy, ambiguous projects who bring structure fast, as a distinct growth path alongside core data engineering.

Transitions and What Enterprises Value

Most transition discussion centered on movement within the discipline: engineers progressing from Informatica to Databricks to agentic AI, a shift that's taken some around three years. Freshers typically enter via computer science or AI degrees; HCLTech closes gaps through lab access and hackathon-style AICM training.

On hiring, Matt pointed to business impact, trade-off reasoning, and collaboration. "The best data engineers," he said, "they're not just coders. They're great collaborators." Shastri named three pillars enterprises value: breadth of technology knowledge, depth of domain understanding, and the ability to help AI agents understand that domain context.

Leadership and AI's Real Effect on Careers

Leadership here was framed as a behavior, responsibility and accountability for outcomes, not a title. On AI's impact on the profession, Matt was direct:

"It's not going to replace what you do as an engineer. It's only going to enhance what you do." (Matt)

New work is emerging instead: maintaining domain ontologies, setting guardrails for agents, and managing how agents interoperate. Shastri admitted he can't predict exactly where the technology goes, "it is moving in all directions," but was confident the profession itself isn't going anywhere.

Practical Career Advice

1. Build a technical base, then prioritize your ability to learn and adapt over any single tool.
2. Seek end-to-end exposure instead of staying in one narrow slice of a project.
3. Take ownership and accountability for your work, even before a leadership title.
4. Quantify your impact by industry and outcome when client details are confidential.
5. Pair external certifications (Databricks, Snowflake, AWS, GCP) with agentic-AI training where available.

Key Takeaways

• Data engineering is shifting from manual coding toward agentic ways of working.
• High-impact professionals stand out through end-to-end ownership, not just tool knowledge.
• HCLTech identifies leadership through responsibility and accountability, surfaced via its cohort model.
• AI agents increasingly need to understand business domain context themselves, creating new engineering work.
• Failure is treated as an acceptable part of taking on more responsibility.

Ready to grow your own career in data and AI? Explore open roles on the HCLTech Careers page.

Source: HCLTech's webinar, "From Data Engineer to Data Leader: Building High-Impact Careers in the AI Era." Watch here