The dawn of the Data Intelligence Engineer
Meet the Data Intelligence Engineer
The jobs to be done for a data person in the age of AI don’t really match any of the typical responsibilities of other data team members. I think it is different enough to merit a new title, which I hereby dub the “Data Intelligence Engineer”
The Data Intelligence Engineer automates the data engineer’s work (ingestion) and the analytics engineer’s work (transformation and modeling) by creating and conducting software agents. They encode their knowledge of the system into shared context that can be used by other agents to answer questions about the organization. These include things like which fields are joinable but also what someone in the company means when they say “ARR”.
Evals are another big part of their responsibility; these are the system tests that validate that when you ask questions with a known answer, the answer is correct. A good eval suite can give confidence that the types of questions that users ask of the system are going to be answered correctly by the agent. Evals also are a way of knowing when something breaks.
The goal of a Data Intelligence Engineer is to enable self-service. That is, anyone in the organization should be able to open Claude or ChatGPT or another agent and answer questions correctly about the business, use that to build apps and visualizations, and in the end, make high quality decisions with the benefit of data. . When the Data Intelligence Engineer does the job well, everyone who asks a question gets the experience of working with the smartest analyst in the company.
Day to day, that means they are going to be building frameworks, building the data model, curating the context, preventing things from getting broken, fixing them when they break anyway, running evals, and being accountable for the surfaces the business depends on. We wrote the whole thing up as a job description, because we believe this is the way things are going and it’s time to give it a name.
If you’re a data engineer, an analytics engineer, or an analyst, you’ll recognize parts of this. Those roles were already blending into each other, and morphing into something new. This is a role you play, whatever your title says.
