May 14, 2020

To Big Data, And Beyond: Why Organizations Need Machine Learning Engineers

By Travis Hegner

Introduction to Machine Learning Engineering

Over the past several years, as data acquisition and storage capabilities have exploded within the information technology landscape, so too has the realization that this historical data can be modeled to provide future insights. On the heels of this big data revolution, data science has emerged as one of the most highly sought-after disciplines — and machine learning engineers among the most sought-after roles.

As companies chase these data-driven dreams, they are quickly understanding that data science only takes you part of the way to bottom-line revolution. Every day, organizations are realizing that they need to go beyond data science and into the realm of Machine Learning Engineering.

So, what is Machine Learning Engineering, and why should you know about it? Put succinctly, a machine learning engineer is the bridge that connects data science to your bottom line. A machine learning engineer employs skills from nearly every facet of information technology to launch data science applications and manage their availability. They design and build the infrastructure to manage the lifecycle of the models, including the data required to train them, and the resulting artifacts. Finally, they make sure that the overall process of training, serving, and updating the model is production-ready: redundant, scalable, and maintainable.

Machine Learning Engineering
The typical IT landscape for organizations today has three major components: Computer Science (Software Engineering), Information Systems (Infrastructure and Operations), and Data Science (Advanced Analytics). As skills from each of these areas overlap, you find more advanced disciplines, which are able to provide vast operational improvements inside the organizations they serve. Within the small overlap of all three sectors, you find the skills required for Machine Learning Engineering.

What does a Machine Learning Engineer do, exactly?

Machine learning engineers build their skill sets in a broad range of disciplines. The phrase “jack of all trades” comes to mind, because their curiosity has led them to explore all things technology. They’ve figured out how to train a model and tinkered with its hyper-parameters. They’ve built scalable and redundant infrastructure. Their grit has helped them toil through web server logs, trying to understand why a particular REST call is returning a “500 Internal Server Error”, determined to stop at nothing until they understand the root cause. Here are the five key categories from which a machine learning engineer will derive their skills.

Software Engineering

A machine learning engineer must be able to take unrefined data science code – the output of a research experiment – and translate it into something robust and maintainable. They must be able to find opportunities to optimize and generalize the code into reusable components. Data structures, algorithms, and computational complexity should be second nature as they build scalable solutions. Naturally, source-control skills are an essential aspect of any software-based project. A machine learning engineer will be collaborating closely with other engineers and data scientists. As with any software collaboration, a branching and merging workflow with version control helps developers stay more productive.


Not only should a machine learning engineer be good at coding, they should be able to build the infrastructure on which that code is intended to run. Whether they are building for the cloud or for home-grown on-prem solutions, they need to have a solid grasp of networking, protocols, load balancing, security, containerization, and task scheduling systems. Appropriately designed systems for data capture, centralized logging, and artifact retention are essential to a successful deployment. These systems will allow for repeatable deployments and fast rollback in the case of a deployment failure. Allowing a production application to rely on data science models requires those models to be served from an environment that’s designed around durability. The ability to continue – or quickly restore – functionality during failure scenarios is paramount.


To make data science models production ready, automation is an absolute necessity. Infrastructure should be reproducible from code and data. A properly configured CI/CD pipeline can greatly increase the efficiency of the data science workflow, as well as help prevent the deployment of bad models into production. More advanced pipelines could even offer A/B or canary testing when deploying models. Results from these experiments would allow data scientists to ensure that a model performs adequately and reliably before committing it to the full production load. Moreover, if the problem being solved is one that has new ground-truth data streaming in over time, the process of scheduled model re-training can also be automated to avoid model drift.

Data Engineering

The most essential piece of the data science process is, of course, data. A machine learning engineer must have the skills to move and transform that data into its necessary places and formats to be consumed by the machine learning algorithms. Once a data scientist has discovered the needed features for a model, a machine learning engineer must employ the skills of a data engineer to automate, scale, and operationalize the feature extraction and storage processes, making it more easily consumed by the training pipelines.

Machine Learning

The main thing that sets a machine learning engineer apart from the many advanced disciplines described previously is a solid comprehension of the data science process. This knowledge is akin to the engineers at John Deere understanding the needs of the farmer when designing and building their tractors. An overall grasp of different model types, their use cases, and their resource needs for training and deployment is essential for building an overarching environment for successful machine learning projects.

Naturally, every machine learning engineer comes through a different background and is stronger in some categories than others. But one common trait is that they insist on building their skill set across the board. It should come as no surprise that this particular overlap of skills is quite rare, and is becoming high in demand. The more that organizations realize the potential their data can provide, the more important it will be to keep those data models available and working correctly.

Need Machine Learning Engineering? We can help!

If your organization’s models are stuck in Powerpoint and are failing to provide any bottom-line value, then you may be in need of the operational expertise that a machine learning engineer can bring. At phData, we have a world-class team of machine learning engineers ready to face the toughest challenges, and we will build the tooling and infrastructure required for your data scientists – and your overall business – to thrive.

If your organization has piles of data and no clue how to derive any value from it, then phData also has a fantastic team of data scientists, ready to find patterns in your data to help you maximize your limited resources.

If you’d like to learn more about how phData’s Machine Learning practice can help you with your data science and machine learning needs, then get in touch with us today!

Data Coach is our premium analytics training program with one-on-one coaching from renowned experts.

Accelerate and automate your data projects with the phData Toolkit