How to Know if Your Data Engineering Projects Will be Successful

Data and analytics platform diagram for successful data engineering projects

To make sure data engineering and analytics projects are successful, not only do you need to pick the right technology and have the right people; you also must have the discipline to apply software engineering best practices. What sort of practices am I talking about? Make sure your requirements are clear and communicated to all […]

Trials and Tribulations Preventing Silent Data Loss

A few weeks ago, a colleague from another project team reached out to me for help with an urgent issue. Errors and irregularities had crept into reports tied to one of their StreamSets pipelines, to the point that the business had complained about the data quality. But when the developers inspected the pipeline, they had […]