High-Profile Digital Advertising Firm Leverages Snowpark to Slash Data Runtimes & Compute Costs
Customer's Challenge
With a mission centered on innovation, a leading advertising firm sought to transition away from a disjointed two-system data engineering operation to a unified Snowflake environment in an effort to simplify data workflows and increase compute efficiency.
phData's Solution
phData proposed leveraging Snowpark to efficiently migrate Spark SQL-based Data Engineering pipelines into Snowflake.
Results
Doing a like-for-like comparison between the two platforms (Databricks & Snowflake), the most optimal compute level in Snowpark showed both a 4x improvement in processing time and an estimated 40% savings in compute costs.
The Full Story
The customer relied on two separate systems for their data processing needs. They performed data engineering in Databricks using Spark SQL, then copied the final results into their Snowflake data warehouse for analytics. They knew that this bifurcated process was not the most efficient or cost-effective way to achieve their data processing goals.Â
The customer reached out to phData to simplify their data engineering pipeline and recommend efficiencies in their processes.
phData ran a 4-week proof of concept to assist the customer in this process. Utilizing the Scala API feature in Snowpark, phaData tackled one of the customer’s most challenging and resource-intensive models to determine whether it made sense to migrate the workloads to Snowflake.Â
The configuration was then optimized to minimize runtime and cost. phData provided the client with comparative measurements for Snowpark and their existing technology in addition to the developed source code.
At the end of this process, phData demonstrated that using Snowpark to consolidate Databricks-based data engineering pipelines into Snowflake – alongside the existing analytics workloads – would reduce cost and runtime in addition to simplifying the data architecture.
Why phData?
The customer knew phData’s reputation for assessing and recommending the most efficient data engineering and analytics solutions.
Take the next step
with phData.
Learn how phData can help solve your most challenging data analytics and machine learning problems.
