Fortune 500 Construction & Engineering Giant Migrates to Snowflake for Improved Flexibility & Scalability
Customer's Challenge
Considered a North American powerhouse among construction and engineering firms, the customer desired to migrate from Teradata to Snowflake to meet their growing and changing business demands. Specifically, the customer sought a solution to improve their current cost, speed, and access challenges for their data pipelines, asset management, and RFP processes.
phData's Solution
phData proposed migrating the customer’s data to Snowflake using Snowpark. Additionally, Fivetran would facilitate ingestion, and Informatica would assist with transformation to meet the customer’s deadline of 18 months, starting to finish.
Results
The migration – which took 6 months less than anticipated – resulted in $500,000 in savings for the customer. Additionally, the customer has witnessed improved speed and reduced operating costs when comparing Snowflake to their previous data platform, Teradata. phData has also assisted in the longevity of the customer’s success by building operational best practices. This customer story is a successful progression, moving from SAP to EEC to Snowflake, with an S4 system update scheduled for 2025.
The Full Story
The customer was experiencing an incomplete implementation of Teradata, which resulted in high operating costs and a lack of trust from the business users. Additionally, business users lacked access to necessary data to evaluate past projects’ financial impact, which would aid in bidding on new RFPs with confidence. Lastly, the existing data platform could not manage its fleet of 25,000 machines to effectively schedule preventative maintenance, transportation, and sale of underutilized equipment, negatively impacting ROI.
The customer knew that migrating to Snowflake would remedy these challenges. However, their internal team lacked the experience and bandwidth to manage the migration.Â
Through a referral from Snowflake, phData began the 12-month engagement aimed at migrating the customer’s data from their legacy Teradata warehouse and ETL technology stack Â
to Snowflake, which would reduce the total cost of ownership and allow for the insights and AI capabilities a Fortune 500 organization needs to scale.
The teams utilized custom automation and the phData toolkit to accelerate the migration timeline, reduce effort, increase productivity, and retain talent.
Why phData?
The customer reached out to phData through a referral from Snowflake. The customer knew that Snowflake was best in class for cloud-based data warehousing, but their internal team did not have the expertise or bandwidth to handle the migration independently. Snowflake recommended phData based on their extensive experience in successfully migrating data for other Fortune 500 companies.
Teradata Temporal Tables Don’t Exist in Snowflake
First, a custom merge procedure was created in Snowpark to merge temporal logic without SQL code changes. Next, an automation for converting IICS packages into SQL Script was built. Lastly, a conversion process that streamlined code and eliminated 1,000s unnecessary BTEQ lines was developed.
Optimizing Ingestion Pipelines from IICS to Snowflake via HVR
A custom automated process was developed to update the necessary code, repoint the pipelines, and perform testing to ensure proper functionality.
Automated Migration Validation with phData Toolkit
Near the middle of the project, a bottleneck occurred, and the customer wasn’t able to validate ingested data fast enough to keep the project on track. So, the phData team implemented a toolkit to profile and validate data ingest and merge processes. This toolkit ensured no data was lost while the tasks were merged in a dedicated warehouse.
In the end, the customer successfully migrated their data to Snowflake six months faster than anticipated, saving $500,000.
The phData leadership team worked exceptionally hard to help the customer meet the project’s timeline and financial goals, even investing additional dollars themselves to accelerate the project.
Take the next step
with phData.
Learn how phData can help solve your most challenging data analytics and machine learning problems.
