Case Study

From On-Prem to In the Cloud: Major Financial Institution Streamlines Its Data

The Customer’s Challenge

A Fortune 500 financial institution still relied on its on-prem data lake to inform business decisions—this often led to delays in customer communication and an inefficient use of legacy systems and human resources. The client needed a partner to both set up a cloud data platform and operationalize the new environment for reporting, alerting and QA. 

phData’s Solution

After an initial training engagement, the client brought phData in to handle both its move to Snowflake and the setup for supporting data workflows. Our data engineering team quickly got the client up and running, setting them up for automation as they continue to build out their presence in the cloud.

The Full Story

The Fortune 500 financial institution we worked with had millions of customers — and tens of millions of data points flowing into its on-prem data lake. 

While the legacy infrastructure had kept customer information secure and centralized, employees didn’t have easy access to the customer insights, business reports and broader analytics that they needed, to serve their customers effectively. 

The company had to run quality assurance and monitoring manually, leading to delays in key reports and higher operational costs for the business. 

As the client realized they needed a real-time approach to the data they were processing, the need for a cloud-based data platform became clear. 

Why phData?

The team at phData first joined the client to provide initial onboarding and training when the company was first making the move to Snowflake. 

With our team’s deep expertise in both Snowflake and data engineering, the client quickly asked phData to take a more hands-on approach with the transition — they had an ambitious timeline and knew they’d need a reliable partner. 

Through this engagement, the phData team helped the client on three different (but interconnected) fronts:

Building a Data Framework for Snowflake

In the initial engagement, a phData Solution architect established an Information Architecture following industry best practices and standards, making the approach repeatable and sustainable. Following this initial engagement, the client needed a partner to move to the cloud. 

phData’s team of data engineers worked quickly to put together the right framework to move the client fully into a cloud environment. 

Beyond moving data to Snowflake, the project team built out all the interconnected tools for a future cloud-based data lake. This is how the framework took shape: 

*Streamliner is phData’s Data Pipeline Automation tool that simplifies the process of ingesting data onto a new platform. It is not a data ingestion tool in and of itself; rather, it automates other commonly used tools that ingest and manipulate data on platforms like Snowflake

Hands-on Work for a Tight Project Timeline

While our team’s expertise built out the framework for moving to the cloud, we didn’t stop short of getting our hands dirty.

Using both team resources and our onboarding accelerator, Tram*, we worked on behalf of the client to complete data profiling, testing, documentation, along with user provisioning and acceptance. 

*Although the Snowflake platform does support authentication federation, accounts still need to be provisioned within Snowflake, along with databases, schemas, and roles, as well as your information architecture. phData built Tram to take the manual gruntwork out of the equation, and make onboarding users in Snowflake both seamless and painless.

Setting up a Streamlined Workflow

Rather than setting up a data platform  and then leaving the client to their own devices, our team recommended and built automation into the framework to set administrators and end users up for success in the long run.

As the phData team got the client up to speed with the new framework, they also wrote up configuration files and templates to streamline the process for moving similar data. The templates will allow the client to create additional pipelines with the same pattern by simply creating a new YAML configuration file. This will significantly reduce the time spent on new pipelines.

Results

The client has seen four key results from this project:

Armed with more accessible data, the financial institution will be able to improve their processes in a way that flows all the way down to the customer. The organization can now identify process inefficiencies more quickly, helping customers get the products they want more easily. 

Take the next step
with phData.

Learn how phData can help solve your most challenging data analytics and machine learning problems.

Dependable data products, delivered faster.

Snowflake Onboarding Accelerator

Infrastructure-as-code Accelerator

Snowflake Account Visualization and Auditing

Operational Monitoring and Observability Accelerator

SaaS SQL Translator