case study

pharmacy benefits manager

1.

the customer:

A Phamacy Benefits Manager serving 27+ million members needed help implementing a mission-critical Operational Data Store (ODS), after three failed attempts to get it in production.

2.

THE CHALLENGE THEY FACED:

Over a year into the project — with spending into the millions of dollars — the Spark-based ODS (developed by a previous Cloudera partner) was still too unstable, slow, and prone to data reliability issues to be released to business users.

3.

HOW WE HELPED:

After diagnosing a range of issues, phData reimplemented the data pipeline and Spark jobs in
line with platform-specific best practices and compliance requirements.

4.

WHAT WE GOT DONE:

The ODS is now ready to serve tens of terabytes of data, ingested, processed, and served using Spark in a sustainable, reliable, and fully compliant way, including:
  • 174 tables ingested
  • 4 derived reporting tables
  • 2 billion claims (2 years of data)
  • 1.5 million new and updated claims per day

Full story: Pharmacy Benefits Manager

For Pharmacy Benefits Managers (PBMs), centralizing pharmacy claims information is what empowers them to identify fraud, waste, and other potential savings for both pharmacies and insurers. In other words, their data is their value.

So, when one PBM (serving 27+ million members throughout the U.S.) found themselves struggling to get their operational data store (ODS) into production — key to delivering critical claims data to their insurance clients — they realized they needed a solution, and needed it fast.

Diagnosing the problem

The PBM in question had contracted a large global outsourcing firm to stand up their ODS in Cloudera. But after a year and a half into the project — and three failed attempts to get the ODS into production — almost nothing was working.

The data coming in from the source systems was unreliable, with frequent duplicates and missing data, and Spark processes were extremely slow. The PBM still couldn’t deliver the platform to their customers.

Realizing that throwing outsourcers at the problem wasn’t the answer, they decided to bring in a small team of specialists from phData. The phData engineers and solutions architects analyzed the code developed by the PBM’s previous partner,
and uncovered a range of critical issues:

  • Manual, error-prone build & deploy process
  • Faulty code and platform implementation
  • Absence of data pipeline monitoring

Altogether, these issues — many stemming from lack of technology-specific best practices — explained why PBM
was still unable to get the system into production.

After years of taking a “go-wide approach” to software development — contracting large teams of lower-cost,less-specialized developers — the PBM needed to try something different. It was time to go deep.

The antidote: depth over breadth

phData brought in a small team of data engineering specialists with hands-on knowledge and experience using the specific technologies and processes involved in delivering an ODS on this scale — from Spark, Spark Streaming, and NiFi, to DevOps, agile, and change data capture (CDC).

Although the team was much smaller than the one provided by the PBMs global outsourcer, they were able to do in a matter of months what the big team of generalists had been unable to do in years: get the ODS platform ready to go into production.

phData reimplemented the data pipeline and Spark jobs in a way that included:

  • Build and deploy process designed with long-term reliability and efficiency in mind (e.g., automation, source control, versioning)
  • Best practices for Spark and Scala, reducing instability and making
  • Operational visibility, and logging, and error handling was built into data pipelines
  • Adherence to healthcare security and compliance standards (e.g., HIPAA)

Results

The PBM’s ODS is finally ready to serve tens of terabytes of data ingested, processed, and served via Spark. By taking a “go-deep” rather than a “go-wide” approach — bringing in a small, experienced team that actually understood the technologies and design principles at hand — they were able to build, deploy and run the data pipeline that their users depend on, in a sustainable, reliable, and fully compliant way.

Ready to learn more about phdata? Let's chat.