October 10, 2024

How to Activate EHR Data from Epic and Caboodle into Snowflake

By Bill McLellan

For those who work in healthcare and use Epic technology, this blog offers a look into a drastically improved healthcare experience where data from both Epic’s EHR and your enterprise systems seamlessly integrates to:

  • Enrich provider-patient conversations

  • Improve patient outcomes and care

  • Drive ongoing high-quality, low-cost care

In this blog, we’ll show you how to ingest Epic EHR data into the Snowflake AI Data Cloud to achieve these outcomes and more.

What is Epic?

Epic is the world’s leading Electronic Health Records (EHR) system, used by healthcare organizations worldwide to manage patient care and records. While you likely are already familiar with Epic, there are two key components for data management to consider:

  • Clarity: Epic’s replication of its operational database that is tailored for customer use.

  • Caboodle: Epic’s data mart, essentially Clarity data modeled for business intelligence (BI) and analytics.

Why Integrate Epic Data with Snowflake?

Epic excels at managing healthcare data within its own system. However, you can do much more by integrating Epic data with Snowflake, creating a more holistic and powerful data environment. Here’s why:

  1. Integrate: Clarity and Caboodle contain only Epic data. By combining this clinical data with operational data from systems like ServiceNow, financial and HR data from Workday, or even raw medical records using phData’s custom streaming solutions, you create a unified dataset that offers greater value for patients and your business.

  2. Remodel: Epic’s data model is designed for health records, but it might not align with your enterprise data model or the needs of your cutting-edge solutions. In Snowflake, you can treat Clarity or Caboodle data as a node in an enterprise-wide data mesh. Then you can remodel the data for integration with other EHRs like NextGen and CareLogic, curating it into a star schema for executive dashboards or preparing it for machine learning (ML) and AI use cases. Snowflake allows you to model your data to fit your specific needs.

  3. Extend: Epic continually introduces new features, like generative AI note-taking during appointments, but you may have your own ideas for data applications unavailable in Epic. The possibilities are endless when your data is combined in a customizable environment like Snowflake. Imagine making value-based care (VBC) a reality, automating patient engagement to ensure compliance with doctor’s orders, correlating nurse retention with patient outcomes, or even detecting signs of staff burnout through clinical notes and team communications.

Requirements

If your organization has licenses for Clarity and Caboodle, you can ingest data from both into Snowflake, unlocking a wide range of possibilities for enhanced patient care and operational efficiency.

How to Successfully Ingest Epic Data into Snowflake

Simply ingesting data from Clarity or Caboodle into Snowflake is straightforward, but to ensure the process is optimized, secure, and scalable, here are some best practices based on phData’s experience with numerous healthcare and life science clients:

  1. ELT: Use a SaaS-based extract-load-transform (ELT) tool like Fivetran for change-data-capture, micro-batch, or streaming ingestion from Epic Clarity or Caboodle into Snowflake.

  2. Security: phData’s Toolkit automates and streamlines the provisioning, auditing, and access control of sensitive HIPAA-protected data, ensuring your data operations are secure and compliant.

  3. History: Maintain a versioned history of your clinical records using automation like Snowflake Dynamic Tables, preserving original table and schema structures.

  4. Transformation: Design custom data models, feature stores, and vector stores using visual data modeling tools like sqlDBM and transformation modeling tools like dbt tailored to your organization’s unique needs.

  5. Innovation: Leverage the latest ML tools in Snowpark and large language models (LLMs) in Cortex to build new AI applications in Streamlit, delivering groundbreaking insights through user-friendly interfaces like mobile alerts or chatbots—all within your secure Snowflake environment.

  6. Commercialization: Use the Snowflake Marketplace, Private Data Exchange, and Data Clean Rooms to securely share or monetize your data. Whether democratizing data across internal domains or commercializing a predictive ML model-as-a-service, Snowflake allows you to maximize the value of your data without compromising security.

Why Hire phData?

phData’s AI and healthcare data experts are ready to help you design and build innovative solutions that securely activate your Epic EHR data. With AI tools fed by clinical and operational data, you can free your providers to focus on what matters most: patient care.

Whether you need assistance with planning, tool selection, data migration, or building custom applications, phData offers comprehensive support tailored to your priorities.

What to Expect with phData

phData provides comprehensive support from the initial planning and architectural design to secure data ingestion and advanced analytics implementation to ensure a seamless and successful integration. You can anticipate customized solutions that not only meet regulatory standards but also the ability to use the full potential of your healthcare data, enabling your organization to achieve superior patient outcomes and operational efficiency.

Conclusion

Integrating Epic’s EHR data with Snowflake enables healthcare organizations to enhance patient care, drive operational efficiency, and leverage advanced analytics like AI. By combining Clarity and Caboodle data with enterprise systems, organizations can unlock transformative insights and make data-driven decisions that improve both patient outcomes and business performance.

Contact phData today to start activating and utilizing your EHR data from Epic Clarity and Caboodle.

Data Coach is our premium analytics training program with one-on-one coaching from renowned experts.

Accelerate and automate your data projects with the phData Toolkit