At phData, we are obsessed with finding ways to make things easier and more efficient for our customers. This is especially true when working with modern data stack technologies like Fivetran and dbt to help our customers make more data-driven decisions.
As the Fivetran Partner of the Year and the dbt Partner of the Year, we have extensive experience helping clients succeed with both platforms, which is why we’re so excited to share some big news that will make running data transformations in dbt Cloud easier than ever for Fivetran customers.
Fivetran has announced a new orchestration integration with dbt Cloud that allows you to seamlessly connect your dbt transformation pipelines with your Fivetran ingestion pipelines.
In this blog, we’ll explore why this news is such a big deal.
What is Fivetran and dbt Cloud?
Fivetran
Fivetran is a leading automated data integration service, providing businesses with an efficient way to move and centralize data from all their sources. Boasting nearly 500 pre-built data connectors, Fivetran simplifies transferring data to, from, and within any cloud data platform available today.
dbt Cloud
dbt Cloud empowers data engineers and analysts with a web-based interface to deploy dbt, enhancing the core functionalities of dbt Core. This platform extends beyond the basic capabilities, offering additional features such as the ability to create and schedule jobs, administer various environments, and much more, thereby providing a comprehensive suite of tools for advanced data operations.
Why we Think Fivetran’s dbt Cloud Orchestration Integration is a Big Deal
Fivetran has always focused on making data integration easy by managing the movement of all your data with just a few clicks. But to operationalize your data, raw data points need to be transformed into business-ready formats.
Before Fivetran’s new dbt Cloud orchestration integration was released, dbt Cloud customers ran their data transformations on schedules asynchronous of Fivetran connector loads. If they wanted a more integrated approach, they had to use additional third-party orchestration tools or workarounds to integrate the execution of those transformations with the ingestion of their data via Fivetran.
That is no longer needed with Fivetran’s new dbt Cloud orchestration integration.
With the integration, you can leverage the enterprise-ready pairing of Fivetran and dbt for each step of your data pipeline in a fully integrated manner. It starts by setting up the connectors in Fivetran to move all of your critical business data and is as simple as a few clicks. Then, with the new integration set up, as soon as Fivetran lands new data in your assigned destination, the integration triggers your chosen dbt Cloud jobs to run immediately without investing in additional software or workarounds.
Key Benefits of Fivetran’s dbt Cloud Orchestration Integration
Simplified Pipeline Integration
With a few clicks, you can connect your Fivetran account to dbt Cloud to link your data integration and transformation workflows.
More Control Over Costs
The flexible integration levels allow you to choose how closely you want to align Fivetran and dbt. You can trigger dbt jobs immediately after Fivetran finishes ingestion, on a scheduled cadence, or fully independently. This means you are more intelligently running data transformations in your destination, decreasing the accrual of unnecessary computational costs.
Improved SLAs
For the tightest integration, dbt Cloud transformations kickoff right after Fivetran import jobs are complete, reducing latency between raw data arrival and transformed output.
Single Pane View
Fivetran surfaces log data in their platform for easy debugging and monitoring of dbt job statuses after they run. You can now run and maintain your entire pipeline right from their platform while you build using dbt Cloud’s best-in-class in-browser UI.
This integration provides the perfect pairing of Fivetran’s strength in effortless data movement with dbt Cloud’s capabilities for scalable data transformation.
Conclusion
Interest in leveraging this integration? Contact phData today to learn how we can help your business enable this integration to make your data pipelines more efficient!
