September 12, 2023

Introducing the SQL Collection Feature from the phData Toolkit CLI

By Anish Kumar

Collecting SQL from various databases is often a challenging and time-consuming process since schemas and syntaxes pretty much always vary across different databases. To solve this challenge, we’ve revamped our phData Toolkit CLI automation tooling to include a SQL collection solution that simplifies collecting SQL across databases. 

In this blog, we’ll explore this new functionality in greater detail and touch on a few other tools within the phData Toolkit that can help gather and analyze SQL.

What is the SQL Collection Feature?

SQL Collection is a feature of the Data Source Tool within the phData Toolkit CLI to help users automate the collection of SQL across various databases.

What Are the Advantages of Automated SQL Collection?

There are several benefits to automating the SQL collection process, but here are four of the most prominent advantages we’ve seen with our team internally and with our customers:

  • Time-Saving: Speeds up the process and reduces time on repetitive tasks 

  • Accuracy: Noticeably less human errors 

  • Efficiency: Multi-threaded approach of SQL collection allows one to handle more tasks simultaneously 

  • Consistency: Follows a predefined approach that ensures uniformity in the process  

How to Collect SQL Using the Data Source Tool from the phData Toolkit CLI

The phData Toolkit CLI has added functionality allowing efficient SQL collection through the Data Source Tool. This tool provides multi-threaded bulk SQL collection and stores the gathered SQL in a structured and organized output directory.

Collecting the SQL for your database is as easy as a single command:

					toolkit ds sql-collect oracle

At the initial release, the supported source database systems are:

  • Snowflake

  • Oracle

  • SQL Server

  • Hive

  • MySQL

  • SQLite

If your database system is missing, let us know!
We’ll be happy to explore adding it to the Toolkit.

Integration with the SQL Translation Tool

The collected SQL can be integrated seamlessly with the SQL Translation Tool from the phData Toolkit in order to analyze migration complexity or translate SQL to the Snowflake equivalent. It can also help with the migration process to the Snowflake Data Cloud.

How to Access the Toolkit CLI

Anyone with a phData Toolkit account can easily download the Toolkit CLI by following the installation instructions. If you don’t have an account with the phData Toolkit, you can sign up today (it’s free).

What’s Next

Get started using the SQL Collection feature today by following our hands-on SQL Collect Tutorial. Be sure to reach out to the team to report bugs, ask questions, and provide feedback.

Data Coach is our premium analytics training program with one-on-one coaching from renowned experts.

Accelerate and automate your data projects with the phData Toolkit