One of the biggest product announcements at Snowflake Summit 2023 was the launch of Snowpark Container Services, a new feature within the Snowflake Data Cloud’s Native App Framework. Snowpark Container Services will extend Snowflake’s programmability to virtually any programming language.
What is a Container?
A container is a unit of software that includes all necessary elements to run in any environment. Containers allow developers to build, test, and run their applications in different environments using a common framework to ensure consistency.
Most containers include application code, external software dependencies, and a lightweight virtual operating system to execute that code. Once a container is built, it can be transferred to any compute environment with a container runtime such as Docker or Kubernetes.
Snowpark Container Services will provide just such an environment through a Kubernetes-like framework for running containers.
Why Does Snowpark Container Services Matter?
Snowpark Container Services will allow virtually any developer to bring code to their data. Cloud-based container services are not a new concept – all major cloud providers have multiple long-standing implementations. But for applications that depend on data in Snowflake, running applications on a cloud-native container service means exporting that data from Snowflake.
Exporting data from Snowflake has two key downsides:
- Performance – For data to leave Snowflake, it needs to be transferred over a network. This can still introduce significant overhead for data applications. Snowpark Container Services will reduce these bottlenecks by colocating the container runtime and data storage.
- Security – When data leaves Snowflake over the network, it opens up the risk surface for sensitive data to be compromised. Keeping data within the Snowflake security perimeter will allow developers to build applications with less worry and eliminate the burden of configuring network routes for security.
What Use Cases are Best for Snowpark Container Services?
Due to their extreme flexibility, containers have been used for virtually any application you could imagine. Here are just a few use cases that we are excited about:
- Running Code in any Language – Snowflake already has support for SQL, Python, Java, and Scala, but there are many more popular languages out there. Snowpark Container Services will open Snowflake up to languages like C++, Ruby, .NET, and more. Got a legacy application in Cobol or Fortran? Containerize it! Wanna play around with Lisp? Go for it! (Just don’t forget the parentheses.)
- Real-Time AI/ML Applications – Most machine uses of machine learning models in production systems fall into one of two categories: batch or real-time. Snowpark Python was already a great choice for batch ML applications. Snowpark Container Services is now the ideal fit for real-time applications that run on one record at a time. Snowpark Container Services will support long-living processes with secured ingress endpoints to serve ML applications over REST interfaces.
- Generative AI and Large Language Models – Many enterprise applications of Large Language Models (LLMs) and Generative AI will be self-hosted and LLMs that are fine-tuned on proprietary data. Running and tuning LLMs is much more efficient on GPU hardware. Snowpark Container Services will include GPU-accelerated compute instances to run containers. NVIDIA will also deploy its NeMo framework for Generative AI on Snowflake.
- Alternative Storage, Retrieval, and Messaging – Some applications have special requirements for purpose-built databases like a Vector Database (e.g., Pinecone) or Cache Database (e.g., Redis). Snowpark Container Services could run those services to accelerate dynamic applications that use data in Snowflake.
How are Application Developers Using Snowpark Container Services?
Snowpark Container Services will become a vital component of the Native Application Framework, which lets developers build and distribute applications to other Snowflake users. Container capabilities considerably push the boundaries of what developers can do with data living in Snowflake.
At Snowflake Summit 2023, many partners demonstrated their applications, including Hex, Dataiku, Astronomer, and phData!
What has phData Done with Snowpark Container Services?
phData has deployed our Advisor Tool on Snowpark Container Services. The Advisor Tool is a free utility that automatically assesses best practices for Snowflake to help users optimize their Snowflake configuration. Snowpark Container Services will eventually allow us to deploy the Advisor Tool as a Snowflake Native App.
In the meanwhile, you can get the Advisor Tool from the phData Toolkit.
How can you get Started with Snowpark Container Services?
As of Snowflake Summit 2023, Snowpark Container Services has launched as a Private Preview feature. Even though it’s not generally available yet, it’s still a great time to imagine the possibilities for developing new applications on Snowflake or re-platforming existing applications to enhance performance and security.
It’s also a great time to think about how your organization can join The Data Cloud by developing Native Apps on Snowflake.
As the 2023 Snowflake Partner of the Year, phData has done all of these things, and we’re happy to help! If you have any questions, please reach out!
