June 18, 2021

Complete Installation Guide of Snowpark on Linux

By Charlie Isaksson

In this technical blog post, we’ll walk you through how to install Snowpark using the Linux version, more specifically, the Ubuntu 20.04 x86-64 version of IntelliJ with version 2021.1.1. It should be noted that this installation process is similar on other operating systems. In particular, once IntelliJ is installed, the process should be identical to this one.

For those unfamiliar with IntelliJ, it’s a Java IDE that supports multiple programming languages as well as operating systems (OS). 

(Please refer to the installation guide for more instructions on other OS)

IntelliJ supports different ways to install the IDE. This includes the following:

  • Download the .tar.gz file
  • Use Toolbox App
  • Command Line / Terminal

On Linux, the straightforward way is the use of a Terminal, which is a standard tool on Linux and comes preinstalled on most of your favorite Linux distros.

To start, open the terminal either via Ctrl+Alt+T keyboard shortcut or from the application menu on your left panel. Type “Terminal” in the search field. 

Opening screen for Terminal

Once the terminal is open, copy this command into your terminal and press enter.

					sudo snap install intellij-idea-community --classic

It will install the community edition as a snap package. You will be asked to enter your password. The install may take some time depending on your internet speed so go ahead and pour yourself a cup of coffee.  

A screenshot of the installation screen of Terminal

Once the installation is complete, click on the application menu from your left panel. Type “IntelliJ” in the search field. You should see the IntelliJ IDEA icon. 

A screenshot of a search bar that has "IntelliJ" typed in

The next step is to set up Scala Development on your IntelliJ. Follow these steps to install all the dependencies:

For the JVM used with Scala version 2.12.x, Java 11 is the recommended version. Before installing Java, let’s make sure you have the Java 11 JDK

Run java -version on the command line and make sure you see java 11.

A screenshot with a few lines of code.

In our case, java wasn’t installed, Fortunately, Ubuntu provides command-line options to install different versions of java open source. You can choose to install version 11:

					$ sudo apt install openjdk-11-jre-headless

However, we are going to install the Oracle Java version 11, which is recommended by Snowpark.

In case you have a version already installed, but the wrong one. In the Figure below, the openJDK is installed but we need the Oracle Java version 11.

A screenshot with five lines of code

Luckily Linux has a handy tool that we can use to change the default Java version. To get a list of installed Java versions, run the command:

					$ update-java-alternatives --list

Note, we have two versions installed on our Ubuntu. The OpenJDK and Oracle.

A screenshot with 4 lines of code

Once you have a list of Java version, set a default one by running the command: 

					$ sudo update-alternatives --config java

Note the start on your left option on the Java 11 OpenJDK. To change the version, type selection number. In our case, we select option 2 and press enter.

A screenshot with a few lines of code.

Check the Java version.

A screenshot displaying Java code

In case you need to download the Oracle version, then you can download the Java SE 11 Development kit from here. It is recommended by Snowflake to install the Oracle version.  

We need “Linux x64 Compressed Archive” “jdk-11.0.11_linux-x64_bin.tar.gz”. You will be asked to register an account with Oracle and accept the license agreement.

A popup message from Oracle saying, "Accept the license agreement.

Once the download is complete, open your Terminal and change the directory to the location where you want to install the JDK, then move the .tar.gz archive file to the current directory.

A screenshot with javascript

Install the software-properties-common package, which adds the add-apt-repository:

					$ sudo apt install software-properties-common

Then use the add-apt-repository command to add the repo to your list of package sources:

					$ sudo add-apt-repository ppa:linuxuprising/java

You’ll see this message:

Oracle Java 11 (LTS) and 12 installer for Ubuntu, Linux Mint and Debian.

Java binaries are not hosted in this PPA due to licensing. The packages in this PPA download and install Oracle Java 11, so a working Internet connection is required.

The packages in this PPA are based on the WebUpd8 Oracle Java PPA packages: https://launchpad.net/~webupd8team/+archive/ubuntu/java

Created for users of https://www.linuxuprising.com/

Installation instructions (with some tips), feedback, suggestions, bug reports etc.:

. . .

Press [ENTER] to continue or ctrl-c to cancel adding it

Press ENTER to continue the installation. You may see a message about no valid OpenPGP data found, but you can safely ignore this.

Update your package list to make the new software available for installation:

					$ sudo apt update

The installer will look for the Oracle JDK you downloaded in /var/cache/oracle-jdk11-installer-local. Create this directory and move the Oracle JDK archive there:

					$ sudo mkdir -p /var/cache/oracle-jdk11-installer-local/
$ sudo cp jdk-11.0.7_linux-x64_bin.tar.gz /var/cache/oracle-jdk11-installer-local/

Finally, install the package with this command:

					$ sudo apt install oracle-java11-installer-local
A screenshot with several lines of code

The installer will first ask you to accept the Oracle license agreement. Accept the agreement, then the installer will extract the Java package and install it.

A screenshot of the Oracle license agreement pop up

Select “yes” and the install will start.

Installer pop up that has two options, yes or no

To check the version of Java running, use the command:

A screenshot of a java command

Then, after starting up IntelliJ, you can download and install the Scala plugin by following the instructions on how to install IntelliJ plugins (search for “Scala” in the plugins menu.)

From the left menu click on the plugins option:

A program that has the message, "Welcome to IntelliJ Idea"

Use the Marketplace tab to browse and install plugins from the JetBrains Plugin Repository or a custom plugin repository. Use the Install tab to install the plugin and then restart IDE.

A screenshot of a variety of plugins including JetBrains

How to Create a New Scala Project in IntelliJ IDEA

Now that we have IntelliJ, Java, and Scala plugin installed, we can move to create a new Scala project in IntelliJ IDEA

Choose New Project:

A program that has the message, "Welcome to IntelliJ Idea"

In the list on the left, select Scala, and on the right, select IDEA:

A screenshot of the IntelliJ startup sequence that says, "New project."

Click Next and then fill in the details for your new project.

A screenshot of the "New project" in IntelliJ IDEA

From above, Figure Scala SDK is empty which means we don’t have Scala SDK installed. We can install the Scala SDK by clicking on the Create buttons >> Download button >>  Choose Scala version (in our case 2.12.11) >> click ok button.

A screenshot with a pop-up message

IntelliJ IDEA downloads the SDK with the required Scala version.

A screenshot of the IntelliJ IDEA download message

Now that the Scala SDK is installed we can click on the Finish button to create the new project.

A pop-up message in IntelliJ IDEA that has a "finish" button.

Configuring the IntelliJ IDEA Project for Snowpark

You have two options to configure the project:

  • Using the  Snowpark library and its dependencies in to your project  
  • Using the Snowpark Library in a Maven Project

Using the Snowpark Library and its Dependencies in to your Project

  1. Download the Snowpark library and dependencies from here.
  2. Choose File >> Project Structure.
  3. Under Project Settings, choose Libraries.
  4. Click the + button to add a new project library, and choose Java.
A screenshot with a pop-up message that's titled, "Project Structure" Mitch Rudolph Mitch Rudolph 6:17 PM Yesterday Meta: A screenshot with a pop-up message that says, "Select Library Files" Mitch Rudolph Mitch Rudolph 6:21 PM Yesterday Meta: A computer screen with some code

Browse to the directory where you extracted the contents of the TAR/ZIP archive file that you downloaded earlier. Select the lib directory.

A screenshot with a pop-up message that says, "Select Library Files"

Click the + button to add a new project library, and choose Java.

Browse to the directory where you extracted the contents of the TAR/ZIP archive file that you downloaded earlier. Select the snowpark-<version>.jar file.

A screenshot with a pop-up message that says, "Select Library Files"

Click OK, and then click Apply.

A screenshot with a pop-up message that says, "Project Structure"

Using the Snowpark Library in a Maven Project

To integrate the Snowpark library into a project that uses a Maven build, add the library as a dependency.

  1. In the pom.xml file for your project, make the following changes.
    1. Add the repository into the repositories tag: “<repositories> … </repositories>”
    2. Add the Snowpark Library into the dependencies tag: “<dependencies> … </dependencies>”
A screenshot of several lines of code

From your Maven Lifecycle tab. Do clean and then compile or whatever your project requires. 

A screenshot of the Maven lifecycle tab

Verifying Your IntelliJ IDEA Project Configuration

To verify you have configured your project to use Snowpark, run a simple example of Snowpark code.

In the Project tool window on the left, expand your project, and select the src folder. Right-click on the folder, and choose New >> Scala class.

In the Create New Scala Class dialog box, enter the name “Main”, select Object, and press the Enter key. In the Main.scala file, replace the contents with code.

Replace the with values that you use to connect to Snowflake.

					import com.snowflake.snowpark._
import com.snowflake.snowpark.functions._

object Main {
  def main(args: Array[String]): Unit = {
    // Replace the <placeholders> below.
    val configs = Map(
      "URL" -> "https://<account>.snowflakecomputing.com:443",
      "USER" -> "<user name>",
      "PASSWORD" -> "<password to the database>",
      "ROLE" -> "<role name>",
      "WAREHOUSE" -> "<warehouse name>",
      "DB" -> "<database name>",
      "SCHEMA" -> "<schema name>"
    val session = Session.builder.configs(configs).create

    session.sql("show tables").show()

Click the green arrow next to the Object line, and choose Run Main to run the example.

A computer screen with some code

If all is good, you should see a table with all your databases that exist on your Snowflake.

					[main]  INFO (Logging.scala:22) - Closing stderr and redirecting to stdout
[main]  INFO (Logging.scala:22) - Done closing stderr and redirecting to stdout
[main]  INFO (Logging.scala:22) - 
SnowflakePlan[show tables]

|"created_on"             |"name"      |"database_name"  |"schema_name"  |"kind"  |"comment"  |"cluster_by"  |"rows"  |"bytes"  |"owner"  |"retention_time"  |"automatic_clustering"  |"change_tracking"  |"is_external"  |
|2021-05-06 13:19:28.397  |UBER_TABLE  |USER_CISAKSSON   |UBER_SCHEMA    |TABLE   |           |              |652435  |3811840  |PUBLIC   |1                 |OFF                     |OFF                |N              |


Our sincere goal is that this technical blog makes it easier to install Snowpark in Linux. For more how-to guides, articles, and videos of Snowpark, be sure to visit our blog or contact us directly.

Data Coach is our premium analytics training program with one-on-one coaching from renowned experts.

Accelerate and automate your data projects with the phData Toolkit