Cloudera Altus – First Look
I was lucky enough to attend StrataEU 2017 and one of the sessions was Deploying and managing Hive, Spark, and Impala in the public cloud led by Philip Langdale, Eugene […]
My first year at phData
Last month marked my one year anniversary working at phData, and since a lot of my friends and future applicants have been asking me about my experience so far, I […]
Archiving Navigator Audit Data with StreamSets and Kafka
Andy Stadtler helped with this post Many of phData’s customers are heavy users of Cloudera Navigator. Cloudera Navigator provides metadata information to the user who can also audit all actions […]
Visualizing NetFlow Data with Apache Kudu, Apache Impala (incubating), StreamSets Data Collector, and D3.js
NetFlow is a data format that reflects the IP statistics of all network interfaces interacting with a network router or switch. Netflow records can be generated and collected in near […]
Configuring Oozie for Spark SQL on a Secure Hadoop Cluster
A secure hadoop cluster requires actions in Oozie to be authenticated. However, due to the way that Oozie workflows execute actions, Kerberos credentials are not available to actions launched by […]
Parquet vs Text Compression
Parquet is a columnar data format. Columnar data formats, which store data grouped by columns, when tuned specifically for a given dataset can achieve compression ratios of up to 95%. […]
Spark Job History Server OutOfMemoryError
One of our customers hit an issue where the Spark Job History running out of memory every few hours. The heap size was set to 4GB and the customer was […]
try-with-resources in Scala
At phData, many of customers are deploying Apache Spark. As such, those customers look to phData not only for Apache Spark, Hadoop, and Kafka expertise, but also Scala. The following […]
Troubleshooting Spark and Kafka escalations for our managed services customers
My favorite technical task is handling a troubleshooting escalation from one of our Managed Services customers. This week I was lucky enough to work on a problem worth sharing. The job […]
Hive corruption due to newlines and carriage returns
phData has customers across the spectrum of use cases. One of our customers stores vast volumes of XML. One of our engineers was recently asked: Hive sometimes corrupts my data […]