Case Study

Data Streaming & IoT

1.

The Customer:

A large U.S. Oil & Gas (O&G) company seeking to minimize risks to onsite workers and equipment by getting more out of their sensor-based safety monitoring system.

2.

The Challenge They Faced:

Adding capabilities like automated alerting and historical forecasting would require building a new data streaming architecture that could ingest sensor data from the O&G-standard WITSML form factor while meeting very high performance and uptime requirements.

3.

How We Helped:

The phData team architected an Internet of Things (IoT) solution to deliver both real-time and historical data into production, using Streamsets and Kudu to handle the sensor data coming in from the field — building custom software to handle data ingestion from WITSML — and HDFS for long-term storage. They implemented a lambda architecture between Kudu and HDFS, providing a unifying Impala view to query both hot and cold datasets.

4.

What We Got Done:

The O&G’s company’s safety monitoring system now goes far beyond simple dashboards to support automated anomaly detection and alerting, and historical forecasting, empowering them to act more decisively to protect their people, their capital investments, and their brand.

Case Study: Fortune 500 O&G Company Taps Data Streaming and IoT to Fuel Best-in-Class Safety Monitoring System

In the unpredictable world of oil and gas (O&G), the safety of onsite workers is paramount; and with all the variables at play in the extraction process, successful operators know to expect the unexpected. That’s what drives one Fortune 500 O&G company based in the U.S. to do whatever they can to improve their safety monitoring systems.

Their drilling rigs are equipped with a variety of sensors that transmit status readings such as well pressure, flow rate, and temperature to a third-party data vendor. This allows the O&G company’s operations team to monitor status indicators — looking for aberrations that might suggest any potential danger to personnel or equipment — via dashboards on their vendor’s web application.

However, because the company was unable to build additional automation via the vendor tooling, they decided to build a more robust solution that went beyond basic dashboards to actually flag anomalies and send out urgent alerts automatically, in order to minimize risk in the event of an incident. In addition, their data scientists sought a long-term solution to store the sensor data coming in, in order to power historical analyses and predictive analytics to help them preempt future accidents even before they happened.

Complexities of real-time data streaming for O&G

To support monitoring and forecasting across all their drilling sites, the O&G company would need to build out a new data streaming framework that could process the high volume of data coming from their IoT sensors in real time — all while ensuring a consistent, unified view across their internal teams, their third-party data vendor, and their ecosystem of smaller contractors and subcontractors.

Challenges included:

Although the O&G company did have a Cloudera data platform in place, they lacked specific data streaming expertise they needed around technologies like Streamsets and Kudu. Realizing they were out of their depths, they turned to phData.

Digging deeper with IoT

The phData team designed a solution architecture to deliver both real-time and historical data into production, using Streamsets and Kudu to handle the sensor data coming in from the field.
StreamSets to Kudu to Spotfire architecture diagram

As a result, the company’s monitoring systems and dashboards can continuously query the Kudu table where this data was stored to compare it against the historical data now being compiled by their data science team.

Safer extraction through smarter technology

With the solution designed by phData, the O&G’s company’s safety monitoring system now goes far beyond simple dashboards to include automated anomaly detection and alerting, as well as the foundation for future forecasting and historical analysis.
Architecture Diagram phData Monitoring and Streaming Data

Results

With their new automated alerting and historical forecasting capabilities — made possible by the data streaming and IoT solution delivered by phData — the company has vastly improved ability to know what’s going on at their drilling sites. And in the unpredictable world of O&G exploration and drilling, knowledge is power. Detecting issues faster empowers them to act more decisively to protect their people, their capital investments, and their brand.

Ready to learn more about phData? Let's chat.