May 4, 2022

Automated Monitoring in Finance with Machine Learning

By Christina Bernard

The financial services industry has always been keen to adapt machine learning rather quickly. Conversations around market analysis or stock market prediction are coupled with discussions about quantitative models or algorithms. 

In this post, we are going to discuss the possibilities in the realm of automated monitoring. 

Fraud Detection

As an avid traveler, modern fraud detection has saved my card more than once. With millions of transactions happening every day, it is impossible for a human to monitor each one. Automated techniques based on AI are the only solution when dealing with such volume.

Fraud detection models are constructed to recognize abnormal customer behavior. These models look at past behavior and patterns to determine if recent transactions are outside the norm. For example, if there was a purchase on your credit card from a different country and you did not purchase a flight or hotel in that location, your bank will likely block that transaction. The reason being is that you most likely did not actually submit that purchase since none of your most recent behaviors indicate that you are traveling. 

These algorithms are designed to score the likelihood of fraud and send a message to the customer or just automatically reject the charge until verified by the customer. These types of automated monitoring systems can save companies millions of dollars in tracking down, reversing, or stopping fraudulent transactions. It also frees up valuable human resources for other important tasks. 

Compliance and Regulation

Millions of documents flow in and out of financial institutions that need to be updated every time a new regulation passes. In 2020, the SEC issued $4.68 billion in compliance enforcement actions with the average action being worth $2M. Compliance management is not an easy task for administrators to handle manually. 

Leveraging a natural language processing (NLP) model can help compliance officers determine which documents may be out of compliance. Instead of manually searching through hundreds of documents, the NLP model can determine which documents do not match the new set of rules. From there, it can help the compliance officers prioritize which documents are most critical to change.

This process can be paired with robotic process automation (RPA) to lessen the burden by automating the changes where possible. For example, let’s consider 401K contributions. In 2021, the maximum contribution amount for those under 50 was $19,500. In 2022, that changes to $20,500. For a case like this, NLP modeling paired with RPA can change all numbers associated with 401K limits from $19,500 to $20,500.

Risk Evaluation Models

A loan default, whether consumer or commercial, results in huge losses for a bank. Credit risk monitoring models can evaluate the likelihood that a new customer will default on a loan or the likelihood of a current customer defaulting on their active loan. 

Credit risk models take a general credit score several steps further to look at factors like income, debt-to-equity ratio, total assets, loan size, and relationship with the bank. For new customers, these models consider the behavior of existing customers and their default rates to see how likely a new customer is to default. If you want to evaluate a current customer, factors like a deviation from a customer’s normal behavior can also be taken into consideration. 

The output of a credit risk model is generally a score associated with the risk of default, which can also be used in the approval process. To automate loan approvals, this score can be paired with thresholds, like a dollar amount, to either automatically approve or deny an application. For routine loans, this can save bankers a great deal of time. 

This method can also help bankers determine which loans they need to spend more time reviewing. In the case of an existing loan, this score can help the bankers proactively work on mitigating the default risk with the customer directly. Leveraging these algorithms can allow you to send out alerts or start actions that can help save the bank from further losses. 

Marketing and Product Recommendations

Most financial institutions have a goal of being the top provider for customers’ financial needs. Because of this, cross-selling is a critical component of the business. Since finances are a very personal matter, the personalization of recommendations can enhance the efficacy of marketing outreach. Recommendation engines can help financial institutions understand which customers are likely to buy a product based on the behavior of similar customers.

Recommendation engines have been at the core of the eCommerce industry, but it also has a place in the financial industry. Monitoring consumer behavior to send automatic recommendations can lead to some interesting cross-selling opportunities. 

For example, let’s assume there is some correlation between mortgages and auto loans. To understand where the best cross-selling opportunities are, a machine learning model would have to consider similar customer groups that bought an auto loan after a mortgage. From there, it’s important to determine factors around when previous customers received the auto loan. When would be the opportune time to send someone who just bought a home an email about an auto loan? And how likely are they to accept the offer? 

A recommendation engine can compare their profile against others with a similar profile to determine the chances of success.

Algorithmic Trading

When people think of stock trading, they often picture the Wolf of Wall Street.The reality is that Wall Street has largely gone digital. These days algorithmic trading is used to generate and execute large amounts of orders. 

It’s estimated that 70 percent of all trades in the US are algorithmic trading. Algorithmic trading is when you automate what trades will be made by a list of rules and parameters. It can be as simple as if this stock exceeds a price, sell. It can be as complex as automatically searching through the stock market and assigning the probability of an underpriced stock based on a key set of variables.

Consider the Flash Crash of 2010, where an estimated $1 trillion dollars was erased from the market in just a few minutes. This would have been near impossible if most of the trading was conducted in-person vs. online. Is there a way for us to place improved monitoring systems on algorithmic trading to further decrease the risk assumed from these events? 

There is research being conducted into how to optimize algorithmic trading with machine learning instead of a rules-based system. Rules-based systems may work well for normal use-cases, but they can easily snowball into huge mistakes if an unexpected event occurs. Leveraging a machine learning model can help safeguard against unforeseen events, like the Flash Crash, by learning that this behavior is abnormal. 

Machine learning engines consider a variety of factors, other than the ones defined by a human, to determine if a stock should be bought or sold. They can also consider deviations from the normal behavior to halt any large transactions to hedge the risk of a volatile market.

Algorithmic trading can also help create robo-advisors. One hesitation of individuals not entering the stock market is a lack of knowledge. By creating a portfolio management tool that aligns with risk tolerance and savings goals, a robo-advisor can be programmed to automatically manage the portfolio for a customer.

How to Implement Automated Monitoring on the Snowflake Data Cloud

Snowflake can not only help data scientists build for automated monitoring, but also provide them with a platform for deploying those models. The data used to train a model may come from a variety of different sources, such as application/form data, web logs (e.g. user access logs for fraud), and transaction-processing systems.

Snowflake can be used to centralize that data in a common location for model development. Depending on the individual user’s needs, appropriate access and security measures are easy to navigate, a critical component of leveraging sensitive data.

Once the automated monitoring models have been trained, the next step is to create a process that can generate predictions (inference) for new data. The models can be deployed using Snowpark Python user-defined functions (UDFs) to package the model and run inference workloads on Snowflake compute. Predictions generated in this way can be written into Snowflake tables to make them available downstream.

The final step in implementing a monitoring solution is to build automation that leverages the predictions. In the case of fraud detection on the web, this would mean blocking users with fraudulent activity in your web application.

For the purposes of marketing and sales, it is important to export those predictions from Snowflake and insert them into a Customer Relationship Management (CRM) system, like Hubspot or Salesforce, so that sales and marketing teams can take action.

For risk monitoring, the predictions can be used to alert underwriters and auditors within their native systems or dedicated reports. 


Automatic monitoring models have a plethora of applications. Competitors who align their AI strategy closely with their business strategy will see the benefits of implementation. Identifying which area could yield a quick win could very well bring synergy to your business. 

Looking to harness the power of AI and ML at your organization? Be sure to check out our free guide on How to Implement a Successful AI Strategy for Your Company.

Data Coach is our premium analytics training program with one-on-one coaching from renowned experts.

Accelerate and automate your data projects with the phData Toolkit