Building a Robust Categorization Engine at Petal
By: Alexander Winemiller
Tags: Machine Learning, Data Science, NLP, Fintech, Text Classification

Our mission at Petal is to make lives easier with credit that is honest, simple, and accessible. A large part of that mission is helping new-to-credit people build their credit history for the first time. To help achieve this goal, applicants’ transaction histories — their checking, savings, and investment data — may be reviewed to get a better view of their overall financial picture. This allows the Petal card to be available to more customers than were they just to be evaluated by their credit score. But interpreting that transactional history isn’t easy, and that’s why we built a model that categorizes transactions: if we can interpret the data, more customers can get approved for cards with terms better tailored to their needs.

One of the most difficult challenges of computer science is translating messy and ad-hoc human language into language that a computer can understand and use to make decisions. Natural language understanding (NLU) is a field that sits at the intersection of machine learning, linguistics, and computer science. It’s informally classified as “AI-complete,” which means that the problems of NLU are too difficult for simple algorithms to solve. Creating a model that understands natural language would require a solution to the strong AI problem, where AI is created that is as flexible, adaptable, and generalizable as human intellect.

Further obstructing our goal of categorizing transactions is the fact that memo lines — the descriptive notes that merchants pass along with transaction data — are not standardized. Memos range from being as clear as “Kroger Groceries Roanoke VA” to as convoluted as “KroGroc718894257XXXXX @ xxxxX ROVA”. It quickly became apparent in reading through these memos that extracting information from such inconsistent strings would be the key to building a well-functioning categorization model.

Since the feature space (or set of inputs) that we need to train against is as large as the possible number of combinations of alphanumeric characters, it suffers from what’s known as the “curse of dimensionality”. Given the scale of data, we decided to train the model against transactions that were pre-labeled. Although these categories contained simple labels such as “Food and Beverages” or “Automotive”, it served as a strong basis from which we could develop our model.

One important nuance in building a well-functioning model is improving on both accuracy and category resolution. Accuracy refers to an input being properly classified; category resolution refers to the number of useful categories that can be predicted from the model. For instance, if clothing, food, office supplies, and electronics purchases all fall into the catch-all category of “Retail,” it will be easier to predict that category accurately, but that category will carry less information for other models to use. Breaking that “Retail” label into more transparent and higher resolution categories is what drives success for models downstream.

While simple mapping can easily reduce the number of labels in a dataset, it is much more difficult to create additional labels. But through various preprocessing techniques, clustering trials, and regular expressions, we were able to enhance the categories of our dataset. We created more detailed and higher resolution categories that ultimately increases the predictive power of the cashflow underwriting model.

Once the dataset was cleaned and labeled, we turned our focus to feature generation. We needed to build features that allowed a model to take only information provided by a transaction’s memo, while leaving aside data such as location or merchant category codes, and produce a category.

Due to misspellings, abbreviations, syntactic information, businesses names that don’t relate to their business (such as “Apple” selling electronics rather than fruit), and characters appended to words that provide no information yet alter the word tokens, it was evident that we couldn’t rely solely on a simple bag-of-words method. We used additional methods to account for lacking or syntactic information from the memos. Ultimately, several hundred thousand features were generated from several million transactions that allowed us to achieve near-perfect category prediction rates.

Lastly, it was important to evaluate how this categorization model would perform at deployment. Since the deployment data was unlabeled, it was necessary to categorize several thousand transactions by hand using Google searches, merchant names, and addresses derived from the transactions. While manual labeling is extremely accurate and provides the closest approximation to the memo’s true categories, it is extremely time-intensive, and thus cannot be used to create the large training set needed to develop the model. The manually labeled transactions did provide key insights on how we could tweak our model, and show us how beneficial our model is to Petal’s overall objective to expand access to safe and affordable credit.

All models, by definition, have discrepancies and inaccuracies when compared to reality. Our categorization model is no different, and there is still room for improvement. We will continue to closely monitor and reduce inaccuracies in our model by retraining it with data that is more comparable with what we see at the start. We’ll scrutinize categories to see if certain ones can be split in order to tighten category fit. Lastly, we’ll explore other data sources to ensure that our model continues to be robust, flexible, and current enough to be able to accurately categorize all transactions that our cardholders make.

This feature will create many downstream opportunities for Petal to build more responsive product features. One example is personal financial management tools: insight into transaction categories will let us help customers budget and better understand their spending. Knowing where customers spend their money could also enable us to create more personally-tailored rewards. Lastly, the insights into what customers are buying could help us build financial products that better match their real-time life needs.

At the end of the day, this work is what powers our financial inclusion mission. When we can properly identify what we’re looking at in cashflow data, we can better understand it. A better understanding facilitates better risk-pricing, which brings down the cost of credit to the cardholders who need it most and lets more “credit invisible” consumers get approved than at the big banks. It’s this value proposition that makes Petal’s card such a powerful credit on-ramp, and one that we’ll use to keep building the future of finance.