Deep Learning Framework for Identifying Future Market Opportunities from Textual User Reviews (2024)

Abstract: The paper develops an application of design gap theory for identification of future market segment growth and capitalization from a set of customer reviews for bought products from the market in a given past period. To build a consumer feature space, an encoded-decoder network with attention is trained over the textual reviews after they are pre-processed through tokenization and embedding layers. The encodings for product reviews are used to train a variational auto encoder network for representation of a product feature space. The sampling capabilities of this network are extended with a function to look for innovative designs with high consumer preferences, characterizing future opportunities in a given market segment. The framework is demonstrated for processing of Amazon reviews in consumer electronics segment.

## introduction in recent years, the large language models (llm) such as gpt, palm, llama demonstrate impressive performance on complex linguistic tasks in automatic text generation. there are a number of milestones marking this success in building such artificial neural network (ann) models -encoder/decoder structures, probabilistic distribution encoding, embedding layers, long short-term memory cells, attention modeling -to name a few @xcite @xcite . since the growth of cloud computational capabilities and intensive research in the area, we see more claims that llm are capable of outperforming or matching humans in number of language or image processing tasks. the so marked progress in ann models allows development of more advanced expert systems to solve computational problems. the focus of the present work is on developing a class of probabilistic predictive models for extracting information about consumer preferences in a particular online market, where the products in the market are divided in categories. as econometric knowledge explains, the consumer utility is the driving force for the current and future dynamics of the market. hence, the model aim at identification of so-called "design gaps" in the selected market segments, which allows for economic decisions for developing new products in the given segment. information about consumer preferences is probabilistically encoded based on an unknown utility function over consumer/design pairs together with prior distributions over customers and designs. the analytical form of the probability densities and embedding representation allow representing the consumer preference information as a multilayer smooth neural network, which allows employment of gradient based methods for learning of hyperparameters from a training data set. the design gap models allow the prediction of consumer preference for an "unknown and not existing products". the prediction cannot exactly tell what will be these future products, even though an effort can be made to extract such information from the underlying feature space. the model output is a bounded subset of the design space or feature space, which will be favored by the customers. such bounded subset can be contrasted with the unbounded set of all possible future designs. in the present research, we follow the approach from @xcite for identification of design gap into a diverse design and consumer spaces, where they identify possible future products in the automotive sector. based on the existing market data, a probabilistic consumer utility model is estimated to account for the likelihood of a certain product design to be selected by a given consumer. they also introduce auxiliary feature dimensions for product and the consumer to improve the efficiency in estimation of the probability distribution. similarly, (ren and scott, 2017) present adaptive approach for identification of optimal product designs from a finite set of possibilities, proposing group identification splitting algorithm (gisa) that picks queries that halves the so defined partworth space but preserves entire groups. the contribution of the present paper is that it develops a way to recover consumer and design vector representations from the raw textual reviews connecting a particular consumer to a particular product. therefore, our assumption is that a review text is that all product or consumer features can be extracted from enough number of reviews. the first component of the model is to build a simple natural language transformer trained for estimating the underlying feature vector from each textual review. the model relies on a long short-term memory (lstm) cells and attention mechanism such to enable the decoder subsystem to predict the next word token in a sequence to match a given input sequence. a specific feature in this model is that we don't aim a high generative accuracy characteristic for llm mentioned above. instead, we need to ensure that the model is simple enough to capture in the low-dimensional state vector the essential review information and the state vector can be used to recover the essence of keywords in the input sequence. the language transformer is used to process all the reviews from the selected market segments and allows for attaching a number of describing vectors to each product, for which we have a published review. on the other hand, we can characterize each customer by the state vectors of all reviews generated by him. given this representation, we build a market transformer as a variational auto encoder (vae) network. the vae models a multivariate probability distribution over a small dimensional feature space, where all product designs from a given market segment are projected. the distribution can be used to generate high probability and low probability designs using a sampling layer. the vae is tuned to recover the vectors describing a given market segment, and on the other hand to match the consumer preferences in this market segment. the rest of the paper is organized as follows. section 2 gives details on structure and tuning of the language transformer. section 3 presents the components of the vae model, and section 4 summarized the evaluation results in 5 product categories from the online shopping site amazon. ## language transformer since the information about consumer preferences is carried by the consumer reviews, which are in natural language, we design a review transformer. the transformer is a connection between an encoder and a decoder network. the purpose of the present research is not to achieve extremely accurate modelling of the natural language. instead, we aim to ensure that basic meaning carried by review is recognized, especially with respect to tokens reflecting consumer preference, individual consumer features, and individual product features. ## market transformer the market transformer is developed as a variational auto encoder, which acts as a model for the probability distribution ## results the following figures present the obtained results for 5 categories of amazon reviews. the design gap in a given market segment is predicted by looking for low probability designs (i.e. not yet discovered) through a monte-carlo sampling of the underlying multivariate probability distribution. the low probability products are evaluated with respect to the consumer preference probability function p m (x d |x c ). hence in case of low probability design, which is highly preferable by customers we can conclude that in a given market segment we have unrealized potential for innovation. to validate the correctness of such predictions, we compare the observed market growth in the 5 segments from 2013 to 2018 year. figure 1 gives information about observed market capitalization growth and predicted design gap in the sectors. we observe positive correlation between models scores and capitalization growth, which means that the model correctly predicts future capital allocation in the observed sectors. in figure 2 we observe the product survival rate vs market prediction, where again we see a positive correlation with the model predictions. the product survive rate is characteristic to how much a given product is in demand during the observed period. in figure 3 we examine correlation between market scores and actually appearing new products in the observed domains. here also a positive correlation with the predicted design gap is observed. for a reference of the size of the observed market segments, we calculate their market capitalization at the starting year in figure 4 . figure 3: model prediction vs new market entrants figure 4: market capitalization ## conclusion the present research combined key techniques from llm research to build a simplified language transformer for encoding essential information in textual user reviews used to characterize the consumer preference in online marketing. the applicability of this encoded information is demonstrated for encoding actual consumer preferences using a variational auto encoder model. the developed market segment transformer allows prediction of possible low probability designs in a given market which might be highly favored by its eventual consumers.