Is it out yet? Automatic Future Product Releases Extraction from Web Data (2022)

Abstract: Identifying the release of new products and their predicted demand in advance is highly valuable for E-Commerce marketplaces and retailers. The information of an upcoming product release is used for inventory management, marketing campaigns and pre-order suggestions. Often, the announcement of an upcoming product release is widely available in multiple web pages such as blogs, chats or news articles. However, to the best of our knowledge, an automatic system to extract future product releases from web data has not been presented. In this work we describe an ML-powered multi-stage pipeline to automatically identify future product releases and rank their predicted demand from unstructured pages across the whole web. Our pipeline includes a novel Longformer-based model which uses a global attention mechanism guided by pre-calculated Named Entity Recognition predictions related to product releases. The model training data is based on a new corpus of 30K web pages manually annotated to identify future product releases. We made the dataset openly available at https://doi.org/10.5281/zenodo.6894770 .

## experiments and results in order to test the product releases identification pipeline we used the aug2020-eng-mc4 dataset for evaluation, described in section 3.1. of note, the aug2020-eng-mc4 dataset was not used during any stage of the pipeline development, and mimics a case of fetching a new monthly snapshot from common crawl to identify future product releases. running the product releases identification pipeline, as described in section 1, on the aug2020-eng-mc4 dataset resulted in 243 overall products for which the release date was identified. example of 10 identified products, can be seen in next, the accuracy of the suggested release dates was evaluated. the true release date was manually labeled per product. since most of the products were categorized as 'video games', 'smartphones', 'electronics' and 'books', the manual labeling of the true release date was done only for the products belonging to these categories. we measure the percentage of products for which the suggested release date was identical to the true release date ('p0') and within the range of 10 or 30 days ('p10', 'p30'). of note, as in some cases only the expected month and year of the future product release date are mentioned in the text (e.g. "will be released in march, 2021"), this results in lower p0 and p10 compared to p30. next, we examine if our buzz calculation can be used to predict at least partly future demand. in order to estimate the product demand we used ebay's search query logs. for each product, that had an actual release date during 2020 (but not before the common crawl snapshot release date of 16-aug, 2020), the demand was estimated by the number of relevant queries found within 30 days since the release date. it is worth noting that not all products were found to be sold on ebay specifically. for simplicity, only products with a dominant single relevant query were examined. figure 2 shows example of a release date correctly identified by the proposed method, and the query logs in the days after the release. next, the correlation between the buzz calculation and the demand was calculated. as can be seen in ## conclusions in this work we demonstrate the capability of automatically identifying future product releases and their ranked demand, from the free monthly snapshot of the common crawl data. the ability to identify product releases in advance is a powerful tool which can be leveraged for multiple downstream applications such as better management of inventory or price updates of outdated models. we also suggest a new ner-guided global attention mechanism to improve long text classification tasks. last, we release a new dataset consisting of web pages labeled as whether the text includes future product releases or not. manzil zaheer, guru guruganesh, kumar avinava dubey, joshua ainslie, chris alberti, santiago ontañón, philip pham, anirudh ravula, qifan wang, li yang, and amr ahmed. 2020. big bird: transformers for longer sequences. arxiv, abs/2007.14062. liang zhao. 2020. event prediction in big data era: a systematic survey. arxiv, abs/2007.09815.