pwc-archive/papers-with-abstracts huggingface dataset:
Starbucks: Improved Training for 2D Matryoshka Embeddings
Authors: Shengyao Zhuang, Shuai Wang, Bevan Koopman, Guido Zuccon
Date: 2024-10-17 00:00:00
Tasks: Language Modelling, text similarity

Effective approaches that can scale embedding model depth (i.e. layers) and embedding size allow for the creation of models that are highly scalable across different computational resources and task requirements. While the recently proposed 2D Matryoshka training approach can efficiently produce a single embedding model such that its sub-layers and sub-dimensions can measure text similarity, its effectiveness is significantly worse than if smaller models were trained separately. To address this issue, we propose Starbucks, a new training strategy for Matryoshka-like embedding models, which encompasses both the fine-tuning and pre-training phases. For the fine-tuning phase, we discover that, rather than sampling a random sub-layer and sub-dimensions for each training steps, providing a fixed list of layer-dimension pairs, from small size to large sizes, and computing the loss across all pairs significantly improves the effectiveness of 2D Matryoshka embedding models, bringing them on par with their separately trained counterparts. To further enhance performance, we introduce a new pre-training strategy, which applies masked autoencoder language modelling to sub-layers and sub-dimensions during pre-training, resulting in a stronger backbone for subsequent fine-tuning of the embedding model. Experimental results on both semantic text similarity and retrieval benchmarks demonstrate that the proposed pre-training and fine-tuning strategies significantly improved the effectiveness over 2D Matryoshka models, enabling Starbucks models to perform more efficiently and effectively than separately trained models.
Transformers4NewsRec: A Transformer-based News Recommendation Framework
Authors: Dairui Liu, Honghui Du, Boming Yang, Neil Hurley, Aonghus Lawlor, Irene Li, Derek Greene, Ruihai Dong
Date: 2024-10-17 00:00:00
Tasks: Model Selection, News Recommendation

Pre-trained transformer models have shown great promise in various natural language processing tasks, including personalized news recommendations. To harness the power of these models, we introduce Transformers4NewsRec, a new Python framework built on the \textbf{Transformers} library. This framework is designed to unify and compare the performance of various news recommendation models, including deep neural networks and graph-based models. Transformers4NewsRec offers flexibility in terms of model selection, data preprocessing, and evaluation, allowing both quantitative and qualitative analysis.
Towards Computational Analysis of Pansori Singing
Authors: Sangheon Park, Danbinaerin Han, Dasaem Jeong
Date: 2024-10-16 00:00:00
Tasks: Information Retrieval, Music Information Retrieval, Retrieval

Pansori is one of the most representative vocal genres of Korean traditional music, which has an elaborated vocal melody line with strong vibrato. Although the music is transmitted orally without any music notation, transcribing pansori music in Western staff notation has been introduced for several purposes, such as documentation of music, education, or research. In this paper, we introduce computational analysis of pansori based on both audio and corresponding transcription, how modern Music Information Retrieval tasks can be used in analyzing traditional music and how it revealed different audio characteristics of what pansori contains.
Enhancing Affinity Propagation for Improved Public Sentiment Insights
Authors: Mayimunah Nagayi, Clement Nyirenda
Date: 2024-10-12 00:00:00
Tasks: Clustering, Dimensionality Reduction, Marketing, Sentiment Analysis

With the large amount of data generated every day, public sentiment is a key factor for various fields, including marketing, politics, and social research. Understanding the public sentiment about different topics can provide valuable insights. However, most traditional approaches for sentiment analysis often depend on supervised learning, which requires a significant amount of labeled data. This makes it both expensive and time-consuming to implement. This project introduces an approach using unsupervised learning techniques, particularly Affinity Propagation (AP) clustering, to analyze sentiment. AP clustering groups text data based on natural patterns, without needing predefined cluster numbers. The paper compares AP with K-means clustering, using TF-IDF Vectorization for text representation and Principal Component Analysis (PCA) for dimensionality reduction. To enhance performance, AP is combined with Agglomerative Hierarchical Clustering. This hybrid method refines clusters further, capturing both global and local sentiment structures more effectively. The effectiveness of these methods is evaluated using the Silhouette Score, Calinski-Harabasz Score, and Davies-Bouldin Index. Results show that AP with Agglomerative Hierarchical Clustering significantly outperforms K-means. This research contributes to Natural Language Processing (NLP) by proposing a scalable and efficient unsupervised learning framework for sentiment analysis, highlighting the significant societal impact of advanced AI techniques in analyzing public sentiment without the need for extensive labeled data.
Enhancing Long Context Performance in LLMs Through Inner Loop Query Mechanism
Authors: Yimin Tang, Yurong Xu, Ning Yan, Masood Mortazavi
Date: 2024-10-11 00:00:00
Tasks: RAG, Retrieval, Retrieval-augmented Generation

Transformers have a quadratic scaling of computational complexity with input size, which limits the input context window size of large language models (LLMs) in both training and inference. Meanwhile, retrieval-augmented generation (RAG) besed models can better handle longer contexts by using a retrieval system to filter out unnecessary information. However, most RAG methods only perform retrieval based on the initial query, which may not work well with complex questions that require deeper reasoning. We introduce a novel approach, Inner Loop Memory Augmented Tree Retrieval (ILM-TR), involving inner-loop queries, based not only on the query question itself but also on intermediate findings. At inference time, our model retrieves information from the RAG system, integrating data from lengthy documents at various levels of abstraction. Based on the information retrieved, the LLM generates texts stored in an area named Short-Term Memory (STM) which is then used to formulate the next query. This retrieval process is repeated until the text in STM converged. Our experiments demonstrate that retrieval with STM offers improvements over traditional retrieval-augmented LLMs, particularly in long context tests such as Multi-Needle In A Haystack (M-NIAH) and BABILong.
Post-Userist Recommender Systems : A Manifesto
Authors: Robin Burke, Morgan Sylvester
Date: 2024-10-09 00:00:00
Tasks: Recommendation Systems

We define userist recommendation as an approach to recommender systems framed solely in terms of the relation between the user and system. Post-userist recommendation posits a larger field of relations in which stakeholders are embedded and distinguishes the recommendation function (which can potentially connect creators with audiences) from generative media. We argue that in the era of generative media, userist recommendation becomes indistinguishable from personalized media generation, and therefore post-userist recommendation is the only path forward for recommender systems research.
A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions
Authors: Shailja Gupta, Rajesh Ranjan, Surya Narayan Singh
Date: 2024-10-03 00:00:00
Tasks: Question Answering, RAG, Retrieval, Retrieval-augmented Generation

This paper presents a comprehensive study of Retrieval-Augmented Generation (RAG), tracing its evolution from foundational concepts to the current state of the art. RAG combines retrieval mechanisms with generative language models to enhance the accuracy of outputs, addressing key limitations of LLMs. The study explores the basic architecture of RAG, focusing on how retrieval and generation are integrated to handle knowledge-intensive tasks. A detailed review of the significant technological advancements in RAG is provided, including key innovations in retrieval-augmented language models and applications across various domains such as question-answering, summarization, and knowledge-based tasks. Recent research breakthroughs are discussed, highlighting novel methods for improving retrieval efficiency. Furthermore, the paper examines ongoing challenges such as scalability, bias, and ethical concerns in deployment. Future research directions are proposed, focusing on improving the robustness of RAG models, expanding the scope of application of RAG models, and addressing societal implications. This survey aims to serve as a foundational resource for researchers and practitioners in understanding the potential of RAG and its trajectory in natural language processing.
Leveraging Large Language Models to Enhance Personalized Recommendations in E-commerce
Authors: Wei Xu, Jue Xiao, Jianlong Chen
Date: 2024-10-02 00:00:00
Tasks: Diversity, Language Modeling, Language Modelling, Large Language Model

This study deeply explores the application of large language model (LLM) in personalized recommendation system of e-commerce. Aiming at the limitations of traditional recommendation algorithms in processing large-scale and multi-dimensional data, a recommendation system framework based on LLM is proposed. Through comparative experiments, the recommendation model based on LLM shows significant improvement in multiple key indicators such as precision, recall, F1 score, average click-through rate (CTR) and recommendation diversity. Specifically, the precision of the LLM model is improved from 0.75 to 0.82, the recall rate is increased from 0.68 to 0.77, the F1 score is increased from 0.71 to 0.79, the CTR is increased from 0.56 to 0.63, and the recommendation diversity is increased by 41.2%, from 0.34 to 0.48. LLM effectively captures the implicit needs of users through deep semantic understanding of user comments and product description data, and combines contextual data for dynamic recommendation to generate more accurate and diverse results. The study shows that LLM has significant advantages in the field of personalized recommendation, can improve user experience and promote platform sales growth, and provides strong theoretical and practical support for personalized recommendation technology in e-commerce.
Optimizing and Evaluating Enterprise Retrieval-Augmented Generation (RAG): A Content Design Perspective
Authors: Sarah Packowski, Inge Halilovic, Jenifer Schlotfeldt, Trish Smith
Date: 2024-10-01 00:00:00
Tasks: Question Answering, RAG, Retrieval, Retrieval-augmented Generation

Retrieval-augmented generation (RAG) is a popular technique for using large language models (LLMs) to build customer-support, question-answering solutions. In this paper, we share our team's practical experience building and maintaining enterprise-scale RAG solutions that answer users' questions about our software based on product documentation. Our experience has not always matched the most common patterns in the RAG literature. This paper focuses on solution strategies that are modular and model-agnostic. For example, our experience over the past few years - using different search methods and LLMs, and many knowledge base collections - has been that simple changes to the way we create knowledge base content can have a huge impact on our RAG solutions' success. In this paper, we also discuss how we monitor and evaluate results. Common RAG benchmark evaluation techniques have not been useful for evaluating responses to novel user questions, so we have found a flexible, "human in the lead" approach is required.
Ads Supply Personalization via Doubly Robust Learning
Authors: Wei Shi, Chen Fu, Qi Xu, Sanjian Chen, Jizhe Zhang, Qinqin Zhu, Zhigang Hua, Shuang Yang
Date: 2024-09-29 00:00:00
Tasks: counterfactual

Ads supply personalization aims to balance the revenue and user engagement, two long-term objectives in social media ads, by tailoring the ad quantity and density. In the industry-scale system, the challenge for ads supply lies in modeling the counterfactual effects of a conservative supply treatment (e.g., a small density change) over an extended duration. In this paper, we present a streamlined framework for personalized ad supply. This framework optimally utilizes information from data collection policies through the doubly robust learning. Consequently, it significantly improves the accuracy of long-term treatment effect estimates. Additionally, its low-complexity design not only results in computational cost savings compared to existing methods, but also makes it scalable for billion-scale applications. Through both offline experiments and online production tests, the framework consistently demonstrated significant improvements in top-line business metrics over months. The framework has been fully deployed to live traffic in one of the world's largest social media platforms.
Disaggregating Embedding Recommendation Systems with FlexEMR
Authors: Yibo Huang, Zhenning Yang, Jiarong Xing, Yi Dai, Yiming Qiu, Dingming Wu, Fan Lai, Ang Chen
Date: 2024-09-28 00:00:00
Tasks: Recommendation Systems

Efficiently serving embedding-based recommendation (EMR) models remains a significant challenge due to their increasingly large memory requirements. Today's practice splits the model across many monolithic servers, where a mix of GPUs, CPUs, and DRAM is provisioned in fixed proportions. This approach leads to suboptimal resource utilization and increased costs. Disaggregating embedding operations from neural network inference is a promising solution but raises novel networking challenges. In this paper, we discuss the design of FlexEMR for optimized EMR disaggregation. FlexEMR proposes two sets of techniques to tackle the networking challenges: Leveraging the temporal and spatial locality of embedding lookups to reduce data movement over the network, and designing an optimized multi-threaded RDMA engine for concurrent lookup subrequests. We outline the design space for each technique and present initial results from our early prototype.
Efficient Training of Self-Supervised Speech Foundation Models on a Compute Budget
Authors: Andy T. Liu, Yi-Cheng Lin, Haibin Wu, Stefan Winkler, Hung-Yi Lee
Date: 2024-09-09 00:00:00
Tasks: Data Augmentation, Self-Supervised Learning

Despite their impressive success, training foundation models remains computationally costly. This paper investigates how to efficiently train speech foundation models with self-supervised learning (SSL) under a limited compute budget. We examine critical factors in SSL that impact the budget, including model architecture, model size, and data size. Our goal is to make analytical steps toward understanding the training dynamics of speech foundation models. We benchmark SSL objectives in an entirely comparable setting and find that other factors contribute more significantly to the success of SSL. Our results show that slimmer model architectures outperform common small architectures under the same compute and parameter budget. We demonstrate that the size of the pre-training data remains crucial, even with data augmentation during SSL training, as performance suffers when iterating over limited data. Finally, we identify a trade-off between model size and data size, highlighting an optimal model size for a given compute budget.
SATS: simplification aware text summarization of scientific documents
Authors: Farooq Zaman, Faisal Kamiran, Matthew Shardlow, Saeed-Ul Hassan, Asim Karim, Naif Radi Aljohani
Date: 2024-07-10 00:00:00
Proceeding: Frontiers in Artificial Intelligence 2024 7
Tasks: Text Simplification, Text Summarization

Simplifying summaries of scholarly publications has been a popular method for conveying scientific discoveries to a broader audience. While text summarization aims to shorten long documents, simplification seeks to reduce the complexity of a document. To accomplish these tasks collectively, there is a need to develop machine learning methods to shorten and simplify longer texts. This study presents a new Simplification Aware Text Summarization model (SATS) based on future n-gram prediction. The proposed SATS model extends ProphetNet, a text summarization model, by enhancing the objective function using a word frequency lexicon for simplification tasks. We have evaluated the performance of SATS on a recently published text summarization and simplification corpus consisting of 5,400 scientific article pairs. Our results in terms of automatic evaluation demonstrate that SATS outperforms state-of-the-art models for simplification, summarization, and joint simplification-summarization across two datasets on ROUGE, SARI, and CSS1. We also provide human evaluation of summaries generated by the SATS model. We evaluated 100 summaries from eight annotators for grammar, coherence, consistency, fluency, and simplicity. The average human judgment for all evaluated dimensions lies between 4.0 and 4.5 on a scale from 1 to 5 where 1 means low and 5 means high.
xx
Authors: x
Date: 2222-12-22 00:00:00
Proceeding: ww 2222 12

xx
Contrastive Multi View Textual Visual Encoding: Towards One Hundred Thousand Scale One Shot Logo Identification
Authors: Nakul Sharma, Abhirama Subramanyam Penamakuri, Anand Mishra
Date: 2023-05-12 00:00:00
Proceeding: ICVGIP 2023 5
Tasks: Logo Recognition

In this paper, we study the problem of identifying logos of business brands in natural scenes in an open-set one-shot setting. This problem setup is significantly more challenging than traditionally-studied ‘closed-set’ and ‘large-scale training samples per category’ logo recognition settings. We propose a novel multi-view textual-visual encoding framework that encodes text appearing in the logos as well as the graphical design of the logos to learn robust contrastive representations. These representations are jointly learned for multiple views of logos over a batch and thereby they generalize well to unseen logos. We evaluate our proposed framework for cropped logo verification, cropped logo identification, and end-to-end logo identification in natural scene tasks; and compare it against state-of-the-art methods. Further, the literature lacks a ‘very-large-scale’ collection of reference logo images that can facilitate the study of one-hundred thousand-scale logo identification. To fill this gap in the literature, we introduce Wikidata Reference Logo Dataset (WiRLD), containing logos for 100K business brands harvested from Wikidata. Our proposed framework that achieves an area under the ROC curve of 91.3% on the QMUL-OpenLogo dataset for the verification task, outperforms state-of-the-art methods by 9.1% and 2.6% on the one-shot logo identification task on the Toplogos-10 and the FlickrLogos32 datasets, respectively. Further, we show that our method is more stable compared to other baselines even when the number of candidate logos is on a 100K scale.
Vincenzo De Rosa
Authors: Charles kp
Date: 2023-10-18 00:00:00
Proceeding: 10/18 2023 10

Early Life
Vincenzo De Rosa è un imprenditore dinamico e visionario fondatore e CEO di Shurity Italia.

CAREER
Vincenzo De Rosa è un imprenditore dinamico e il visionario fondatore e CEO di Shurity Italia. Originario di Napoli e attualmente residente a Latina, Vincenzo incarna lo spirito di innovazione e determinazione. Il suo viaggio è iniziato in un negozio di telefonia a conduzione familiare, dove ha trascorso anni ad affinare il suo acume negli affari prima di intraprendere un percorso di trasformazione. Ispirato dalla convinzione che "non puoi avere 1000 idee e non realizzarne nemmeno una", ha fatto il grande passo per fondare la sua azienda.

In soli due anni dalla sua fondazione, Shurity Italia ha ottenuto un notevole successo, generando un fatturato di 2 milioni di euro. Vincenzo ha iniziato con umili origini, operando in soli 15 metri quadrati in un ufficio improvvisato. Con l'aumento della domanda, si è rapidamente espanso in un magazzino di 50 metri quadrati e poi in una struttura di 400 metri quadrati, a dimostrazione della rapida crescita della sua azienda. Oggi, Shurity vanta un team di circa 10 dipendenti, tutti dedicati a promuovere la missione aziendale.

Shurity Italia è specializzata in soluzioni ecologiche per il trattamento dell'acqua, offrendo una gamma di prodotti tra cui filtri, purificatori, addolcitori e miscelatori. I loro sistemi innovativi non sono solo popolari tra i consumatori, ma sono stati anche installati nelle case di importanti influencer italiani, evidenziando un impegno per la sostenibilità.

Il percorso imprenditoriale di Vincenzo è caratterizzato dalla sua intraprendenza: ha avviato l'azienda con soli 3.000 euro e ha scelto di reinvestire i profitti anziché cercare prestiti bancari. Guardando al futuro, è pronto ad ampliare l'offerta di Shurity con l'introduzione di due nuove divisioni: Shurity Lab, focalizzata su incisioni su vetro personalizzate e stampa a caldo, e Shurity Faucet Manufacturing, dedicata alla progettazione e produzione di rubinetti per il trattamento dell'acqua. Inoltre, ha in programma di lanciare Shurity Service, una piattaforma rivolta a professionisti e installatori nel settore della depurazione dell'acqua.

Social media platform

Linkedin

Instagram

Official website

[www.vincenzoderosa.com]
OKM: une extension des k-moyennes pour la recherche de classes recouvrantes.
Authors: Guillaume Cleuziou
Date: 2010-03-22 00:00:00
Proceeding: EGC 2010 3
Tasks: Classification

Dans cet article nous abordons le problème de la classification (ou clustering) dans le but de découvrir des classes avec recouvrements. Malgré quelques avancées récentes dans ce domaines, motivées par des besoins applicatifs importants (traitements des données multimédia par exemple), nous constatons l’absence de solutions théoriques à ce problème. Notre étude consiste alors à proposer une nouvelle formulation du problème de classification par partitionnement, adaptée à la recherche d’un recouvrement des données en classes d’objets similaires. Cette approche se fonde sur la définition d’un critère objectif de qualité d’un recouvrement et d’une solution algorithmique visant à optimiser ce critère. Nous proposons deux évaluations de ce travail permettant d’une part d’appréhender le fonctionnement global de l’algorithme sur des données simples (vitesse de convergence, visualisation des résultats) et d’autre part d’évaluer quantitativement le bénéfice d’une telle approche sur une application de classification de documents textuels.
A 1.2 mm$^2$ 416 mW 1.44 Mmat/s 64$\times$16 Matrix Preprocessing ASIC for Massive MIMO in 22FDX
Authors: Darja Nonaca, Christoph Studer
Date: 2024-10-17 00:00:00

Massive multiuser (MU) multiple-input multiple-output (MIMO) enables concurrent transmission of multiple users to a multi-antenna basestation (BS). To detect the users' data using linear equalization, the BS must perform preprocessing, which requires, among other tasks, the inversion of a matrix whose dimension equals the number of user data streams. Explicit inversion of large matrices is notoriously difficult to implement due to high complexity, stringent data dependencies that lead to high latency, and high numerical precision requirements. We propose a novel preprocessing architecture based on the block-LDL matrix factorization, which improves parallelism and, hence, reduces latency. We demonstrate the effectiveness of our architecture through (i) massive MU-MIMO system simulations with mmWave channel vectors and (ii) measurements of a 22FDX ASIC, which is, to our knowledge, the first fabricated preprocessing engine for massive MU-MIMO with 64 BS antennas and 16 single-antenna users. Our ASIC reaches a clock frequency of 870 MHz while consuming 416 mW. At its peak throughput, the ASIC preprocesses 1.44 M 64$\times$16 matrices per second at a latency of only 0.7 $\mu$s.