OpenRaiser/Intern-Atlas huggingface dataset:
A Mixture of Experts in Forecasting Student Performance in Classroom Programming Activities
Authors: Moqsadur Rahman, Monika Akbar, Justice T. Walker, Mahmud Shahriar Hossain
Venue: CIKM
Year: 2024

Predicting students' performance early in programming courses is crucial because it allows instructors to intervene early, improving learning outcomes. Currently, no existing platforms can effectively forecast student performance in programming activities based on students' developed code. Forecasting student scores based on their programming activities is challenging because the accuracy of different predictive models often varies throughout these activities. To address this challenge, we introduce a novel framework utilizing Mixture of Experts (MoE). The MoE method combines insights from various neural networks and dynamically picks the most accurate predictions. This system significantly enhances the reliability of forecasting each student's performance within the first 15 minutes of a 30-minute programming session. By enabling early predictions, the MoE provides instructors with a powerful mechanism to understand and support the student learning process in real-time.
Compressed Models are NOT Miniature Versions of Large Models
Authors: Rohit Raj Rai, Rishant Pal, Amit Awekar
Venue: CIKM
Year: 2024

Large neural models are often compressed before deployment. Model compression is necessary for many practical reasons, such as inference latency, memory footprint, and energy consumption. Compressed models are assumed to be miniature versions of corresponding large neural models. However, we question this belief in our work. We compare compressed models with corresponding large neural models using four model characteristics: prediction errors, data representation, data distribution, and vulnerability to adversarial attack. We perform experiments using the BERT-large model and its five compressed versions. For all four model characteristics, compressed models significantly differ from the BERT-large model. Even among compressed models, they differ from each other on all four model characteristics. Apart from the expected loss in model performance, there are major side effects of using compressed models to replace large neural models.
No Query Left Behind: Query Refinement via Backtranslation
Authors: Delaram Rajaei, Zahra Taheri, Hossein Fani
Venue: CIKM
Year: 2024

Query refinement is to enhance the relevance of search results by modifying users' original queries to refined versions. State-of-the-art query refinement models have been trained on web query logs, which are predisposed to topic drifts. To fill the gap, little work has been proposed to generate benchmark datasets of (query ’ refined query) pairs through an overwhelming application of unsupervised or supervised modifications to the original query while controlling topic drifts. In this paper, however, we propose leveraging natural language backtranslation, a round-trip translation of a query from a source language via target languages, as a simple yet effective unsupervised approach to scale up generating gold-standard benchmark datasets. Backtranslation can (1) uncover terms that are omitted in a query for being commonly understood in a source language, but may not be known in a target language (e.g., 'figs'’(tamil) 'in a target language (e.g., ‘figs’→(tamil) ‘அத்திமரங்கள்’→‘the fig trees’), (2) augment a query with context-aware synonyms in a target language (e.g., ‘italian nobel prize winners’→(farsi) ’برنده های ایتالیایی جایزه نوبل‘ →‘italian nobel laureates’, and (3) help with the semantic disambiguation of polysemous terms and collocations (e.g., 'custer's last stand'’(malay)`pertahan terakhir custer'’`custer's last defence'. Our experiments across 5 query sets with different query lengths and topics and 10 languages from 7 language families using 2 neural machine translators validated the effectiveness of query backtranslation in generating a more extensive gold-standard dataset for query refinement. We open-sourced our research at https://github.com/fani-lab/RePair/tree/nqlb.
Low Carbon Footprint Training for 1D-CNNs with Temporal Max-Pooling
Authors: Anandharaju Durai Raju, Ke Wang
Venue: CIKM
Year: 2024

Training convolutional neural networks (CNNs) demands huge GPU memory consumption and training time, leading to increased carbon emissions, and impacting sustainability. In this paper, we propose HotConv, a low GPU memory and low carbon footprint learning strategy for training the class of 1D CNNs that have a temporal max-pooling layer. Such CNNs are widely used in various domains for learning large-sized inputs, including genomics and malware detection. HotConv reduces the GPU memory usage of such CNNs by harnessing the sparsity of relevant activations and gradients at the temporal max-pooling layer, which produces the same model as the full computation of activations and gradients, without trading-off model performance. Evaluations using the public benchmark BODMAS and VirusTotal datasets for malware detection with HotConv applied to the public MalConv network architecture show that the carbon footprint reduction using HotConv is superior to existing approaches. For instance, HotConv uses only 1/22 of the GPU memory used by MalConv2 - the memory-efficient variant of MalConv, while also consuming less training time than MalConv2. This is equivalent to reducing the carbon footprint up to 1/4 of that of MalConv2 without compromising performance.
Periormer: Periodic Transformer for Seasonal and Irregularly Sampled Time Series
Authors: Xiaobin Ren, Kaiqi Zhao, Katerina Taskova, Patricia Riddle, Lianyan Li
Venue: CIKM
Year: 2024

Time series prediction presents a significant challenge across various domains, such as transportation systems, environmental science, and multiple industrial sectors. Real-world time series data commonly exhibit periodic patterns and irregular sampling rates. Recent advancements in long sequence time series forecasting have made significant progress in adopting deep neural networks, particularly the Transformers, renowned for their robust representational capabilities. However, current Transformer-based models consider time steps as discrete tokens, thereby failing to account for periodicity and temporal intervals when selecting relevant time steps in the past. To address this limitation, we propose an end-to-end framework called Periormer for forecasting irregularly sampled time series. Periormer comprises three key components: (1) a novel input embedding layer that encodes the periodicity and time interval information, analogous to positional encoding in Transformers; (2) a feature-wise periodic attention mechanism that selects essential data points considering the periods and amplitudes of the periodic signals; and (3) a cross-feature periodic attention mechanism that identifies essential features relevant to the prediction. Experiments on four real-world datasets and one synthetic dataset demonstrate that Periormer reduces the mean squared error by 14.9% compared to state-of-the-art models.
Generative AI for Energy: Multi-Horizon Power Consumption Forecasting using Large Language Models
Authors: Kevin Roitero, Gianluca D&apos, Abrosca, Andrea Zancola, Vincenzo Della Mea, Stefano Mizzaro
Venue: CIKM
Year: 2024

We leverage generative NLP-based models, specifically Transformer-Based models, for multi-horizon univariate and multivariate power consumption forecasting. We apply our approach to various datasets, focusing on short-term (1 day) and long-term (1 week) forecasts. We test several lag configurations with and without additional contextual information and achieve promising results. We evaluate the forecasts' effectiveness using a range of metrics, and aggregate the results on a monthly basis for a comprehensive understanding of the performance throughout the year.
Relevance Filtering for Embedding-based Retrieval
Authors: Nicholas Rossi, Juexin Lin, Feng Liu, Zhen Yang, Tony Lee, Alessandro Magnani, Ciya Liao
Venue: CIKM
Year: 2024

In embedding-based retrieval, Approximate Nearest Neighbor (ANN) search enables efficient retrieval of similar items from large-scale datasets. While maximizing recall of relevant items is usually the goal of retrieval systems, a low precision may lead to a poor search experience. Unlike lexical retrieval, which inherently limits the size of the retrieved set through keyword matching, dense retrieval via ANN search has no natural cutoff. Moreover, the cosine similarity scores of embedding vectors are often optimized via contrastive or ranking losses, which make them difficult to interpret. Consequently, relying on top-K or cosine-similarity cutoff is often insufficient to filter out irrelevant results effectively. This issue is prominent in product search, where the number of relevant products is often small. This paper introduces a novel relevance filtering component (called "Cosine Adapter") for embedding-based retrieval to address this challenge. Our approach maps raw cosine similarity scores to interpretable scores using a query-dependent mapping function. We then apply a global threshold on the mapped scores to filter out irrelevant results. We are able to significantly increase the precision of the retrieved set, at the expense of a small loss of recall. The effectiveness of our approach is demonstrated through experiments on both public MS MARCO dataset and internal Walmart product search data. Furthermore, online A/B testing on the Walmart site validates the practical value of our approach in real-world e-commerce settings.
AnnoRank: A Comprehensive Web-Based Framework for Collecting Annotations and Assessing Rankings
Authors: Clara Rus, Gabrielle Poerwawinata, Andrew Yates, Maarten de Rijke
Venue: CIKM
Year: 2024

We present AnnoRank, a web-based user interface (UI) framework designed to facilitate collecting crowdsource annotations in the context of information retrieval. AnnoRank enables the collection of explicit and implicit annotations for a specified query and a single or multiple documents, allowing for the observation of user-selected items and the assignment of relevance judgments. Furthermore, AnnoRank allows for ranking comparisons, allowing for the visualization and evaluation of a ranked list generated by different fairness interventions, along with its utility and fairness metrics. Fairness interventions in the annotation pipeline are necessary to prevent the propagation of bias when a user selects the top-k items in a ranked list. With the widespread use of ranking systems, the application supports multimodality through text and image document formats. We also support the assessment of agreement between annotators to ensure the quality of the annotations. AnnoRank is integrated with the Ranklib library, offering a vast range of ranking models that can be applied to the data and displayed in the UI. AnnoRank is designed to be flexible, configurable, and easy to deploy to meet diverse annotation needs in information retrieval. AnnoRank is publicly available as open-source software, together with detailed documentation at https://github.com/ClaraRus/AnnoRank.
Product Retrieval and Ranking for Alphanumeric Queries
Authors: Hadeel Saadany, Swapnil Bhosale, Samarth Agrawal, Zhe Wu, Constantin Orasan, Diptesh Kanojia
Venue: CIKM
Year: 2024

This talk addresses the challenge of improving user experience on e-commerce platforms by enhancing product ranking relevant to user's search queries. Queries such as S2716DG consist of alphanumeric characters where a letter or number can signify important detail for the product/model. Speaker describes recent research where we curate samples from existing datasets at eBay, manually annotated with buyer-centric relevance scores, and centrality scores which reflect how well the product title matches the user's intent. We introduce a User-intent Centrality Optimization (UCO) approach for existing models, which optimizes for the user intent in semantic product search. To that end, we propose a dual-loss based optimization to handle hard negatives, i.e., product titles that are semantically relevant but do not reflect the user's intent. Our contributions include curating a challenging evaluation set and implementing UCO, resulting in significant improvements in product ranking efficiency, observed for different evaluation metrics. Our work aims to ensure that the most buyer-centric titles for a query are ranked higher, thereby, enhancing the user experience on e-commerce platforms.
EFfECT-RL: Enabling Framework for Establishing Causality and Triggering engagement through RL
Authors: Debanjan Sadhukhan, Deepanshi Seth, Sanjay Agrawal, Tridib Mukherjee
Venue: CIKM
Year: 2024

Skill-based games offer an exceptional avenue for entertainment, fostering self-esteem, relaxation, and social satisfaction. Engagement in online skill gaming platforms is however heavily dependent on the outcomes and experience (e.g., wins/losses). Understanding the factors driving increased engagement is crucial within skill gaming platforms. In this study, we aim to address two key questions: (1) "What factors are driving users to increase their engagement?" and (2) "How can we personalize users journey accordingly to further optimize their engagement?". In skill gaming platforms, the impact of causal relationships often manifests with a delay, which varies significantly as users? personas evolve. Without a detailed information on treatments (such as timing and frequency), estimating the impact of a causal-treatment-effect in a highly volatile game-play data becomes exceedingly challenging. This work proposes a framework called EFfECT-RL that establishes causal discovery by integrating change-point detection and explainable K-means clustering, while leveraging users' game-play and transactional-data. Unlike existing methods which were unable to detect causal-effects in extremely volatile-data, EFfECT-RL generates threshold-trees (~ 79% accuracy) elucidating causal-relationships. Once the causal relationship is established, we personalize treatments by developing a novel offline deep reinforcement learning-based approach. Our online recommendations show a 3% improvement in user engagement (platform-centric) with 70% relevancy (user-centric).
SmartHash: Perceptual Hashing for Image Tampering Detection and Authentication
Authors: Priyanka Samanta, Shweta Jain
Venue: CIKM
Year: 2024

Perceptual hashing algorithms have been used extensively to detect duplicate images, similar images for reverse image search, inappropriate and explicit images, and child sexual abuse (CSAM) images. These algorithms use various techniques to extract perceptual features from an image to create a succinct representation called the hash which is akin to the bio-metric marker of an image. This paper explores the ability of perceptual hashes to determine whether an image is a tampered version of a previously known image, i.e., it was created by object addition or removal from the original known image. In particular, a fast and efficient DCT-based perceptual hashing algorithm called SmartHash is proposed. SmartHash is an extension of a well-known and widely used pHash algorithm. It is evaluated on several publicly available datasets of tampered images and is shown to have high accuracy and precision in detecting such images. Additionally, an in-depth examination of the results is provided to quantify the limitations and hence, operating parameters of any software that integrates SmartHash in its workflow. Comparison with the state of the art Apple's NeuralHash and Microsoft's PhotoDNA is provided in the context of detecting tampered images. This work contributes to the content authentication initiative which is led by Adobe to establish content provenance and authentication across the Internet.
EDGE: Evaluation Framework for Logical vs. Subgraph Explanations for Node Classifiers on Knowledge Graphs
Authors: Rupesh Sapkota, Dominik Köhler, Stefan Heindorf
Venue: CIKM
Year: 2024

As machine learning and deep learning become increasingly integrated into our daily lives, understanding how these technologies make decisions is crucial. To ensure transparency, accountability, and ethical adherence, these so-called "black-box" models should be accompanied by human-comprehensible explanations of their predictions. This clarity is essential for establishing trust in their real-world applications. Similarly, it is crucial to compare different types of explanations to evaluate and understand their effectiveness, interpretability, and generalization capabilities for informed selection in various applications. To this end, we propose a framework called EDGE to evaluate diverse knowledge graph explanations, assessing logical rule-based and subgraph-based explanations by various explainers in terms of prediction accuracy and fidelity to the Graph Neural Network (GNN) model. Our evaluations reveal that logical methods excel in explaining complex and structured data, while subgraph-based models exhibit higher fidelity to the GNN model, earning them the label "GNN Explainers". Although further diversified evaluations are necessary to determine the superiority of one explanation type over another, our study shows that each type has pros and cons.
Mitigating Extreme Cold Start in Graph-based RecSys through Re-ranking
Authors: Alessandro Sbandi, Federico Siciliano, Fabrizio Silvestri
Venue: CIKM
Year: 2024

Recommender systems based on Graph Neural Networks (GNN) have become the state-of-the-art approach in recommendation, but they struggle with in extreme cold-start settings, where most users or items lack interaction data. This paper proposes a novel framework to address this challenge in four steps: (i) a propensity model to predict item purchase behaviour, with associated explainability to identify the most relevant features, (ii) a link augmentation module to connect users based on previously obtained similarities, (iii) a GNN-based link prediction step on the obtained dense graph and (iv) a final re-ranking stage to increase diversity in predictions leveraging users embeddings. By exploiting the enriched graph structure, the framework generates embeddings for cold-start users and items, enabling diverse recommendations, containing long tail and unsold items, for both established and new users. We validate the framework's effectiveness on real-world industrial data from TIM S.p.A.
Leveraging Trustworthy Node Attributes for Effective Network Alignment
Authors: Dong-Hyuk Seo, Jae-Hwan Lim, Won-Yong Shin, Sang-Wook Kim
Venue: CIKM
Year: 2024

With the prevalence of social media platforms, accurately identifying the same users across different networks through network alignment has become crucial. Existing methods often struggle due to sparse or absent user-identifiable information (node attributes), highlighting the need for augmenting node attributes. However, research on attribute augmentation remains largely under-explored. In this study, we aim to design augmented attributes that enhance network alignment by reflecting three key structural C haracteristics: (C1) global structural characteristic, reflects the global network structure; (C2) seed-based structural characteristic, leverages cross-network structural information associated with seed nodes; (C3) multi-aspect structural characteristic, employs diverse structural relationship measures. To this end, we propose a novel approach for designing trustworthy Augmented Seed-baSed and multI-aspect STructurAl iNformaTion (ASSISTANT) attributes. To enhance alignment performance, we also present a learning module that utilizes a gate mechanism to select the most effective measure dynamically. Extensive experiments across various datasets demonstrate the following: 1) Our network alignment framework, which includes a gate mechanism module, significantly outperforms state-of-the-art methods in alignment accuracy; 2) other state-of-the-art methods using ASSISTANT attributes as input substantially boosts their own alignment accuracy; and 3) using only ASSISTANT attributes without any training process also leads to effective alignment, showcasing their high trustworthiness.
Empowering Traffic Speed Prediction with Auxiliary Feature-Aided Dependency Learning
Authors: Dong-Hyuk Seo, Jiwon Son, Namhyuk Kim, Won-Yong Shin, Sang-Wook Kim
Venue: CIKM
Year: 2024

Traffic speed prediction is a crucial task for optimizing navigation systems and reducing traffic congestion. Although there have been efforts to improve the accuracy of speed prediction by incorporating auxiliary features, such as traffic flow, weather, and time, types of auxiliary features are limited and their detailed relationships with speed have not been explored yet. In our study, we present the individual spatio-temporal (IST) dependencies on flow and speed, and characterize three types of IST-dependencies with the flow-to-flow, speed-to-speed, and flow-to-speed graphs. Then, we propose Auxiliary feature-aided Attention Network (ARIAN), a novel approach to judiciously learning the degrees of IST-dependencies with the three graphs and predicting the future speed by leveraging various auxiliary features. Through comprehensive experiments using 3 real-world datasets, we validate the superiority of ARIAN over 10 state-of-the-art methods and the effectiveness of each auxiliary feature and each dependency learner in ARIAN.
Structural Representation Learning and Disentanglement for Evidential Chinese Patent Approval Prediction
Authors: Jinzhi Shan, Qi Zhang, Chongyang Shi, Mengting Gui, Shoujin Wang, Usman Naseem
Venue: CIKM
Year: 2024

Automatic Chinese patent approval prediction is an emerging and valuable task in patent analysis. However, it involves a rigorous and transparent decision-making process that includes patent comparison and examination to assess its innovation and correctness. This resultant necessity of decision evidentiality, coupled with intricate patent comprehension presents significant challenges and obstacles for the patent analysis community. Consequently, few existing studies are addressing this task. This paper presents the pioneering effort on this task using a retrieval-based classification approach. We propose a novel framework called DiSPat, which focuses on structural representation learning and disentanglement to predict the approval of Chinese patents and offer decision-making evidence. DiSPat comprises three main components: base reference retrieval to retrieve the Top-k most similar patents as a reference base; structural patent representation to exploit the inherent claim hierarchy in patents for learning a structural patent representation; disentangled representation learning to learn disentangled patent representations that enable the establishment of an evidential decision-making process. To ensure a thorough evaluation, we have meticulously constructed three datasets of Chinese patents. Extensive experiments on these datasets unequivocally demonstrate our DiSPat surpasses state-of-the-art baselines on patent approval prediction, while also exhibiting enhanced evidentiality.
M2IoU: A Min-Max Distance-based Loss Function for Bounding Box Regression in Medical Imaging
Authors: Anurag Shandilya, Kalash Shah, Bhavik Kanekar, Akshat Gautam, Pavni Tandon, Ganesh Ramakrishnan, Kshitij S. Jadhav
Venue: CIKM
Year: 2024

Computer vision applications such as object detection have increased manifolds in the medical domain for diagnosis and treatment purposes. Generally, object detection models such as YOLO(You Only Look Once) involve identifying the correct bounding box and classifying the objects inside the bounding box. However, medical imaging object detection is a challenging endeavor, requiring models that are both efficient and extremely accurate in the face of limited data and expensive annotations. In this paper, we propose a Min-Max IoU (M2IoU) loss function by introducing a new min-max-based penalty term in the loss equation, between the predicted box and the ground truth coordinates. We further compare the results of several loss functions on the YOLOv8 model trained on multiple medical datasets and demonstrate that the M2IoU loss function leads to faster learning and outperforms other existing loss functions like CIoU and GIoU.