Reviewerly/exHarmony huggingface dataset:
AutoOpt: Automatic Hyperparameter Scheduling and Optimization for Deep Click-through Rate Prediction

Click-through Rate (CTR) prediction is essential for commercial recommender systems. Recently, to improve the prediction accuracy, plenty of deep learning-based CTR models have been proposed, which are sensitive to hyperparameters and difficult to optimize well. General hyperparameter optimization methods fix these hyperparameters across the entire model training and repeat them multiple times. This trial-and-error process not only leads to suboptimal performance but also requires non-trivial computation efforts. In this paper, we propose an automatic hyperparameters scheduling and optimization method for deep CTR models, AutoOpt, making the optimization process more stable and efficient. Specifically, the whole training regime is firstly divided into several consecutive stages, where a data-efficient model is learned to model the relation between model states and prediction performance. To optimize the stage-wise hyperparameters, AutoOpt uses the global and local scheduling modules to propose proper hyperparameters for the next stage based on the training in the current stage. Extensive experiments on three public benchmarks are conducted to validate the effectiveness of AutoOpt. Moreover, AutoOpt has been deployed onto an advertising platform and a music platform, where online A/B tests also demonstrate superior improvement. In addition, the code of our algorithm is publicly available in MindSpore1.
An Analysis of Susceptibility to Phishing via Business Chat through Online Survey

Phishing via email has been spreading not only to individuals but also to companies, and various studies have been conducted on it. In addition, the use of chat has been increasing, and phishing via chat is assumed to be occurring, but the risk and susceptibility to phishing via chat have not been clarified. In this study, we conducted a questionnaire survey of 304 participants. In the survey, we divided the participants into two groups, designed similar phishing scenarios using email and chat, and conducted a role play susceptibility study. The results of the study indicated that the susceptibility of chat was as high as or higher than that of email, suggesting that phishing via chat is risky. On the basis of the results, we also summarized methods for reducing the risk of phishing in chat and recommendations for future research.
Evaluating The Effects of Calibrated Popularity Bias Mitigation: A Field Study

Despite their proven various benefits, Recommender Systems can cause or amplify certain undesired effects. In this paper, we focus on Popularity Bias, i.e., the tendency of a recommender system to utilize the effect of recommending popular items to the user. Prior research has studied the negative impact of this type of bias on individuals and society as a whole and proposed various approaches to mitigate this in various domains. However, almost all works adopted offline methodologies to evaluate the effectiveness of the proposed approaches. Unfortunately, such offline simulations can potentially be rather simplified and unable to capture the full picture. To contribute to this line of research and given a particular lack of knowledge about how debiasing approaches work not only offline, but online as well, we present in this paper the results of user study on a national broadcaster movie streaming platform in Norway, i.e., TV 2, following the A/B testing methodology. We deployed an effective mitigation approach for popularity bias, called Calibrated Popularity (CP), and monitored its performance in comparison to the platform’s existing collaborative filtering recommendation approach as a baseline over a period of almost four months. The results obtained from a large user base interacting in real-time with the recommendations indicate that the evaluated debiasing approach can be effective in addressing popularity bias while still maintaining the level of user interest and engagement.
Distribution-based Learnable Filters with Side Information for Sequential Recommendation

Sequential Recommendation aims to predict the next item by mining out the dynamic preference from user previous interactions. However, most methods represent each item as a single fixed vector, which is incapable of capturing the uncertainty of item-item transitions that result from time-dependent and multifarious interests of users. Besides, they struggle to effectively exploit side information that helps to better express user preferences. Finally, the noise in user’s access sequence, which is due to accidental clicks, can interfere with the next item prediction and lead to lower recommendation performance. To deal with these issues, we propose DLFS-Rec, a simple and novel model that combines Distribution-based Learnable Filters with Side information for sequential Recommendation. Specifically, items and their side information are represented by stochastic Gaussian distribution, which is described by mean and covariance embeddings, and then the corresponding embeddings are fused to generate a final representation for each item. To attenuate noise, stacked learnable filter layers are applied to smooth the fused embeddings. Extensive experiments on four public real-world datasets demonstrate the superiority of the proposed model over state-of-the-art baselines, especially on cold start users and items. Codes are available at https://github.com/zxiang30/DLFS-Rec.
BehavRec: Workshop on Recommendations for Behavior Change

The workshop aims to discuss open problems, challenges, and innovative research approaches in the area of persuasive and behavior change recommender systems, that is, recommender systems aimed at modifying people's habits and behavior. Some questions that motivate this workshop are: What kind of theory is more suitable to inform the design of behavior change recommender systems? What kind of personal data (e.g., coming from environmental sensors, wearable devices, etc.) should we use to design behavior change recommendations? How should we deliver them (i.e., what kind of communication channels and interfaces should we use)? What kind of strategies should we implement to design timely and contextualized recommendations? How can we support the user's motivation to adhere to the recommendations provided? How can we “persuade” users in the long term?
Investigating the effects of incremental training on neural ranking models

Recommender systems are an essential component of online platforms providing users with personalized experiences. Some recommendation scenarios such as social networks and news are extremely dynamic in nature with user interests changing over time and new items being continuously added due to breaking news and trending events.
Reward innovation for long-term member satisfaction

Recommender systems commonly train on user engagements because of their abundance, immediacy of feedback, and the insights they provide into users preferences. However, this approach may unintentionally prioritize optimizing short-term engagements over a product’s or business’s long-term objectives. At Netflix, our recommender systems are designed with the goal of maximizing long-term member satisfaction. To achieve this objective, we adopt a practical approach that augments engagement data with reward signals aligned with long term member satisfaction. This process of identifying, evaluating, and integrating reward signals into an existing learning algorithm is what we term reward innovation. In this work, we present the challenges of applying this approach to a large-scale recommender system and share our approach to addressing them.
Improving Recommender Systems Through the Automation of Design Decisions

Recommender systems developers are constantly faced with difficult design decisions. Additionally, the number of options that a recommender systems developer has to consider continually grows over time with new innovations. The machine learning community is in a similar situation and has come together to tackle the problem. They invented concepts and tools to make machine learning development both easier and faster. These developments are categorized as automated machine learning (AutoML). As a result, the AutoML community formed and continuously innovates new approaches. Inspired by AutoML, the recommender systems community has recently understood the need for automation and sparsely introduced AutoRecSys. The goal of AutoRecSys is not to replace recommender systems developers but to improve performance through the automation of design decisions. With AutoRecSys, recommender systems engineers do not have to focus on easy but time-consuming tasks and are free to pursue difficult engineering tasks instead. Additionally, AutoRecSys enables easier access to recommender systems for beginners as it reduces the amount of knowledge required to get started with the development of recommender systems. AutoRecSys, like AutoML, is still early in its development and does not yet cover the whole development pipeline. Additionally, it is not yet clear, under which circumstances AutoML approaches can be transferred to recommender systems. Our research intends to close this gap by improving AutoRecSys both with regard to the transfer of AutoML and novel approaches. Furthermore, we focus specifically on the development of novel automation approaches for data processing and training. We note that the realization of AutoRecSys is going to be a community effort. Our part in this effort is to research AutoRecSys fundamentals, build practical tools for the community, raise awareness of the advantages of automation, and catalyze AutoRecSys development.
Multiple Connectivity Views for Session-based Recommendation

Session-based recommendation (SBR), which makes the next-item recommendation based on previous anonymous actions, has drawn increasing attention. The last decade has seen multiple deep learning-based modeling choices applied on SBR successfully, e.g., recurrent neural networks (RNNs), convolutional neural networks (CNNs), graph neural networks (GNNs), and each modeling choice has its intrinsic superiority and limitation. We argue that these modeling choices differentiate from each other by (1) the way they capture the interactions between items within a session and (2) the operators they adopt for composing the neural network, e.g., convolutional operator or self-attention operator.
Poster: Enriching people’s experience at CH sites with IoT technology

The Internet of Things (IoT) technology is valuable for connecting physical objects to the digital world. IoT-based systems have the potential to enrich people’s experiences with CH collections; indeed, sensors and other data-collecting devices embedded into tangible artifacts can capture real-time information about visitors’ interactions with CH objects and permit the creation of customized narratives and experiences based on visitors’ interests, preferences, and prior knowledge. To identify the strengths and weaknesses of IoT solutions and the factors that impact on the experiences of visitors of CH sites, we created an IoT-based game called Magic Torch and performed an experimental study involving a total of 18 visitors, in order to analyze whether and how this game affects the experience of museum visitors. Its results show that the use of IoT positively impacts visitors’ emotions and overall experience; they also indicate some drawbacks.
Demo: MetaCUX - a multi-user, multi-scenario environment for a cooperative workspace

We present MetaCUX - Cooperative User eXperience in the Metaverse, a multi-user, multi-scenario environment that simulates a business reality through the reproduction of five characteristic scenarios and interactive objects through the use of the Meta Quest 2 device and virtual avatars. In this demo, we show a demonstration of the system involving all 5 scenarios created. The demo will allow participants to experiment collaboration in simulated real situations.
A Model-Agnostic Framework for Recommendation via Interest-aware Item Embeddings

Item representation holds significant importance in recommendation systems, which encompasses domains such as news, retail, and videos. Retrieval and ranking models utilise item representation to capture the user-item relationship based on user behaviours. While existing representation learning methods primarily focus on optimising item-based mechanisms, such as attention and sequential modelling. However, these methods lack a modelling mechanism to directly reflect user interests within the learned item representations. Consequently, these methods may be less effective in capturing user interests indirectly. To address this challenge, we propose a novel Interest-aware Capsule network (IaCN) recommendation model, a model-agnostic framework that directly learns interest-oriented item representations. IaCN serves as an auxiliary task, enabling the joint learning of both item-based and interest-based representations. This framework adopts existing recommendation models without requiring substantial redesign. We evaluate the proposed approach on benchmark datasets, exploring various scenarios involving different deep neural networks, behaviour sequence lengths, and joint learning ratios of interest-oriented item representations. Experimental results demonstrate significant performance enhancements across diverse recommendation models, validating the effectiveness of our approach.
ReCon: Reducing Congestion in Job Recommendation using Optimal Transport

Recommender systems may suffer from congestion, meaning that there is an unequal distribution of the items in how often they are recommended. Some items may be recommended much more than others. Recommenders are increasingly used in domains where items have limited availability, such as the job market, where congestion is especially problematic: Recommending a vacancy—for which typically only one person will be hired—to a large number of job seekers may lead to frustration for job seekers, as they may be applying for jobs where they are not hired. This may also leave vacancies unfilled and result in job market inefficiency.
What We Evaluate When We Evaluate Recommender Systems: Understanding Recommender Systems’ Performance using Item Response Theory

Current practices in offline evaluation use rank-based metrics to measure the quality of top-n recommendation lists. This approach has practical benefits as it centres assessment on the output of the recommender system and, therefore, measures performance from the perspective of end-users. However, this methodology neglects how recommender systems more broadly model user preferences, which is not captured by only considering the top-n recommendations. In this article, we use item response theory (IRT), a family of latent variable models used in psychometric assessment, to gain a comprehensive understanding of offline evaluation. We use IRT to jointly estimate the latent abilities of 51 recommendation algorithms and the characteristics of 3 commonly used benchmark data sets. For all data sets, the latent abilities estimated by IRT suggest that higher scores from traditional rank-based metrics do not reflect improvements in modeling user preferences. Furthermore, we show that the top-n recommendations with the most discriminatory power are biased towards lower difficulty items, leaving much room for improvement. Lastly, we highlight the role of popularity in evaluation by investigating how user engagement and item popularity influence recommendation difficulty.
Poster: Human Presence Detection After Earthquakes: An AI-Based Implicit User Interface on the Smartphone

One of the main challenges of rescue operations after a devastating earthquake is the timely location of people trapped under debris. We propose a system that exploits the smartphone to detect the presence and implicitly interact with a person trapped in buildings. It leverages the phone microphone to detect sound waves generated by human breathing, heartbeat, and movement. It analyzes the signals on the smartphone itself using deep learning. A server collecting the results can support search-and-rescue operations or trigger further actions, such as an emergency call. The results of a preliminary evaluation of such a system based on a proof-of-concept Android app demonstrate an accurate detection of human presence within a specific range of the smartphone.
Plug-and-Play Attribute-Aware Text Infilling via A New Attention Mechanism and Two-Level Positional Encoding

Text infilling aims to restore incomplete texts by filling in blanks and has attracted increasing attention recently because of its wide application in ancient text restoration, conversation generation, and text rewriting. However, attribute-aware text infilling is yet to be explored, and existing methods seldom focus on the infilling length of each blank or the number and location of the blanks. In this study, we propose a plug-and-play Attribute-aware Text Infilling method using a Pre-trained language model (A-TIP) that contains a text-infilling component and a plug-and-play discriminator. Specifically, we first designed a unified text-infilling component with modified attention mechanisms and intra- and inter-blank positional encoding to better perceive the number of blanks and the infilling length for each blank. We then propose a plug-and-play discriminator to guide generation and improve attribute relevance without decreasing text fluency. Finally, automatic and human evaluations on three open-source datasets indicate that, compared to all the baselines, A-TIP achieves state-of-the-art performance. An additional ablation study demonstrated the robustness of A-TIP.
Container-native Managed Data Sharing

Cross-industrial collaboration can yield disruptive innovations. However, there are fears regarding data sharing across different organizations. Even if data providers make contracts covering the data shared with their consumers, they will not be able to delete the shared data in accordance with the expiration dates and modify them after they were delivered to the consumers. Data consumers need to be extremely careful about management of shared data since huge penalties are imposed against violations of data protection laws. To allay these fears, we propose a system to handle external data management instead of data providers and consumers. In our system, containerized lifetime controllers delete expired shared data in accordance with contracts for shared data called life cycle policies. To allow only service programs stipulated in the policies to manipulate shared data, containerized volume controllers enforce the access control on the basis of the FUSE interceptions and the /proc file system. The proposed system is transparent to service programs because the containerized controllers run in execution environments that are separate from service programs. The proposed system can be applied to multiple container orchestration clusters in which a provider and consumer independently administer Kubernetes container orchestrators, as well as a single container orchestration cluster. We built a prototype system on Kubernetes container orchestrators presented by the Kubernetes community and public cloud service providers. Experimental results demonstrate that the proposed system achieves data sharing between a provider and consumer with moderate overheads for disk consumption of the containerized controllers, the extensions of the volume drivers, and execution time of the FUSE access control.
Knowledge-based Multiple Adaptive Spaces Fusion for Recommendation

Since Knowledge Graphs (KGs) contain rich semantic information, recently there has been an influx of KG-enhanced recommendation methods. Most of existing methods are entirely designed based on euclidean space without considering curvature. However, recent studies have revealed that a tremendous graph-structured data exhibits highly non-euclidean properties. Motivated by these observations, in this work, we propose a knowledge-based multiple adaptive spaces fusion method for recommendation, namely MCKG. Unlike existing methods that solely adopt a specific manifold, we introduce the unified space that is compatible with hyperbolic, euclidean and spherical spaces. Furthermore, we fuse the multiple unified spaces in an attention manner to obtain the high-quality embeddings for better knowledge propagation. In addition, we propose a geometry-aware optimization strategy which enables the pull and push processes benefited from both hyperbolic and spherical spaces. Specifically, in hyperbolic space, we set smaller margins in the area near to the origin, which is conducive to distinguishing between highly similar positive items and negative ones. At the same time, we set larger margins in the area far from the origin to ensure the model has sufficient error tolerance. The similar manner also applies to spherical spaces. Extensive experiments on three real-world datasets demonstrate that the MCKG has a significant improvement over state-of-the-art recommendation methods. Further ablation experiments verify the importance of multi-space fusion and geometry-aware optimization strategy, justifying the rationality and effectiveness of MCKG.
Challenges for Anonymous Session-Based Recommender Systems in Indoor Environments

In the last two decades, recommender systems have become more popular since they can provide personalized recommendations in different fields. However, the current research landscape in this area suggests that there is still considerable potential for applying novel recommendation techniques in indoor environments. In addition, the growing attention to privacy raises even more challenges. Anonymous session-based recommender systems represent attractive solutions in this scenario, given their natural predisposition to model the indoor domain by treating each visit to a particular location as an anonymous session. This paper presents some noteworthy challenges regarding several aspects related to the application of these models in indoor environments. We expose our research questions on issues related to the representation of user behavior, cold-start problem, and fairness. Although these problems affect any RS, they become even more challenging in the chosen environment. Finally, we outline a possible use case in a real application scenario to make more transparent and concrete the line of research we intend to pursue in the near future.
Poster: LLMs for online customer reviews analysis: oracles or tools? Experiments with GPT 3.5

Generative Large Language Models, pre-trained on a huge amount of human authored text, are showing emergent capabilities in understanding and accomplishing a variety of NLP and text comprehension tasks. Recently, interest is growing in understanding to what extent LLMs can support humans, or even replace them, in accomplishing non trivial data analysis tasks.