Reviewerly/exHarmony huggingface dataset:
Evaluating Conversational Information Retrieval

This chapter provides a detailed discussion of techniques for evaluating a CIR system—a goal-oriented conversational AI system with a human in the loop. We present two approaches. System-oriented evaluation captures the user’s requests and preferences in a fixed dataset. User-oriented evaluation studies the interactions of a real user with the search system. Eventually, we describe two emerging forms of evaluation: CIR user simulation and responsible CIR.
Budget in the Cloud: Analyzing Cost and Recommending Virtual Machine Workload

Due to the increasing popularity of cloud computing, the cost companies pay to use the cloud and its services are growing significantly. Higher costs of cloud computing technology contribute to the volatility of the cloud and the financial instability of companies that depend on the cloud. Companies have their own unique cloud budget, and staying within that budget can become problematic in light of increasing costs. To save cost, many cloud users look to their cloud provider rather than looking at their own cloud purchases. A lack of studies on cloud users’ virtual machine (VM) workload and how cloud users spend money necessitates analysis of the spending of cloud customers. In this research paper, we analyzed a real-world data set from Microsoft Azure collected in 2019 that includes approximately 2.7 million VM traces. We developed a linear regression based pricing model to calculate the cost and used this model to analyze Microsoft Azure’s VM workload by comparing the cores, memory, lifetime, average utilization, and cost of each trace. By analyzing Microsoft’s data, we observed that users are not fully utilizing the cloud resources they have paid for. With this idea in mind, we then quantified the waste and developed an algorithm to determine which VMs are the most ineffective. We applied our algorithm to Microsoft Azure’s data set, and our results show that our algorithm discovered over one million wasteful VMs and helped 6,600 users save $3.3 million dollars. Even though cloud computing prices are increasing, cloud customers can save significantly by understanding VM workload better and selecting better-fitting VMs.
Automated Web Annotator of Biomedical Entities in Spanish Language

In Natural Language Processing (NLP) and supervised machine learning, the scarcity of labeled corpora results in poor performance of machine learning models. In the medical domain, there are also fewer labeled corpora in Spanish than in English. We propose a method to identify biomedical entities in Spanish-language clinical texts, through automatic translation and word alignment, by translating the source text (Spanish) to the target text (English), then labeling the target text with automatic biomedical entity taggers, and finally projecting the biomedical entities from the target text to the corresponding text sections in the source text by means of word alignment generated in the translation process. This is done with the objective of annotating the source text with English language tools (automatic annotators). As a result, an efficient method capable of processing and annotating biomedical entities in the Spanish language with high precision is obtained, since it integrates several automatic annotators in a single web system.
Query-Focused Multi-document Summarization

This chapter discusses algorithms and methods for query-focused multi-document summarization, which aims at producing a concise summary of a set of documents returned by the document search module in response to a user query.
Case Study of Commercial Systems

In this chapter, we review a variety of commercial systems for CIR and related tasks. Due to the proprietary nature of many of these systems, we limit our review to summarizing published material about the systems. We first present an overview of research platforms and toolkits which enable scientists and practitioners to build conversational experiences. Then we conclude by reviewing historical highlights and recent trends in a variety of application areas.
Conclusions and Research Trends

We conclude the book with a brief discussion of research trends and areas for future work.
Person Re-identification System in a Controlled Environment Based on Soft Biometric Features: Clothing Color and Body Silhouette Collected on Short Video Sequences Using Computer Vision and Machine Learning Algorithms

Person re-identification is one of the most critical activities in the security area, specifically in video-surveillance since it has wide applications such as access control, people tracking and behavior detection. In this paper, a system of Re-Identification of people through 3 stages is proposed. The first one, detection and segmentation of people using Mask-RCNN method, the second, feature extraction with convolutional neural networks (CNN), and finally, the identification of people in different places with a multi-input neural network model and an output composed of a CNN. The model uses two types of descriptors based on soft-biometric appearance features, body silhouette and color in RGB space. These are treated and handled independently by deep learning techniques, which allows to generate as output the identification of persons. The experiments are carried out with a dataset created in a controlled environment by capturing videos with 2 counterposed cameras. Through a detailed comparison and the analysis of different models with different accuracy metrics, it can be indicated that the fusion of the silhouette and color features improve the solution robustness, than when treated individually. In terms of accuracy metrics, training time and validation, the multiple input model is the best evaluated in our experiments.
Conversational Machine Comprehension

This chapter discusses neural approaches to conversational machine comprehension (CMC). A CMC module, which is often referred to as reader, generates a direct answer to a user query based on query-relevant documents retrieved by the document search module.
Application and Optimization of Long Short-term Memory in Time Series Forcasting

Learning to keep information over the long time intervals by backpropagation usually spend a lot of time. Therefore, long short-term memory (LSTM) is introduced in this task with novelty and efficiency. With the advantage of truncating the gradient harmlessly, LSTM is able to connect thousands of time steps by forcing a constant error flow within special cells contains constant error carousels. The multiplication gate unit learns open and close access to a constant stream of errors. In comparison with recurrent cascade correlation, real-time recurrent learning, and chunking of neural sequence, LSTM brings about more successful results and has a short learning time. LSTM is also able to solve complicated, artificially long-lag tasks. Based on its superiority, LSTM network serves as a useful tool in time series forecasting. We use the LSTM network to forecast the cases of varicella in the future. And the current LSTM network is optimized to increase the efficiency. Finally, the training time is decreased and the accuracy of trained network is increased simultaneously when the quantity of hidden units is changed from 200 to 100.
Evaluating Extended Reality Application for a Virtual Museum. Case Study: Remigio Crespo Museum

The COVID-19 pandemic was the main reason why many organisations decided to include information and communication technologies in their processes to allow them to continue with their activities, be it providing services to users (food, medicine, etc.), training/education or disseminating culture. In the field of culture, some museums incorporated technology into their operating environment, moving from face-to-face visits to virtual visits. However, in many museums, the lack of apps designed to solve the problem of virtual visits caused some to stop receiving visitors during the pandemic. In this context, this paper describes the development of an application with a user-centred design that incorporates extended reality to allow virtual visits to the Remigio Crespo Museum in the city of Cuenca (Ecuador). The evaluation carried out to verify the application's usability and learnability is also included. The results obtained indicate that users/visitors found the application usable and easy to learn.
Person Re-identification using soft-biometric features: body silhouette and clothing texture in a multi-camera video surveillance environment

People Re-Identification has become a topic of interest due to the increasing use of intelligent video surveillance systems in the security industry. In this paper, we implement a people Re-Id system comprising three important modules: a) a person detection, responsible for detecting people in the image, b) preprocessing module, responsible of extracting the soft-biometric features of the detected persons, and c) an identification module, capable of identifying the detected person. For this purpose, a two branches multi-input and one output network model is built. The first one receives the body silhouette descriptor and the other the clothing texture descriptor. To train this model a dataset of 7 identities was built, with 1862 and 481 images for training and validation respectively, facing problems such as the existing bias in the public datasets. In addition, two videos and one validation image set were used to evaluate the system performance. The results of our proposal are positive, demonstrating that the combination of soft-biometrics features, body silhouette and clothing textures of the person increases the system ability to Re-Identify a person in images and videos.
Application of “Design Thinking” in the Development of Virtual Platforms with Gamified Elements

This article presents the development of a virtual platform with the Design Thinking methodology and gamified elements, to strengthen the teaching-learning process in the teachers of the González Suárez Educational Unit. This research arises from the need of a virtual platform with gamified elements that are free, easy to use, interactive and motivational. In this study user stories with the agile POV format, test cases to verify the functionality and abuse cases for robustness and to prevent the vulnerability of the virtual platform are used, as well as a b-Learning approach for testing in the classroom. Obtaining in the end as significant results a high and satisfactory assessment of the use of the virtual platform.
Conversational QA over Knowledge Bases

This chapter describes Conversational Question Answering over Knowledge Bases (C-KBQA) as a special case of CIR, where the answers to input queries are generated not from a collection of indexed documents but from a (set of) structured knowledge base(s).
Proactive Human-Machine Conversations

This chapter discusses methods and techniques that aim to equip a CIR system with the capability of proactively assisting a user to clarify her search intent by asking clarifying questions, guiding the user to explore the topic being discussed by suggesting useful queries, and recommending new topics and potential items of interest such as news articles in the hope of producing more fruitful search results.
Neural Machine Translation tool from Spanish to English in the Medical Domain

In Natural Language Processing (NLP), the scarcity of linguistic resources (labeled corpus, parallel corpus, pre-trained models, etc.) can lead to poor performance when applying machine learning models, however, this can be solved by applying cross-lingual approaches (machine translation, word alignment, multilingual embedding, etc.), which is a paradigm for transferring knowledge from one language with resources to another language with fewer resources. In the medical domain, there are also few resources in Spanish compared to English, due to economic, legal, and ethical issues. In this regard, there is little evidence of evaluation and optimization of machine translations from Spanish to English in the medical domain. For this purpose, a neural machine translation tool with an induced word alignment is generated in this research, on which different optimization parameters have been experimented with and applying various parallel corpora within the medical domain, as reference results with the corpora EMA with 15 epochs, a BLUE of 88.55 in English-Spanish and Scielo Spanish - English with 25 epochs, a BLEU of 53.74, being a differential in evaluation results to convolutional translators and even greatly outperforming the pre-trained Fairseq results.
Conversational Search

This chapter discusses conversational document search, which can be viewed as a sub-system of the CIR system. We start with an introduction to the task and public benchmarks; review transformation-based pre-trained language models, which are the building blocks of many CIR modules; and then describe the main components of the conversational search system, including contextual query understanding, sparse and dense document retrieval, and neural document ranking.
APPly: A Design of an Online Tertiary Level Scholarship Application Management System

Scholarship applications are continually escalating, 58% of families availed scholarships as it serves as a financial aid for college. The increasing number of scholarship applications caused difficulty in processing the applications without a time delay due to a large volume of requests received by the scholarship institutions. The present study investigated the effectiveness, efficacy, and efficiency of using scholarship management systems to manage scholarships at a Philippine University. Considering the rising applications for the Need-Based Academic Scholarship (NBAS), it became the primary focus of the study. The study utilizes the descriptive quantitative method wherein the data is statistically treated using descriptive statistics. The respondents consisted of 15 applicants and three scholarship administrators. The results showed that the web-based management system, APPly, received an overall PSSUQ score of 1.35 for the applicant-user and 1.46 for the administrator-user. The users rated the system’s usefulness at 1.27 and 1.33, respectively. The application was met positively by both use cases and was satisfied with its ease and the efficiency it offers for their tasks. Therefore, it is concluded that APPly is a functional scholarship management system as it satisfies the parameters set by the researchers. The study can be further improved by adding an SMS notification system, connecting the application to a centralized file and fund management system to simplify the scholarship transaction processes, and integrating it to be available for mobile use and into the university’s existing system.
Improved Quality of Online Education Using Prioritized Multi-Agent Reinforcement Learning for Video Traffic Scheduling

The recent global pandemic has transformed the way education is delivered, increasing the importance of video-based online learning. However, this puts a significant pressure on the underlying communication networks and the limited available bandwidth needs to be intelligently allocated to support a much higher transmission load, including video-based services. In this context, this paper proposes a Machine Learning (ML)-based solution that dynamically prioritizes content viewers with heterogeneous video services to increase their Quality of Service (QoS) and perceived Quality of Experience (QoE). The proposed approach makes use of the novel Prioritized Multi-Agent Reinforcement Learning solution (PriMARL) to decide the prioritization order of the video-based services based on networking conditions. However, the performance in terms of QoS and QoE provisioning to learners with different profiles and networking conditions depends on the type of scheduler employed in the frequency domain to conduct the scheduling and the radio resource allocation. To decide the best approach to be followed, we employ the proposed PriMARL solution with different types of scheduling rules and compare them with other state-of-the-art solutions in terms of throughput, delay, packet loss, Peak Signal-to-Noise Ratio (PSNR), and Mean Opinion Score (MOS) for different traffic loads and characteristics. We show that the proposed solution achieves the best user QoE results.
Automatic and Analytical Field Weighting for Structured Document Retrieval

Probabilistic models such as BM25 and LM have established themselves as the standard in atomic retrieval. In structured document retrieval (SDR), BM25F could be considered the most established model. However, without optimization BM25F does not benefit from the document structure. The main contribution of this paper is a new field weighting method, denoted Information Content Field Weighting (ICFW). It applies weights over the structure without optimization and overcomes issues faced by some existing SDR models, most notably the issue of saturating term frequency across fields. ICFW is similar to BM25 and LM in its analytical grounding and transparency, making it a potential new candidate for a standard SDR model. For an optimised retrieval scenario ICFW does as well, or better than baselines. More interestingly, for a non-optimised retrieval scenario we observe a considerable increase in performance. Extensive analysis is performed to understand and explain the underlying reasons for this increase.
Document-Level Relation Extraction with Distance-Dependent Bias Network and Neighbors Enhanced Loss

Document-level relation extraction (DocRE), in contrast to sentence-level, requires additional context to be considered. Recent studies, when extracting contextual information about entities, treat information about the whole document equally, which inevitably suffers from irrelevant information. This has been demonstrated to make the model not robust: it predicts correctly when an entire document is fed but errs when non-evidence sentences are removed. In this work, we propose three novel components to improve the robustness of the model by selectively considering the context of the entities. Firstly, we propose a new method for computing the distance between tokens that reduces the distance between evidence sentences and entities. Secondly, we add a distance-dependent bias network to each self-attention building block to exploit the distance information between tokens. Finally, we design an auxiliary loss for entities with higher attention to close tokens in the attention mechanism. Experimental results on three DocRE benchmark datasets show that our model not only outperforms existing models but also has strong robustness.
MOO-CMDS+NER: Named Entity Recognition-Based Extractive Comment-Oriented Multi-document Summarization

In this work, we propose an unsupervised extractive summarization framework for generating good quality summaries which are supplemented by the comments posted by the end-users. Using the evolutionary multi-objective optimization concept, different objective functions for assessing the quality of a summary, like diversity and the relevance of sentences in relation to comments, are optimized simultaneously. In the literature, named entity recognition (NER) has been shown to be useful in the summarization process. The current work is the first of its kind where we have introduced a new objective function that utilizes the concept of NER in news documents and user comments to score the news sentences. To test how well the new objective function works, different combinations of the NER-based objective function with already existing objective functions were tested on the English and French datasets using ROUGE 1, 2, and SU4 F1-scores. We have also investigated the abstractive and compressive summarization approaches for our comparative analysis. The code of the proposed work is available at the github repository https://github.com/vishalsinghroha/Unsupervised-Comment-based-Multi-document-Extractive-Summarization .
Adversarial Adaptation for French Named Entity Recognition

Named Entity Recognition (NER) is the task of identifying and classifying named entities in large-scale texts into predefined classes. NER in French and other relatively limited-resource languages cannot always benefit from approaches proposed for languages like English due to a dearth of large, robust datasets. In this paper, we present our work that aims to mitigate the effects of this dearth of large, labeled datasets. We propose a Transformer-based NER approach for French, using adversarial adaptation to similar domain or general corpora to improve feature extraction and enable better generalization. Our approach allows learning better features using large-scale unlabeled corpora from the same domain or mixed domains to introduce more variations during training and reduce overfitting. Experimental results on three labeled datasets show that our adaptation framework outperforms the corresponding non-adaptive models for various combinations of Transformer models, source datasets, and target corpora. We also show that adversarial adaptation to large-scale unlabeled corpora can help mitigate the performance dip incurred on using Transformer models pre-trained on smaller corpora.
Justifying Multi-label Text Classifications for Healthcare Applications

The healthcare domain is a very active area of research for Natural Language Processing (NLP). The classification of medical records according to codes from the International Classification of Diseases (ICD) is an essential task in healthcare. As a very sensitive application, the automatic classification of personal medical records cannot be immediately trusted without human approval. As such, it is desirable for classification models to provide reasons for each decision, such that the medical coder can validate model predictions without reading the entire document. AttentionXML is a multi-label classification model that has shown high applicability for this task and can provide attention distributions for each predicted label. In practice, we have found that these distributions do not always provide relevant spans of text. We propose a simple yet effective modification to AttentionXML for finding spans of text that can better aid the medical coders: splitting the BiLSTM of AttentionXML into a forward and a backward LSTM, creating two attention distributions that find the leftmost and rightmost limits of the text spans. We also propose a novel metric for the usefulness of our model’s suggestions by computing the drop in confidence from masking out the selected text spans. We show that our model has a similar classification performance to AttentionXML while surpassing it in obtaining relevant text spans.
Theoretical Analysis on the Efficiency of Interleaved Comparisons

This study presents a theoretical analysis on the efficiency of interleaving, an efficient online evaluation method for rankings. Although interleaving has already been applied to production systems, the source of its high efficiency has not been clarified in the literature. Therefore, this study presents a theoretical analysis on the efficiency of interleaving methods. We begin by designing a simple interleaving method similar to ordinary interleaving methods. Then, we explore a condition under which the interleaving method is more efficient than A/B testing and find that this is the case when users leave the ranking depending on the item’s relevance, a typical assumption made in click models. Finally, we perform experiments based on numerical analysis and user simulation, demonstrating that the theoretical results are consistent with the empirical results.