OpenRaiser/Intern-Atlas huggingface dataset:
One Class One Click: Quasi Scene-level Weakly Supervised Point Cloud Semantic Segmentation with Active Learning
Venue: arXiv cs.CV 2022
Year: 2022

Reliance on vast annotations to achieve leading performance severely restricts the practicality of large-scale point cloud semantic segmentation. For the purpose of reducing data annotation costs, effective labeling schemes are developed and contribute to attaining competitive results under weak supervision strategy. Revisiting current weak label forms, we introduce One Class One Click (OCOC), a low cost yet informative quasi scene-level label, which encapsulates point-level and scene-level annotations. An active weakly supervised framework is proposed to leverage scarce labels by involving weak supervision from global and local perspectives. Contextual constraints are imposed by an auxiliary scene classification task, respectively based on global feature embedding and point-wise prediction aggregation, which restricts the model prediction merely to OCOC labels. Furthermore, we design a context-aware pseudo labeling strategy, which effectively supplement point-level supervisory signals. Finally, an active learning scheme with a uncertainty measure - temporal output discrepancy is integrated to examine informative samples and provides guidance on sub-clouds query, which is conducive to quickly attaining desirable OCOC annotations and reduces the labeling cost to an extremely low extent. Extensive experimental analysis using three LiDAR benchmarks collected from airborne, mobile and ground platforms demonstrates that our proposed method achieves very promising results though subject to scarce labels. It considerably outperforms genuine scene-level weakly supervised methods by up to 25\% in terms of average F1 score and achieves competitive results against full supervision schemes. On terrestrial LiDAR dataset - Semantics3D, using approximately 2\textpertenthousand{} of labels, our method achieves an average F1 score of 85.2\%, which increases by 11.58\% compared to the baseline model.
DyRRen: A Dynamic Retriever-Reranker-Generator Model for Numerical Reasoning over Tabular and Textual Data
Venue: arXiv cs.CL 2022
Year: 2022

Numerical reasoning over hybrid data containing tables and long texts has recently received research attention from the AI community. To generate an executable reasoning program consisting of math and table operations to answer a question, state-of-the-art methods use a retriever-generator pipeline. However, their retrieval results are static, while different generation steps may rely on different sentences. To attend to the retrieved information that is relevant to each generation step, in this paper, we propose DyRRen, an extended retriever-reranker-generator framework where each generation step is enhanced by a dynamic reranking of retrieved sentences. It outperforms existing baselines on the FinQA dataset.
Quality Assurance in MLOps Setting: An Industrial Perspective
Venue: arXiv cs.SE 2022
Year: 2022

Today, machine learning (ML) is widely used in industry to provide the core functionality of production systems. However, it is practically always used in production systems as part of a larger end-to-end software system that is made up of several other components in addition to the ML model. Due to production demand and time constraints, automated software engineering practices are highly applicable. The increased use of automated ML software engineering practices in industries such as manufacturing and utilities requires an automated Quality Assurance (QA) approach as an integral part of ML software. Here, QA helps reduce risk by offering an objective perspective on the software task. Although conventional software engineering has automated tools for QA data analysis for data-driven ML, the use of QA practices for ML in operation (MLOps) is lacking. This paper examines the QA challenges that arise in industrial MLOps and conceptualizes modular strategies to deal with data integrity and Data Quality (DQ). The paper is accompanied by real industrial use-cases from industrial partners. The paper also presents several challenges that may serve as a basis for future studies.
Zero Forcing Uplink Detection through Large-Scale RIS: System Performance and Phase Shift Design
Venue: arXiv cs.IT 2022
Year: 2022

A multiple-input multiple-output wireless communication system is analytically studied, which operates with the aid of a large-scale reconfigurable intelligent surface (LRIS). LRIS is equipped with multiple passive elements with discrete phase adjustment capabilities, and independent Rician fading conditions are assumed for both the transmitter-to-LRIS and LRIS-to-receiver links. A direct transceiver link is also considered which is modeled by Rayleigh fading distribution. The system performance is analytically studied when the linear yet efficient zero-forcing detection is implemented at the receiver. In particular, the outage performance is derived in closed-form expression for different system configuration setups with regards to the available channel state information (CSI) at the receiver. In fact, the case of both perfect and imperfect CSI is analyzed. Also, an efficient phase shift design approach at LRIS is introduced, which is linear on the number of passive elements and receive antennas. The proposed phase shift design can be applied on two different modes of operation; namely, when the system strives to adapt either on the instantaneous or statistical CSI. Finally, some impactful engineering insights are provided, such as how the channel fading conditions, CSI, discrete phase shift resolution, and volume of antenna/LRIS element arrays impact on the overall system performance.
Tracking biomedical articles along the translational continuum: a measure based on biomedical knowledge representation
Venue: arXiv cs.DL 2022
Year: 2022

Keeping track of translational research is essential to evaluating the performance of programs on translational medicine. Despite several indicators in previous studies, a consensus measure is still needed to represent the translational features of biomedical research at the article level. In this study, we first trained semantic representations of biomedical entities and documents (i.e., bio entity2vec and bio doc2vec) based on over 30 million PubMed articles. With these vectors, we then developed a new measure called Translational Progression (TP) for tracking biomedical articles along the translational continuum. We validated the effectiveness of TP from two perspectives (Clinical trial phase identification and ACH classification), which showed excellent consistency between TP and other indicators. Meanwhile, TP has several advantages. First, it can track the degree of translation of biomedical research dynamically and in real time. Second, it is straightforward to interpret and operationalize. Third, it doesn%u2019t require labor-intensive MeSH labeling and it is suitable for big scholarly data as well as papers that are not indexed in PubMed. In addition, we examined the translational progressions of biomedical research from three dimensions (including overall distribution, time, and research topic), which revealed three significant findings. The proposed measure in this study could be used by policymakers to monitor biomedical research with high translational potential in real time and make better decisions. It can also be adopted and improved for other domains, such as physics or computer science, to assess the application value of scientific discoveries.
Assessment of Human Behavior in Virtual Reality by Eye Tracking
Venue: arXiv cs.HC 2022
Year: 2022

Virtual reality (VR) is not a new technology but has been in development for decades, driven by advances in computer technology. Currently, VR technology is increasingly being used in applications to enable immersive, yet controlled research settings. Education and entertainment are two important application areas, where VR has been considered a key enabler of immersive experiences and their further advancement. At the same time, the study of human behavior in such innovative environments is expected to contribute to a better design of VR applications. Therefore, modern VR devices are consistently equipped with eye-tracking technology, enabling thus further studies of human behavior through the collection of process data. In particular, eye-tracking technology in combination with machine learning techniques and explainable models can provide new insights for a deeper understanding of human behavior during immersion in virtual environments. In this work, a systematic computational framework based on eye-tracking and behavioral user data and state-of-the-art machine learning approaches is proposed to understand human behavior and individual differences in VR contexts. This computational framework is then employed in three user studies across two different domains. In the educational domain, two different immersive VR classrooms were created where students can learn and teachers can train. In terms of VR entertainment, eye movements open a new avenue to evaluate VR locomotion techniques from the perspective of user cognitive load and user experience. This work paves the way for assessing human behavior in VR scenarios and provides profound insights into the way of designing, evaluating, and improving interactive VR systems. In particular, more effective and customizable virtual environments can be created to provide users with tailored experiences.
A Dynamic Weighted Federated Learning for Android Malware Classification
Venue: arXiv cs.CR 2022
Year: 2022

Android malware attacks are increasing daily at a tremendous volume, making Android users more vulnerable to cyber-attacks. Researchers have developed many machine learning (ML)/ deep learning (DL) techniques to detect and mitigate android malware attacks. However, due to technological advancement, there is a rise in android mobile devices. Furthermore, the devices are geographically dispersed, resulting in distributed data. In such scenario, traditional ML/DL techniques are infeasible since all of these approaches require the data to be kept in a central system; this may provide a problem for user privacy because of the massive proliferation of Android mobile devices; putting the data in a central system creates an overhead. Also, the traditional ML/DL-based android malware classification techniques are not scalable. Researchers have proposed federated learning (FL) based android malware classification system to solve the privacy preservation and scalability with high classification performance. In traditional FL, Federated Averaging (FedAvg) is utilized to construct the global model at each round by merging all of the local models obtained from all of the customers that participated in the FL. However, the conventional FedAvg has a disadvantage: if one poor-performing local model is included in global model development for each round, it may result in an under-performing global model. Because FedAvg favors all local models equally when averaging. To address this issue, our main objective in this work is to design a dynamic weighted federated averaging (DW-FedAvg) strategy in which the weights for each local model are automatically updated based on their performance at the client. The DW-FedAvg is evaluated using four popular benchmark datasets, Melgenome, Drebin, Kronodroid and Tuandromd used in android malware classification research.
Cost Splitting for Multi-Objective Conflict-Based Search
Venue: arXiv cs.AI 2022
Year: 2022

The Multi-Objective Multi-Agent Path Finding (MO-MAPF) problem is the problem of finding the Pareto-optimal frontier of collision-free paths for a team of agents while minimizing multiple cost metrics. Examples of such cost metrics include arrival times, travel distances, and energy consumption.In this paper, we focus on the Multi-Objective Conflict-Based Search (MO-CBS) algorithm, a state-of-the-art MO-MAPF algorithm. We show that the standard splitting strategy used by MO-CBS can lead to duplicate search nodes and hence can duplicate the search effort that MO-CBS needs to make. To address this issue, we propose two new splitting strategies for MO-CBS, namely cost splitting and disjoint cost splitting. Our theoretical results show that, when combined with either of these two new splitting strategies, MO-CBS maintains its completeness and optimality guarantees. Our experimental results show that disjoint cost splitting, our best splitting strategy, speeds up MO-CBS by up to two orders of magnitude and substantially improves its success rates in various settings.
Implementation and Evaluation of a System for Assessment of The Quality of Long-Term Management of Patients at a Geriatric Hospital
Venue: arXiv cs.AI 2022
Year: 2022

Background The use of a clinical decision support system for assessing the quality of care, based on computerized clinical guidelines (GLs), is likely to improve care, reduce costs, save time, and enhance the staff's capabilities. Objectives Implement and evaluate a system for assessment of the quality of the care, in the domain of management of pressure ulcers, by investigating the level of compliance of the staff to the GLs. Methods Using data for 100 random patients from the local EMR system we performed a technical evaluation, checking the applicability and usability, followed by a functional evaluation of the system investigating the quality metrics given to the compliance of the medical's staff to the protocol. We compared the scores given by the nurse when supported by the system, to the scores given by the nurse without the system's support, and to the scores given by the system. We also measured the time taken to perform the assessment with and without the system's support. Results There were no significant differences in the scores of most measures given by the nurse using the system, compared to the scores given by the system. There were also no significant differences across the values of most quality measures given by the nurse without support compared to the values given by the nurse with support. Using the system, however, significantly reduced the nurse's average assessment time. Conclusions Using an automated quality-assessment system, may enable a senior nurse, to quickly and accurately assess the quality of care. In addition to its accuracy, the system considerably reduces the time taken to assess the various quality measures.
A Calculus for Amortized Expected Runtimes
Venue: arXiv cs.LO 2022
Year: 2022

We develop a weakest-precondition-style calculus à la Dijkstra for reasoning about amortized expected runtimes of randomized algorithms with access to dynamic memory - the $\textsf{aert}$ calculus. Our calculus is truly quantitative, i.e. instead of Boolean valued predicates, it manipulates real-valued functions. En route to the $\textsf{aert}$ calculus, we study the $\textsf{ert}$ calculus for reasoning about expected runtimes of Kaminski et al. [2018] extended by capabilities for handling dynamic memory, thus enabling compositional and local reasoning about randomized data structures. This extension employs runtime separation logic, which has been foreshadowed by Matheja [2020] and then implemented in Isabelle/HOL by Haslbeck [2021]. In addition to Haslbeck's results, we further prove soundness of the so-extended $\textsf{ert}$ calculus with respect to an operational Markov decision process model featuring countably-branching nondeterminism, provide intuitive explanations, and provide proof rules enabling separation logic-style verification for upper bounds on expected runtimes. Finally, we build the so-called potential method for amortized analysis into the $\textsf{ert}$ calculus, thus obtaining the $\textsf{aert}$ calculus. Since one needs to be able to handle changes in potential which can be negative, the $\textsf{aert}$ calculus needs to be capable of handling signed random variables. A particularly pleasing feature of our solution is that, unlike e.g. Kozen [1985], we obtain a loop rule for our signed random variables, and furthermore, unlike e.g. Kaminski and Katoen [2017], the $\textsf{aert}$ calculus makes do without the need for involved technical machinery keeping track of the integrability of the random variables. Finally, we present case studies, including a formal analysis of a randomized delete-insert-find-any set data structure [Brodal et al. 1996].
Contrastive Multi-View Textual-Visual Encoding: Towards One Hundred Thousand-Scale One-Shot Logo Identification
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we study the problem of identifying logos of business brands in natural scenes in an open-set one-shot setting. This problem setup is significantly more challenging than traditionally-studied 'closed-set' and 'large-scale training samples per category' logo recognition settings. We propose a novel multi-view textual-visual encoding framework that encodes text appearing in the logos as well as the graphical design of the logos to learn robust contrastive representations. These representations are jointly learned for multiple views of logos over a batch and thereby they generalize well to unseen logos. We evaluate our proposed framework for cropped logo verification, cropped logo identification, and end-to-end logo identification in natural scene tasks; and compare it against state-of-the-art methods. Further, the literature lacks a 'very-large-scale' collection of reference logo images that can facilitate the study of one-hundred thousand-scale logo identification. To fill this gap in the literature, we introduce Wikidata Reference Logo Dataset (WiRLD), containing logos for 100K business brands harvested from Wikidata. Our proposed framework that achieves an area under the ROC curve of 91.3% on the QMUL-OpenLogo dataset for the verification task, outperforms state-of-the-art methods by 9.1% and 2.6% on the one-shot logo identification task on the Toplogos-10 and the FlickrLogos32 datasets, respectively. Further, we show that our method is more stable compared to other baselines even when the number of candidate logos is on a 100K scale.
Look, Read and Ask: Learning to Ask Questions by Reading Text in Images
Venue: arXiv cs.CV 2022
Year: 2022

We present a novel problem of text-based visual question generation or TextVQG in short. Given the recent growing interest of the document image analysis community in combining text understanding with conversational artificial intelligence, e.g., text-based visual question answering, TextVQG becomes an important task. TextVQG aims to generate a natural language question for a given input image and an automatically extracted text also known as OCR token from it such that the OCR token is an answer to the generated question. TextVQG is an essential ability for a conversational agent. However, it is challenging as it requires an in-depth understanding of the scene and the ability to semantically bridge the visual content with the text present in the image. To address TextVQG, we present an OCR consistent visual question generation model that Looks into the visual content, Reads the scene text, and Asks a relevant and meaningful natural language question. We refer to our proposed model as OLRA. We perform an extensive evaluation of OLRA on two public benchmarks and compare them against baselines. Our model OLRA automatically generates questions similar to the public text-based visual question answering datasets that were curated manually. Moreover, we significantly outperform baseline approaches on the performance measures popularly used in text generation literature.
How do Authors' Perceptions of their Papers Compare with Co-authors' Perceptions and Peer-review Decisions?
Venue: arXiv cs.LG 2022
Year: 2022

How do author perceptions match up to the outcomes of the peer-review process and perceptions of others? In a top-tier computer science conference (NeurIPS 2021) with more than 23,000 submitting authors and 9,000 submitted papers, we survey the authors on three questions: (i) their predicted probability of acceptance for each of their papers, (ii) their perceived ranking of their own papers based on scientific contribution, and (iii) the change in their perception about their own papers after seeing the reviews. The salient results are: (1) Authors have roughly a three-fold overestimate of the acceptance probability of their papers: The median prediction is 70% for an approximately 25% acceptance rate. (2) Female authors exhibit a marginally higher (statistically significant) miscalibration than male authors; predictions of authors invited to serve as meta-reviewers or reviewers are similarly calibrated, but better than authors who were not invited to review. (3) Authors' relative ranking of scientific contribution of two submissions they made generally agree (93%) with their predicted acceptance probabilities, but there is a notable 7% responses where authors think their better paper will face a worse outcome. (4) The author-provided rankings disagreed with the peer-review decisions about a third of the time; when co-authors ranked their jointly authored papers, co-authors disagreed at a similar rate -- about a third of the time. (5) At least 30% of respondents of both accepted and rejected papers said that their perception of their own paper improved after the review process. The stakeholders in peer review should take these findings into account in setting their expectations from peer review.
Adversarial Attacks are a Surprisingly Strong Baseline for Poisoning Few-Shot Meta-Learners
Venue: arXiv cs.LG 2022
Year: 2022

This paper examines the robustness of deployed few-shot meta-learning systems when they are fed an imperceptibly perturbed few-shot dataset. We attack amortized meta-learners, which allows us to craft colluding sets of inputs that are tailored to fool the system's learning algorithm when used as training data. Jointly crafted adversarial inputs might be expected to synergistically manipulate a classifier, allowing for very strong data-poisoning attacks that would be hard to detect. We show that in a white box setting, these attacks are very successful and can cause the target model's predictions to become worse than chance. However, in opposition to the well-known transferability of adversarial examples in general, the colluding sets do not transfer well to different classifiers. We explore two hypotheses to explain this: 'overfitting' by the attack, and mismatch between the model on which the attack is generated and that to which the attack is transferred. Regardless of the mitigation strategies suggested by these hypotheses, the colluding inputs transfer no better than adversarial inputs that are generated independently in the usual way.
Mitigating Negative Transfer in Multi-Task Learning with Exponential Moving Average Loss Weighting Strategies
Venue: arXiv cs.LG 2022
Year: 2022

Multi-Task Learning (MTL) is a growing subject of interest in deep learning, due to its ability to train models more efficiently on multiple tasks compared to using a group of conventional single-task models. However, MTL can be impractical as certain tasks can dominate training and hurt performance in others, thus making some tasks perform better in a single-task model compared to a multi-task one. Such problems are broadly classified as negative transfer, and many prior approaches in the literature have been made to mitigate these issues. One such current approach to alleviate negative transfer is to weight each of the losses so that they are on the same scale. Whereas current loss balancing approaches rely on either optimization or complex numerical analysis, none directly scale the losses based on their observed magnitudes. We propose multiple techniques for loss balancing based on scaling by the exponential moving average and benchmark them against current best-performing methods on three established datasets. On these datasets, they achieve comparable, if not higher, performance compared to current best-performing methods.