OpenRaiser/Intern-Atlas huggingface dataset:
Open-Set Semi-Supervised Learning for 3D Point Cloud Understanding
Venue: arXiv cs.CV 2022
Year: 2022

Semantic understanding of 3D point cloud relies on learning models with massively annotated data, which, in many cases, are expensive or difficult to collect. This has led to an emerging research interest in semi-supervised learning (SSL) for 3D point cloud. It is commonly assumed in SSL that the unlabeled data are drawn from the same distribution as that of the labeled ones; This assumption, however, rarely holds true in realistic environments. Blindly using out-of-distribution (OOD) unlabeled data could harm SSL performance. In this work, we propose to selectively utilize unlabeled data through sample weighting, so that only conducive unlabeled data would be prioritized. To estimate the weights, we adopt a bi-level optimization framework which iteratively optimizes a metaobjective on a held-out validation set and a task-objective on a training set. Faced with the instability of efficient bi-level optimizers, we further propose three regularization techniques to enhance the training stability. Extensive experiments on 3D point cloud classification and segmentation tasks verify the effectiveness of our proposed method. We also demonstrate the feasibility of a more efficient training strategy.
Token sliding on graphs of girth five
Venue: arXiv cs.CC 2022
Year: 2022

In the Token Sliding problem we are given a graph $G$ and two independent sets $I_s$ and $I_t$ in $G$ of size $k \geq 1$. The goal is to decide whether there exists a sequence $\langle I_1, I_2, \ldots, I_\ell \rangle$ of independent sets such that for all $i \in \{1,\ldots, \ell\}$ the set $I_i$ is an independent set of size $k$, $I_1 = I_s$, $I_\ell = I_t$ and $I_i \triangle I_{i + 1} = \{u, v\} \in E(G)$. Intuitively, we view each independent set as a collection of tokens placed on the vertices of the graph. Then, the problem asks whether there exists a sequence of independent sets that transforms $I_s$ into $I_t$ where at each step we are allowed to slide one token from a vertex to a neighboring vertex. In this paper, we focus on the parameterized complexity of Token Sliding parameterized by $k$. As shown by Bartier et al., the problem is W[1]-hard on graphs of girth four or less, and the authors posed the question of whether there exists a constant $p \geq 5$ such that the problem becomes fixed-parameter tractable on graphs of girth at least $p$. We answer their question positively and prove that the problem is indeed fixed-parameter tractable on graphs of girth five or more, which establishes a full classification of the tractability of Token Sliding parameterized by the number of tokens based on the girth of the input graph.
A Unified Approach to Discrepancy Minimization
Venue: arXiv cs.DS 2022
Year: 2022

We study a unified approach and algorithm for constructive discrepancy minimization based on a stochastic process. By varying the parameters of the process, one can recover various state-of-the-art results. We demonstrate the flexibility of the method by deriving a discrepancy bound for smoothed instances, which interpolates between known bounds for worst-case and random instances.
Machine Learning and Artificial Intelligence in Circular Economy: A Bibliometric Analysis and Systematic Literature Review
Venue: arXiv cs.CY 2022
Year: 2022

With unorganized, unplanned and improper use of limited raw materials, an abundant amount of waste is being produced, which is harmful to our environment and ecosystem. While traditional linear production lines fail to address far-reaching issues like waste production and a shorter product life cycle, a prospective concept, namely circular economy (CE), has shown promising prospects to be adopted at industrial and governmental levels. CE aims to complete the product life cycle loop by bringing out the highest values from raw materials in the design phase and later on by reusing, recycling, and remanufacturing. Innovative technologies like artificial intelligence (AI) and machine learning(ML) provide vital assistance in effectively adopting and implementing CE in real-world practices. This study explores the adoption and integration of applied AI techniques in CE. First, we conducted bibliometric analysis on a collection of 104 SCOPUS indexed documents exploring the critical research criteria in AI and CE. Forty papers were picked to conduct a systematic literature review from these documents. The selected documents were further divided into six categories: sustainable development, reverse logistics, waste management, supply chain management, recycle & reuse, and manufacturing development. Comprehensive research insights and trends have been extracted and delineated. Finally, the research gap needing further attention has been identified and the future research directions have also been discussed.
Coding Concepts and Reed-Solomon Codes
Venue: arXiv cs.IT 2022
Year: 2022

The material in this book is presented to graduate students in Information and Communication theory. The idea is that we give an introduction to particular applications of information theory and coding in digital communications. The goal is to bring understanding of the underlying concepts, both in theory as well as in practice. We mainly concentrate on our own research results. After showing obtainable performance, we give a specific implementation using Reed-Solomon (RS) codes. The reason for using RS codes is that they can be seen as optimal codes with maximum obtainable minimum distance. Furthermore, the structure of RS codes enables specific applications that fit perfectly into the developed concepts. We do not intend to develop the theory of error correcting codes.
Meta Transfer Learning for Early Success Prediction in MOOCs
Venue: arXiv cs.CY 2022
Year: 2022

Despite the increasing popularity of massive open online courses (MOOCs), many suffer from high dropout and low success rates. Early prediction of student success for targeted intervention is therefore essential to ensure no student is left behind in a course. There exists a large body of research in success prediction for MOOCs, focusing mainly on training models from scratch for individual courses. This setting is impractical in early success prediction as the performance of a student is only known at the end of the course. In this paper, we aim to create early success prediction models that can be transferred between MOOCs from different domains and topics. To do so, we present three novel strategies for transfer: 1) pre-training a model on a large set of diverse courses, 2) leveraging the pre-trained model by including meta information about courses, and 3) fine-tuning the model on previous course iterations. Our experiments on 26 MOOCs with over 145,000 combined enrollments and millions of interactions show that models combining interaction data and course information have comparable or better performance than models which have access to previous iterations of the course. With these models, we aim to effectively enable educators to warm-start their predictions for new and ongoing courses.
Classification of Buildings' Potential for Seismic Damage by Means of Artificial Intelligence Techniques
Venue: arXiv cs.LG 2022
Year: 2022

Developing a rapid, but also reliable and efficient, method for classifying the seismic damage potential of buildings constructed in countries with regions of high seismicity is always at the forefront of modern scientific research. Such a technique would be essential for estimating the pre-seismic vulnerability of the buildings, so that the authorities will be able to develop earthquake safety plans for seismic rehabilitation of the highly earthquake-susceptible structures. In the last decades, several researchers have proposed such procedures, some of which were adopted by seismic code guidelines. These procedures usually utilize methods based either on simple calculations or on the application of statistics theory. Recently, the increase of the computers' power has led to the development of modern statistical methods based on the adoption of Machine Learning algorithms. These methods have been shown to be useful for predicting seismic performance and classifying structural damage level by means of extracting patterns from data collected via various sources. A large training dataset is used for the implementation of the classification algorithms. To this end, 90 3D R/C buildings with three different masonry infills' distributions are analysed utilizing Nonlinear Time History Analysis method for 65 real seismic records. The level of the seismic damage is expressed in terms of the Maximum Interstory Drift Ratio. A large number of Machine Learning algorithms is utilized in order to estimate the buildings' damage response. The most significant conclusion which is extracted is that the Machine Learning methods that are mathematically well-established and their operations that are clearly interpretable step by step can be used to solve some of the most sophisticated real-world problems in consideration with high accuracy.
A Probabilistic Interpretation of Transformers
Venue: arXiv cs.LG 2022
Year: 2022

We propose a probabilistic interpretation of exponential dot product attention of transformers and contrastive learning based off of exponential families. The attention sublayer of transformers is equivalent to a gradient ascent step of the log normalizer, which is the log-sum-exp term in the Hopfield theory of attention. This ascent step induces a parallel expansion of points, which is counterbalanced by a contraction from layer normalization. We also state theoretical limitations of our theory and the Hopfield theory and suggest directions for resolution.
Distributed intelligence on the Edge-to-Cloud Continuum: A systematic literature review
Venue: arXiv cs.LG 2022
Year: 2022

The explosion of data volumes generated by an increasing number of applications is strongly impacting the evolution of distributed digital infrastructures for data analytics and machine learning (ML). While data analytics used to be mainly performed on cloud infrastructures, the rapid development of IoT infrastructures and the requirements for low-latency, secure processing has motivated the development of edge analytics. Today, to balance various trade-offs, ML-based analytics tends to increasingly leverage an interconnected ecosystem that allows complex applications to be executed on hybrid infrastructures where IoT Edge devices are interconnected to Cloud/HPC systems in what is called the Computing Continuum, the Digital Continuum, or the Transcontinuum.Enabling learning-based analytics on such complex infrastructures is challenging. The large scale and optimized deployment of learning-based workflows across the Edge-to-Cloud Continuum requires extensive and reproducible experimental analysis of the application execution on representative testbeds. This is necessary to help understand the performance trade-offs that result from combining a variety of learning paradigms and supportive frameworks. A thorough experimental analysis requires the assessment of the impact of multiple factors, such as: model accuracy, training time, network overhead, energy consumption, processing latency, among others.This review aims at providing a comprehensive vision of the main state-of-the-art libraries and frameworks for machine learning and data analytics available today. It describes the main learning paradigms enabling learning-based analytics on the Edge-to-Cloud Continuum. The main simulation, emulation, deployment systems, and testbeds for experimental research on the Edge-to-Cloud Continuum available today are also surveyed. Furthermore, we analyze how the selected systems provide support for experiment reproducibility. We conclude our review with a detailed discussion of relevant open research challenges and of future directions in this domain such as: holistic understanding of performance; performance optimization of applications;efficient deployment of Artificial Intelligence (AI) workflows on highly heterogeneous infrastructures; and reproducible analysis of experiments on the Computing Continuum.
Hausa Visual Genome: A Dataset for Multi-Modal English to Hausa Machine Translation
Venue: arXiv cs.CL 2022
Year: 2022

Multi-modal Machine Translation (MMT) enables the use of visual information to enhance the quality of translations. The visual information can serve as a valuable piece of context information to decrease the ambiguity of input sentences. Despite the increasing popularity of such a technique, good and sizeable datasets are scarce, limiting the full extent of their potential. Hausa, a Chadic language, is a member of the Afro-Asiatic language family. It is estimated that about 100 to 150 million people speak the language, with more than 80 million indigenous speakers. This is more than any of the other Chadic languages. Despite a large number of speakers, the Hausa language is considered low-resource in natural language processing (NLP). This is due to the absence of sufficient resources to implement most NLP tasks. While some datasets exist, they are either scarce, machine-generated, or in the religious domain. Therefore, there is a need to create training and evaluation data for implementing machine learning tasks and bridging the research gap in the language. This work presents the Hausa Visual Genome (HaVG), a dataset that contains the description of an image or a section within the image in Hausa and its equivalent in English. To prepare the dataset, we started by translating the English description of the images in the Hindi Visual Genome (HVG) into Hausa automatically. Afterward, the synthetic Hausa data was carefully post-edited considering the respective images. The dataset comprises 32,923 images and their descriptions that are divided into training, development, test, and challenge test set. The Hausa Visual Genome is the first dataset of its kind and can be used for Hausa-English machine translation, multi-modal research, and image description, among various other natural language processing and generation tasks.
The Theory of Artificial Immutability: Protecting Algorithmic Groups Under Anti-Discrimination Law
Venue: arXiv cs.CY 2022
Year: 2022

Artificial Intelligence (AI) is increasingly used to make important decisions about people. While issues of AI bias and proxy discrimination are well explored, less focus has been paid to the harms created by profiling based on groups that do not map to or correlate with legally protected groups such as sex or ethnicity. This raises a question: are existing equality laws able to protect against emergent AI-driven inequality? This article examines the legal status of algorithmic groups in North American and European non-discrimination doctrine, law, and jurisprudence and will show that algorithmic groups are not comparable to traditional protected groups. Nonetheless, these new groups are worthy of protection. I propose a new theory of harm - "the theory of artificial immutability" - that aims to bring AI groups within the scope of the law. My theory describes how algorithmic groups act as de facto immutable characteristics in practice that limit people's autonomy and prevent them from achieving important goals.
Reversing an Imperative Concurrent Programming Language
Venue: arXiv cs.PL 2022
Year: 2022

We introduce a method of reversing the execution of imperative concurrent programs. Given an irreversible program, we describe the process of producing two versions. The first performs forward execution and saves information necessary for reversal. The second uses this saved information to simulate reversal. We propose using identifiers to overcome challenges of reversing concurrent programs. We prove this reversibility to be correct, showing that the initial program state is restored and that all saved information is used (garbage-free).
Using Machine Learning to Evaluate Real Estate Prices Using Location Big Data
Venue: arXiv cs.LG 2022
Year: 2022

With everyone trying to enter the real estate market nowadays, knowing the proper valuations for residential and commercial properties has become crucial. Past researchers have been known to utilize static real estate data (e.g. number of beds, baths, square footage) or even a combination of real estate and demographic information to predict property prices. In this investigation, we attempted to improve upon past research. So we decided to explore a unique approach: we wanted to determine if mobile location data could be used to improve the predictive power of popular regression and tree-based models. To prepare our data for our models, we processed the mobility data by attaching it to individual properties from the real estate data that aggregated users within 500 meters of the property for each day of the week. We removed people that lived within 500 meters of each property, so each property's aggregated mobility data only contained non-resident census features. On top of these dynamic census features, we also included static census features, including the number of people in the area, the average proportion of people commuting, and the number of residents in the area. Finally, we tested multiple models to predict real estate prices. Our proposed model is two stacked random forest modules combined using a ridge regression that uses the random forest outputs as predictors. The first random forest model used static features only and the second random forest model used dynamic features only. Comparing our models with and without the dynamic mobile location features concludes the model with dynamic mobile location features achieves 3/% percent lower mean squared error than the same model but without dynamic mobile location features.
Triangular Dropout: Variable Network Width without Retraining
Venue: arXiv cs.LG 2022
Year: 2022

One of the most fundamental design choices in neural networks is layer width: it affects the capacity of what a network can learn and determines the complexity of the solution. This latter property is often exploited when introducing information bottlenecks, forcing a network to learn compressed representations. However, such an architecture decision is typically immutable once training begins; switching to a more compressed architecture requires retraining. In this paper we present a new layer design, called Triangular Dropout, which does not have this limitation. After training, the layer can be arbitrarily reduced in width to exchange performance for narrowness. We demonstrate the construction and potential use cases of such a mechanism in three areas. Firstly, we describe the formulation of Triangular Dropout in autoencoders, creating models with selectable compression after training. Secondly, we add Triangular Dropout to VGG19 on ImageNet, creating a powerful network which, without retraining, can be significantly reduced in parameters. Lastly, we explore the application of Triangular Dropout to reinforcement learning (RL) policies on selected control problems.
Universal Optimality and Robust Utility Bounds for Metric Differential Privacy
Venue: arXiv cs.CR 2022
Year: 2022

We study the privacy-utility trade-off in the context of metric differential privacy. Ghosh et al. introduced the idea of universal optimality to characterise the best mechanism for a certain query that simultaneously satisfies (a fixed) $ε$-differential privacy constraint whilst at the same time providing better utility compared to any other $ε$-differentially private mechanism for the same query. They showed that the Geometric mechanism is "universally optimal" for the class of counting queries. On the other hand, Brenner and Nissim showed that outside the space of counting queries, and for the Bayes risk loss function, no such universally optimal mechanisms exist. In this paper we use metric differential privacy and quantitative information flow as the fundamental principle for studying universal optimality. Metric differential privacy is a generalisation of both standard (i.e., central) differential privacy and local differential privacy, and it is increasingly being used in various application domains, for instance in location privacy and in privacy preserving machine learning. Using this framework we are able to clarify Nissim and Brenner's negative results, showing (a) that in fact all privacy types contain optimal mechanisms relative to certain kinds of non-trivial loss functions, and (b) extending and generalising their negative results beyond Bayes risk specifically to a wide class of non-trivial loss functions. We also propose weaker universal benchmarks of utility called "privacy type capacities". We show that such capacities always exist and can be computed using a convex optimisation algorithm.
How Do Captions Affect Visualization Reading?
Venue: arXiv cs.HC 2022
Year: 2022

Captions help readers better understand visualizations. However, if the visualization is intended to communicate specific features, should the caption be statistical, and focus on specific values, or perceptual, and focus on general patterns? Prior work has shown that when captions mention visually salient features, readers tend to recall those features. Still, we lack explicit guidelines for how to compose the appropriate caption. Further, what if the author wishes to emphasize a less salient feature? In this paper, we study how the visual salience of the feature described in a caption, and the semantic level of the caption description, affect a reader's takeaways from line charts. For each single- or multi-line chart, we generate 4 captions that 1) describe either the primary or secondary salient feature in a chart, and 2) describe the feature either at the statistical or perceptual levels. We then show participants random chart-caption pairs and record their takeaways. We find that the primary salient feature is more memorable for single-line charts when the caption is expressed at the statistical level; for primary and secondary features in multi-line charts, the perceptual level is more memorable. We also find that many readers will tend to recall y-axis numerical values when a caption is present.
Hidden behind the obvious: misleading keywords and implicitly abusive language on social media
Venue: arXiv cs.CL 2022
Year: 2022

While social media offers freedom of self-expression, abusive language carry significant negative social impact. Driven by the importance of the issue, research in the automated detection of abusive language has witnessed growth and improvement. However, these detection models display a reliance on strongly indicative keywords, such as slurs and profanity. This means that they can falsely (1a) miss abuse without such keywords or (1b) flag non-abuse with such keywords, and that (2) they perform poorly on unseen data. Despite the recognition of these problems, gaps and inconsistencies remain in the literature. In this study, we analyse the impact of keywords from dataset construction to model behaviour in detail, with a focus on how models make mistakes on (1a) and (1b), and how (1a) and (1b) interact with (2). Through the analysis, we provide suggestions for future research to address all three problems.