Reviewerly/exHarmony huggingface dataset:
Combined Pruning for Nested Cross-Validation to Accelerate Automated Hyperparameter Optimization for Embedded Feature Selection in High-Dimensional Data with Very Small Sample Sizes

Background: Embedded feature selection in high-dimensional data with very small sample sizes requires optimized hyperparameters for the model building process. For this hyperparameter optimization, nested cross-validation must be applied to avoid a biased performance estimation. The resulting repeated training with high-dimensional data leads to very long computation times. Moreover, it is likely to observe a high variance in the individual performance evaluation metrics caused by outliers in tiny validation sets. Therefore, early stopping applying standard pruning algorithms to save time risks discarding promising hyperparameter sets. Result: To speed up feature selection for high-dimensional data with tiny sample size, we adapt the use of a state-of-the-art asynchronous successive halving pruner. In addition, we combine it with two complementary pruning strategies based on domain or prior knowledge. One pruning strategy immediately stops computing trials with semantically meaningless results for the selected hyperparameter combinations. The other is a new extrapolating threshold pruning strategy suitable for nested-cross-validation with a high variance of performance evaluation metrics. In repeated experiments, our combined pruning strategy keeps all promising trials. At the same time, the calculation time is substantially reduced compared to using a state-of-the-art asynchronous successive halving pruner alone. Up to 81.3\% fewer models were trained achieving the same optimization result. Conclusion: The proposed combined pruning strategy accelerates data analysis or enables deeper searches for hyperparameters within the same computation time. This leads to significant savings in time, money and energy consumption, opening the door to advanced, time-consuming analyses.
Optimal alphabet for single text compression

A text written using symbols from a given alphabet can be compressed using the Huffman code, which minimizes the length of the encoded text. It is necessary, however, to employ a text-specific codebook, i.e. the symbol-codeword dictionary, to decode the original text. Thus, the compression performance should be evaluated by the full code length, i.e. the length of the encoded text plus the length of the codebook. We studied several alphabets for compressing texts -- letters, n-grams of letters, syllables, words, and phrases. If only sufficiently short texts are retained, an alphabet of letters or two-grams of letters is optimal. For the majority of Project Gutenberg texts, the best alphabet (the one that minimizes the full code length) is given by syllables or words, depending on the representation of the codebook. Letter 3 and 4-grams, having on average comparable length to syllables/words, perform noticeably worse than syllables or words. Word 2-grams also are never the best alphabet, on the account of having a very large codebook. We also show that the codebook representation is important -- switching from a naive representation to a compact one significantly improves the matters for alphabets with large number of symbols, most notably the words. Thus, meaning-expressing elements of the language (syllables or words) provide the best compression alphabet.
The Human Age in Education

American society is riddled with political corruption variously called graft, scams, scandals, or other description. In fact, misfeasance is so widespread a majority of our citizens believe that all optimistic predictions for the human race are examples of Pollyanna thinking. However, in spite of ubiquitous mischief, it is possible to identify clear-cut positive trends in our society. That is, despite the broad road of deviousness there remain constructive events that can lead to healthy outcomes. We need to remember the old truism, "It is an ill wind that blows no good." The purpose of this writing is to describe a path of hope that could become a viable alternative for a sign
Multi-Class and Automated Tweet Categorization

Twitter is among the most prevalent social media platform being used by millions of people all over the world. It is used to express ideas and opinions about political, social, business, sports, health, religion, and various other categories. The study reported here aims to detect the tweet category from its text. It becomes quite challenging when text consists of 140 characters only, with full of noise. The tweet is categorized under 12 specified categories using Text Mining or Natural Language Processing (NLP), and Machine Learning (ML) techniques. It is observed that a huge number of trending topics are provided by Twitter but it is really challenging to find out that what these trending topics are all about. Therefore, it is extremely crucial to automatically categorize the tweets into general categories for plenty of information extraction tasks. A large dataset is constructed by combining two different nature of datasets having varying levels of category identification complexities. It is annotated by experts under proper guidelines for increased quality and high agreement values. It makes the proposed model quite robust. Various types of ML algorithms were used to train and evaluate the proposed model. These models have explored over three datasets separately. It is explored that the nature of the dataset is highly non-linear therefore complex or non-linear models perform better. The best ensemble model named, Gradient Boosting achieved an AUC score of 85\%. That is much better than the other related studies conducted.
Learning Online Multi-Sensor Depth Fusion

Many hand-held or mixed reality devices are used with a single sensor for 3D reconstruction, although they often comprise multiple sensors. Multi-sensor depth fusion is able to substantially improve the robustness and accuracy of 3D reconstruction methods, but existing techniques are not robust enough to handle sensors which operate with diverse value ranges as well as noise and outlier statistics. To this end, we introduce SenFuNet, a depth fusion approach that learns sensor-specific noise and outlier statistics and combines the data streams of depth frames from different sensors in an online fashion. Our method fuses multi-sensor depth streams regardless of time synchronization and calibration and generalizes well with little training data. We conduct experiments with various sensor combinations on the real-world CoRBS and Scene3D datasets, as well as the Replica dataset. Experiments demonstrate that our fusion strategy outperforms traditional and recent online depth fusion approaches. In addition, the combination of multiple sensors yields more robust outlier handling and more precise surface reconstruction than the use of a single sensor. The source code and data are available at https://github.com/tfy14esa/SenFuNet.
Completeness of Discrete Translates in $H^1(\mathbb{R})$

We provide a characterization of discrete sets $Λ\subset \mathbb{R}$ that admit a function whose $Λ$-translates are complete in the Hardy space $H^1(\mathbb{R})$. In particular, we show that such a set cannot be uniformly discrete. We then give a uniformly discrete $Λ\subset \mathbb{R}$ which admits a pair of functions such that their $Λ$-translates are complete in $H^1(\mathbb{R})$.
New formulas for moments and functions of the multivariate normal distribution extending Stein's lemma and Isserlis theorem

We prove a formula for the evaluation of expectations containing a scalar function of a Gaussian random vector multiplied by a product of the random vector components, each one raised at a non-negative integer power. Some powers could be of zeroth-order, and, for expectations containing only one vector component to the first power, the formula reduces to Stein's lemma for the multivariate normal distribution. On the other hand, by setting the said function inside expectation equal to one, we easily derive Isserlis theorem and its generalizations, regarding higher order moments of a Gaussian random vector. We provide two proofs of the formula, with the first being a rigorous proof via mathematical induction. The second is a formal, constructive derivation based on treating the expectation not as an integral, but as the consecutive actions of pseudodifferential operators defined via the moment-generating function of the Gaussian random vector.
Question Generation for Evaluating Cross-Dataset Shifts in Multi-modal Grounding

Visual question answering (VQA) is the multi-modal task of answering natural language questions about an input image. Through cross-dataset adaptation methods, it is possible to transfer knowledge from a source dataset with larger train samples to a target dataset where training set is limited. Suppose a VQA model trained on one dataset train set fails in adapting to another, it is hard to identify the underlying cause of domain mismatch as there could exists a multitude of reasons such as image distribution mismatch and question distribution mismatch. At UCLA, we are working on a VQG module that facilitate in automatically generating OOD shifts that aid in systematically evaluating cross-dataset adaptation capabilities of VQA models.
Implications of the Theory of Mind in Learning: A Look from the Virtual Education Approach

This research was born as an unknown related to the recent application of Virtual Education in educational centers, not only in Costa Rica, but worldwide. All of the above for reasons of public health prevention, given the presence of the SARS-CoV-2 virus. This is how it is then tried to investigate whether there are implications in relation to the theoretical proposal of the Theory of Mind (TM) and its relationship with Distance Education (virtuality), as part of a new training model (teaching and learning).
Computationally Efficient Approximations for Matrix-based Renyi's Entropy

The recently developed matrix based Renyi's entropy enables measurement of information in data simply using the eigenspectrum of symmetric positive semi definite (PSD) matrices in reproducing kernel Hilbert space, without estimation of the underlying data distribution. This intriguing property makes the new information measurement widely adopted in multiple statistical inference and learning tasks. However, the computation of such quantity involves the trace operator on a PSD matrix $G$ to power $α$(i.e., $tr(G^α)$), with a normal complexity of nearly $O(n^3)$, which severely hampers its practical usage when the number of samples (i.e., $n$) is large. In this work, we present computationally efficient approximations to this new entropy functional that can reduce its complexity to even significantly less than $O(n^2)$. To this end, we leverage the recent progress on Randomized Numerical Linear Algebra, developing Taylor, Chebyshev and Lanczos approximations to $tr(G^α)$ for arbitrary values of $α$ by converting it into matrix-vector multiplications problem. We also establish the connection between the matrix-based Renyi's entropy and PSD matrix approximation, which enables exploiting both clustering and block low-rank structure of $G$ to further reduce the computational cost. We theoretically provide approximation accuracy guarantees and illustrate the properties of different approximations. Large-scale experimental evaluations on both synthetic and real-world data corroborate our theoretical findings, showing promising speedup with negligible loss in accuracy.
Mechanization of LAGC Semantics in Isabelle

Formal programming language semantics are imperative when trying to verify properties of programs in an automated manner. Using a new approach, Din et al. strengthen the ability of reasoning about concurrent programs by proposing a modular trace semantics, which can flexibly adapt to the most prominent imperative programming language paradigms. These semantics decouple the evaluation in the local environments from the evaluation in the global environment by generating abstract, symbolic traces for the individual, local systems. The traces are then composed and concretized, resulting in global traces for the global system. Hence, these semantics are called Locally Abstract, Globally Concrete (LAGC). In this work, we present a formalization of the LAGC semantics in the popular theorem proving environment Isabelle/HOL. The given model is based on the prior work on the theory of LAGC semantics by Din et al. and includes formalizations of the basic theorems, the LAGC semantics for the While Language (WL), as well as the LAGC semantics for an extended version of the While Language (WLEXT). We furthermore use our Isabelle model in order to provide formal proofs for several advanced properties of the LAGC semantics, which have not been analyzed in the original paper. Whilst the main goal of the work was to formalize the LAGC semantics in a mathematically rigorous manner, we also achieve a high level of proof automatization and manage to contribute an efficient code-generation for the computation of program traces. As the formalization of the semantics is highly modular, the given theories could in the future be extended with even more sophisticated programming language paradigms.
On the role of the Fibonacci matrix as key in modified ECC

In this paper, we have proposed a modified cryptographic scheme based on the application of recursive matrices as key in ECC and ElGamal. For encryption, we consider mapping analogous to affine Hill cipher in which a plaintext matrix has been constructed by points corresponding to letters on elliptic curves. In the formation of key-space, the generalized Fibonacci matrices have been taken into account, which is the sequence of matrices. The beauty of considering Fibonacci matrices is their construction where we need only two parameters(integers) in place of $n^2$ elements. The use of a recursive matrix makes a large keyspace for our proposed scheme and increases its efficiency. Thus, it reduces time as well space complexity, and its security \& strength is based on EC-DLP which is a hard problem in number theory.
Learning to Combine Instructions in LLVM Compiler

Instruction combiner (IC) is a critical compiler optimization pass, which replaces a sequence of instructions with an equivalent and optimized instruction sequence at basic block level. There can be thousands of instruction-combining patterns which need to be frequently updated as new coding idioms/applications and novel hardware evolve over time. This results in frequent updates to the IC optimization pass thereby incurring considerable human effort and high software maintenance costs. To mitigate these challenges associated with the traditional IC, we design and implement a Neural Instruction Combiner (NIC) and demonstrate its feasibility by integrating it into the standard LLVM compiler optimization pipeline. NIC leverages neural sequence-to-sequence (Seq2Seq) models for generating optimized encoded IR sequence from the unoptimized encoded IR sequence. To the best of our knowledge, ours is the first work demonstrating the feasibility of a neural instruction combiner built into a full-fledged compiler pipeline. Given the novelty of this task, we built a new dataset for training our NIC neural model. We show that NIC achieves exact match results percentage of 72% for optimized sequences as compared to traditional IC and neural machine translation metric Bleu precision score of 0.94, demonstrating its feasibility in a production compiler pipeline.
On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification

Aleatoric uncertainty captures the inherent randomness of the data, such as measurement noise. In Bayesian regression, we often use a Gaussian observation model, where we control the level of aleatoric uncertainty with a noise variance parameter. By contrast, for Bayesian classification we use a categorical distribution with no mechanism to represent our beliefs about aleatoric uncertainty. Our work shows that explicitly accounting for aleatoric uncertainty significantly improves the performance of Bayesian neural networks. We note that many standard benchmarks, such as CIFAR, have essentially no aleatoric uncertainty. Moreover, we show data augmentation in approximate inference has the effect of softening the likelihood, leading to underconfidence and profoundly misrepresenting our honest beliefs about aleatoric uncertainty. Accordingly, we find that a cold posterior, tempered by a power greater than one, often more honestly reflects our beliefs about aleatoric uncertainty than no tempering -- providing an explicit link between data augmentation and cold posteriors. We show that we can match or exceed the performance of posterior tempering by using a Dirichlet observation model, where we explicitly control the level of aleatoric uncertainty, without any need for tempering.
The Curious Layperson: Fine-Grained Image Recognition without Expert Labels

Most of us are not experts in specific fields, such as ornithology. Nonetheless, we do have general image and language understanding capabilities that we use to match what we see to expert resources. This allows us to expand our knowledge and perform novel tasks without ad-hoc external supervision. On the contrary, machines have a much harder time consulting expert-curated knowledge bases unless trained specifically with that knowledge in mind. Thus, in this paper we consider a new problem: fine-grained image recognition without expert annotations, which we address by leveraging the vast knowledge available in web encyclopedias. First, we learn a model to describe the visual appearance of objects using non-expert image descriptions. We then train a fine-grained textual similarity model that matches image descriptions with documents on a sentence-level basis. We evaluate the method on two datasets and compare with several strong baselines and the state of the art in cross-modal retrieval. Code is available at: https://github.com/subhc/clever
Integrating an electronic compass for position tracking on a wheeled tricycle mobile robot

Dead-reckoning via encoders on wheeled-mobile robots is a simple but inaccurate method to estimate position. The major drawback of encoders is wheel slippage errors that accumulate over time. This problem is often addressed by using additional sensors such as compass, gyroscope, or GPS. This paper details the integration and effectiveness of a relatively low-cost solution using an electronic compass to reduce positioning error on a wheeled tricycle mobile robot. A customised Visual Studio program has been developed to adjust the settings of the electronic compass and integrate it with the Visual Studio based robot control system. The electronic compass heading data is fused with the encoder odometry heading data in three different ways: simple fusion, linear weighted fusion, and Kalman filter fusion. Simple fusion and linear weighted fusion rely on parameters determined from angular acceleration and angular velocity, respectively. The Kalman filter uses variance data for the encoders and electronic compass to determine an optimal heading. Experiments have been conducted in an indoor corridor environment to evaluate and compare the various fusion methods. Position error is successfully reduced and is sufficient to locate the robot within the corridor.
An unsupervised extractive summarization method based on multi-round computation

In recent years, text summarization methods have attracted much attention again thanks to the researches on neural network models. Most of the current text summarization methods based on neural network models are supervised methods which need large-scale datasets. However, large-scale datasets are difficult to obtain in practical applications. In this paper, we model the task of extractive text summarization methods from the perspective of Information Theory, and then describe the unsupervised extractive methods with a uniform framework. To improve the feature distribution and to decrease the mutual information of summarization sentences, we propose a new sentence extraction strategy which can be applied to existing unsupervised extractive methods. Experiments are carried out on different datasets, and results show that our strategy is indeed effective and in line with expectations.
Astronomical Image Colorization and upscaling with Generative Adversarial Networks

Automatic colorization of images without human intervention has been a subject of interest in the machine learning community for a brief period of time. Assigning color to an image is a highly ill-posed problem because of its innate nature of possessing very high degrees of freedom; given an image, there is often no single color-combination that is correct. Besides colorization, another problem in reconstruction of images is Single Image Super Resolution, which aims at transforming low resolution images to a higher resolution. This research aims to provide an automated approach for the problem by focusing on a very specific domain of images, namely astronomical images, and process them using Generative Adversarial Networks (GANs). We explore the usage of various models in two different color spaces, RGB and L*a*b. We use transferred learning owing to a small data set, using pre-trained ResNet-18 as a backbone, i.e. encoder for the U-net and fine-tune it further. The model produces visually appealing images which hallucinate high resolution, colorized data in these results which does not exist in the original image. We present our results by evaluating the GANs quantitatively using distance metrics such as L1 distance and L2 distance in each of the color spaces across all channels to provide a comparative analysis. We use Frechet inception distance (FID) to compare the distribution of the generated images with the distribution of the real image to assess the model's performance.
Faster Algorithms and Constant Lower Bounds for the Worst-Case Expected  Error

The study of statistical estimation without distributional assumptions on data values, but with knowledge of data collection methods was recently introduced by Chen, Valiant and Valiant (NeurIPS 2020). In this framework, the goal is to design estimators that minimize the worst-case expected error. Here the expectation is over a known, randomized data collection process from some population, and the data values corresponding to each element of the population are assumed to be worst-case. Chen, Valiant and Valiant show that, when data values are $\ell_{\infty}$-normalized, there is a polynomial time algorithm to compute an estimator for the mean with worst-case expected error that is within a factor $\frac{\pi}{2}$ of the optimum within the natural class of semilinear estimators. However, their algorithm is based on optimizing a somewhat complex concave objective function over a constrained set of positive semidefinite matrices, and thus does not come with explicit runtime guarantees beyond being polynomial time in the input. In this paper we design provably efficient algorithms for approximating the optimal semilinear estimator based on online convex optimization. In the setting where data values are $\ell_{\infty}$-normalized, our algorithm achieves a $\frac{\pi}{2}$-approximation by iteratively solving a sequence of standard SDPs. When data values are $\ell_2$-normalized, our algorithm iteratively computes the top eigenvector of a sequence of matrices, and does not lose any multiplicative approximation factor. We complement these positive results by stating a simple combinatorial condition which, if satisfied by a data collection process, implies that any (not necessarily semilinear) estimator for the mean has constant worst-case expected error.