Reviewerly/exHarmony huggingface dataset:
From MNIST to ImageNet and Back: Benchmarking Continual Curriculum Learning

Continual learning (CL) is one of the most promising trends in recent machine learning research. Its goal is to go beyond classical assumptions in machine learning and develop models and learning strategies that present high robustness in dynamic environments. The landscape of CL research is fragmented into several learning evaluation protocols, comprising different learning tasks, datasets, and evaluation metrics. Additionally, the benchmarks adopted so far are still distant from the complexity of real-world scenarios, and are usually tailored to highlight capabilities specific to certain strategies. In such a landscape, it is hard to objectively assess strategies. In this work, we fill this gap for CL on image data by introducing two novel CL benchmarks that involve multiple heterogeneous tasks from six image datasets, with varying levels of complexity and quality. Our aim is to fairly evaluate current state-of-the-art CL strategies on a common ground that is closer to complex real-world scenarios. We additionally structure our benchmarks so that tasks are presented in increasing and decreasing order of complexity -- according to a curriculum -- in order to evaluate if current CL models are able to exploit structure across tasks. We devote particular emphasis to providing the CL community with a rigorous and reproducible evaluation protocol for measuring the ability of a model to generalize and not to forget while learning. Furthermore, we provide an extensive experimental evaluation showing that popular CL strategies, when challenged with our benchmarks, yield sub-par performance, high levels of forgetting, and present a limited ability to effectively leverage curriculum task ordering. We believe that these results highlight the need for rigorous comparisons in future CL works as well as pave the way to design new CL strategies that are able to deal with more complex scenarios.
Iterative Partial Fulfillment of Counterfactual Explanations: Benefits and Risks

Counterfactual (CF) explanations, also known as contrastive explanations and algorithmic recourses, are popular for explaining machine learning models in high-stakes domains. For a subject that receives a negative model prediction (e.g., mortgage application denial), the CF explanations are similar instances but with positive predictions, which informs the subject of ways to improve. While their various properties have been studied, such as validity and stability, we contribute a novel one: their behaviors under iterative partial fulfillment (IPF). Specifically, upon receiving a CF explanation, the subject may only partially fulfill it before requesting a new prediction with a new explanation, and repeat until the prediction is positive. Such partial fulfillment could be due to the subject's limited capability (e.g., can only pay down two out of four credit card accounts at this moment) or an attempt to take the chance (e.g., betting that a monthly salary increase of \$800 is enough even though \$1,000 is recommended). Does such iterative partial fulfillment increase or decrease the total cost of improvement incurred by the subject? We mathematically formalize IPF and demonstrate, both theoretically and empirically, that different CF algorithms exhibit vastly different behaviors under IPF. We discuss implications of our observations, advocate for this factor to be carefully considered in the development and study of CF algorithms, and give several directions for future work.
Adversarial Attacks against Binary Similarity Systems

In recent years, binary analysis gained traction as a fundamental approach to inspect software and guarantee its security. Due to the exponential increase of devices running software, much research is now moving towards new autonomous solutions based on deep learning models, as they have been showing state-of-the-art performances in solving binary analysis problems. One of the hot topics in this context is binary similarity, which consists in determining if two functions in assembly code are compiled from the same source code. However, it is unclear how deep learning models for binary similarity behave in an adversarial context. In this paper, we study the resilience of binary similarity models against adversarial examples, showing that they are susceptible to both targeted and untargeted attacks (w.r.t. similarity goals) performed by black-box and white-box attackers. In more detail, we extensively test three current state-of-the-art solutions for binary similarity against two black-box greedy attacks, including a new technique that we call Spatial Greedy, and one white-box attack in which we repurpose a gradient-guided strategy used in attacks to image classifiers.
TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalization

Recent years have seen the ever-increasing importance of pre-trained models and their downstream training in deep learning research and applications. At the same time, the defense for adversarial examples has been mainly investigated in the context of training from random initialization on simple classification tasks. To better exploit the potential of pre-trained models in adversarial robustness, this paper focuses on the fine-tuning of an adversarially pre-trained model in various classification tasks. Existing research has shown that since the robust pre-trained model has already learned a robust feature extractor, the crucial question is how to maintain the robustness in the pre-trained model when learning the downstream task. We study the model-based and data-based approaches for this goal and find that the two common approaches cannot achieve the objective of improving both generalization and adversarial robustness. Thus, we propose a novel statistics-based approach, Two-WIng NormliSation (TWINS) fine-tuning framework, which consists of two neural networks where one of them keeps the population means and variances of pre-training data in the batch normalization layers. Besides the robust information transfer, TWINS increases the effective learning rate without hurting the training stability since the relationship between a weight norm and its gradient norm in standard batch normalization layer is broken, resulting in a faster escape from the sub-optimal initialization and alleviating the robust overfitting. Finally, TWINS is shown to be effective on a wide range of image classification datasets in terms of both generalization and robustness. Our code is available at https://github.com/ziquanliu/CVPR2023-TWINS.
Finding Similar Exercises in Retrieval Manner

When students make a mistake in an exercise, they can consolidate it by ``similar exercises'' which have the same concepts, purposes and methods. Commonly, for a certain subject and study stage, the size of the exercise bank is in the range of millions to even tens of millions, how to find similar exercises for a given exercise becomes a crucial technical problem. Generally, we can assign a variety of explicit labels to the exercise, and then query through the labels, but the label annotation is time-consuming, laborious and costly, with limited precision and granularity, so it is not feasible. In practice, we define ``similar exercises'' as a retrieval process of finding a set of similar exercises based on recall, ranking and re-rank procedures, called the \textbf{FSE} problem (Finding similar exercises). Furthermore, comprehensive representation of the semantic information of exercises was obtained through representation learning. In addition to the reasonable architecture, we also explore what kind of tasks are more conducive to the learning of exercise semantic information from pre-training and supervised learning. It is difficult to annotate similar exercises and the annotation consistency among experts is low. Therefore this paper also provides solutions to solve the problem of low-quality annotated data. Compared with other methods, this paper has obvious advantages in both architecture rationality and algorithm precision, which now serves the daily teaching of hundreds of schools.
Parent Hamiltonian reconstruction via inverse quantum annealing

Finding a local Hamiltonian having a given many-body wavefunction as its ground state is a serious challenge of fundamental importance in quantum technologies. Here we introduce a method, inspired by quantum annealing, that efficiently performs this task through an artificial inverse dynamics: a slow deformation of the state generates an adiabatic evolution of the corresponding Hamiltonian. We name this approach 'inverse quantum annealing'. This method only requires the knowledge of local expectation values. As an example, we apply inverse quantum annealing to find the local Hamiltonian of fermionic Gaussian states.
Towards Goldilocks Zone in Child-centered AI

Using YouTube Kids as an example, in this work, we argue the need to understand a child's interaction process with AI and its broader implication on a child's emotional, social, and creative development. We present several design recommendations to create value-driven interaction in child-centric AI that can guide designing compelling, age-appropriate, beneficial AI experiences for children.
Learning to Generate 3D Representations of Building Roofs Using Single-View Aerial Imagery

We present a novel pipeline for learning the conditional distribution of a building roof mesh given pixels from an aerial image, under the assumption that roof geometry follows a set of regular patterns. Unlike alternative methods that require multiple images of the same object, our approach enables estimating 3D roof meshes using only a single image for predictions. The approach employs the PolyGen, a deep generative transformer architecture for 3D meshes. We apply this model in a new domain and investigate the sensitivity of the image resolution. We propose a novel metric to evaluate the performance of the inferred meshes, and our results show that the model is robust even at lower resolutions, while qualitatively producing realistic representations for out-of-distribution samples.
Non-Euclidean Monotone Operator Theory and Applications

While monotone operator theory is traditionally studied on Hilbert spaces, many interesting problems in data science and machine learning arise naturally in finite-dimensional vector spaces endowed with non-Euclidean norms, such as diagonally-weighted $\ell_1$ or $\ell_\infty$ norms. This paper provides a natural generalization of monotone operator theory to finite-dimensional non-Euclidean spaces. The key tools are weak pairings and logarithmic norms. We show that the resolvent and reflected resolvent operators of non-Euclidean monotone mappings exhibit similar properties to their counterparts in Hilbert spaces. Furthermore, classical iterative methods and splitting methods for finding zeros of monotone operators are shown to converge in the non-Euclidean case. We apply our theory to equilibrium computation and Lipschitz constant estimation of recurrent neural networks, obtaining novel iterations and tighter upper bounds via forward-backward splitting.
Memorization Capacity of Neural Networks with Conditional Computation

Many empirical studies have demonstrated the performance benefits of conditional computation in neural networks, including reduced inference time and power consumption. We study the fundamental limits of neural conditional computation from the perspective of memorization capacity. For Rectified Linear Unit (ReLU) networks without conditional computation, it is known that memorizing a collection of $n$ input-output relationships can be accomplished via a neural network with $O(\sqrt{n})$ neurons. Calculating the output of this neural network can be accomplished using $O(\sqrt{n})$ elementary arithmetic operations of additions, multiplications and comparisons for each input. Using a conditional ReLU network, we show that the same task can be accomplished using only $O(\log n)$ operations per input. This represents an almost exponential improvement as compared to networks without conditional computation. We also show that the $\Theta(\log n)$ rate is the best possible. Our achievability result utilizes a general methodology to synthesize a conditional network out of an unconditional network in a computationally-efficient manner, bridging the gap between unconditional and conditional architectures.
Tangent Bundle Convolutional Learning: from Manifolds to Cellular Sheaves and Back

In this work we introduce a convolution operation over the tangent bundle of Riemann manifolds in terms of exponentials of the Connection Laplacian operator. We define tangent bundle filters and tangent bundle neural networks (TNNs) based on this convolution operation, which are novel continuous architectures operating on tangent bundle signals, i.e. vector fields over the manifolds. Tangent bundle filters admit a spectral representation that generalizes the ones of scalar manifold filters, graph filters and standard convolutional filters in continuous time. We then introduce a discretization procedure, both in the space and time domains, to make TNNs implementable, showing that their discrete counterpart is a novel principled variant of the very recently introduced sheaf neural networks. We formally prove that this discretized architecture converges to the underlying continuous TNN. Finally, we numerically evaluate the effectiveness of the proposed architecture on various learning tasks, both on synthetic and real data.
Moderate deviation expansion for fully quantum tasks

The moderate deviation regime is concerned with the finite block length trade-off between communication cost and error for information processing tasks in the asymptotic regime, where the communication cost approaches a capacity-like quantity and the error vanishes at the same time. We find exact characterisations of these trade-offs for a variety of fully quantum communication tasks, including quantum source coding, quantum state splitting, entanglement-assisted quantum channel coding, and entanglement-assisted quantum channel simulation. The main technical tool we derive is a tight relation between the partially smoothed max-information and the hypothesis testing relative entropy. This allows us to obtain the expansion of the partially smoothed max-information for i.i.d. states in the moderate deviation regime.
Topological Hoarded Graphs

In this paper, we first introduced the steps to be taken to get the set-family corresponding to a hoarded graph, and an example implementation of these steps. We then give the notion of topological hoarded graph and show when a set-family induced by a topological hoarded graph is a topology on a set. We also present some useful facts about topological hoarded graphs.
A Novel Robust Iterated CKF for GNSS/SINS Integrated Navigation Applications

Abstract In challenging circumstances, the estimation performance of integrated navigation parameters for tightly coupled GNSS/SINS is impacted by outlier measurements. An effective solution that employs a novel iterative sigma-point structure with a modified robustness optimization approach for enhancing the error compensation effectiveness and robustness of filters utilized in GNSS challenge conditions is proposed in this paper. The proposed method modifies the CKF scheme by incorporating nonlinear regression and numerous iteration processes for ameliorating error compensation. Subsequently, a loss function and penalty mechanism are implemented to enhance the filter's robustness to outlier measurements. Further, to fully absorb the valid information of the innovation and speed up the operation of the proposed method, the outlier measurement detection criteria is established to bypass the penalty mechanism against measurement weights in the absence of outliers in GNSS measurements. Field experiments demonstrate that the proposed method outperforms traditional methods in mitigating navigation errors when multipath errors and non-line-of-sight (NLOS) reception are increased.
Mining for Conserved Constituents and Quantities

Explaining the presence or absence of transformations in nature, such as chemical or elementary particle reactions, is fundamental to our thinking about nature. This paper describes a generic approach to the search for such conserved quantities. In the work that follows we formulate a generic approach to conserved such explanations by summoning techniques from Linear Algebra.
The Mathematical Foundation of the Pareto Rule

Vilfredo Pareto (1848-1923) once said: “For many events, roughly 80% of the effects come from 20% of the causes.”In this brief paper, I provide the well known AT Math to show why the Pareto Principle works.
Representation of Nigerian Internet Scamsters in selected Nigerian and International News Reports

Linguistic studies on Nigerian internet scamsters have dwelt on the aspects of language use. These studies focused on the linguistic features, slangy expressions, discourse and persuasive strategies used by scamsters in scam messages. However, the aspect of media representation of the social actors of Nigerian internet fraud has not been given adequate attention. This study, therefore, explores the representation of social actors of Nigerian internet fraud in Nigerian and international news reports. Headlines and news contents are purposively sampled from forty-five news articles published between 2019 and 2021, from two Nigerian newspapers and two international news media. Guided by Halliday’s Systemic Functional Grammar (SFG) and Van Leeuwen’s early work on social actor representation (1996), ten forms of representations were identified. Nigerian Internet Scamsters were represented as educated, role models, imposters, parentally supported, abductors, wrong ambassadors/negative nationals, friends to the security agents, desperate/money ritualists, guilty and convicted, exploiters and victimizers. The negative connotation of these labels indicates that the media are not neutral in their representation of scammers. This study shows that the representation of the social actions and actors in internet fraud are linguistically and discursively framed to fit their ideology.
Sentiment Analysis and Trend Detection of Tweets using Machine Learning Techniques

Social media has become the most important source for decisionmaking procedures in many areas due to its wide applications.People all over the world use their accounts to express personal views, experiences and opinions on diverse topics.Tweets on Twitter are mainly based on the public opinion on a product, event or topic and thus hold large volumes of unprocessed data.Researchers classify and cluster twitter data for different purposes such as Sentiment Analysis, topic detection, topic tracking, culture propagation, opinion mining and others.Analysis of this data is important and difficult due to the size of the dataset.In this review article, we cover most of research articles in tweets classification and clustering.We also address machine learning algorithms for analyzing the data.
Method of Performing Machine Learning

Currently, there is no one common measure to judge machine learning methods and results.Through, this paper, I introduce such a measure to judge quality of machine learning methods and results.We just measure machine learning algorithms based on accuracy of prediction.If there is a common measure to define quality of machine learning methods and results then it could be used in different situations like interdisciplinary research.Moreover, the results are just limited to the data that we have in hand and any change in data changes the machine learning methods and results.Therefore, to become independent of data a measure is required to show the quality of methods in a way that is independent of data.This may be the first paper to present such a measure.When a measure independent of data is present, it will help us in determining the applicability of machine learning methods and results in a way that is independent of data.Correctness is such a measure.When I say correctness, I mean justified mathematically.
A Detailed Survey of Image Steganography Methods

Security systems are very popular in many places because technology is getting better and better every day.Information security can be achieved by utilizing encryption and steganography.Cryptography is typically associated with scrambling plain text into cipher ext and back again in today's world.Maintaining the safety of confidential information has always been a major concern, from ancient times to the present.As a result, numerous methods for securely transferring data were developed, one of which is steganography.Steganography takes cryptography one step further by concealing an encrypted message so that no one is aware of its existence.The primary goal of Steganography is to conceal the embedded data's existence.Due to the remarkable advancement in computational power, the steganography technique has enhanced the security of existing methods for hiding data.Steganography system fundamentals, evaluation methods, and security aspects, as well as various spatial and transform domain embedding schemes, are discussed in this article.In addition, a brief overview of various image steganography methods is provided in this paper.
Fuzzy Elgamal Technique for Securing Data Over Cloud

Cloud computing is the most popular computing approach among the scientists and engineers and lots of data are uploaded and downloaded from the cloud servers across 365x24x7 hours using high speed internet services.As the hackers are also watching transfer of the important data especially related to digital currency, hence, there is need to develop secure technique for uploading and downloading the data in secure manner.The present work is an attempt in this direction and Elgamal approach has been converted into the fuzzy based Elgamal approach for secure access of the data through cloud servers.The approach is tested by taking a case study and the results are reported in the form of tables and graphs.
Educational Technology to Support Reading Comprehension and Writing Competency Processes Through the Use of Writing in Virtual Learning Environments

Information and Communication Technologies (ICT) have energized communication and facilitated interaction in society, eliminating spatial and temporal barriers and becoming an essential tool for the contemporary world. In the educational field, reading comprehension and writing skills, approached through purposeful writing practices mediated by ICTs, can have a significant impact on teaching and learning processes, for which it is essential that teachers encourage the development of digital and media competence through the renewal of their pedagogical practices, and guide students in the assertive use of ICTs so that they can participate in the construction of their own knowledge and contribute to the improvement of society.
Alternative Approach to Achieve a Solution of Derangement Problems by Dynamic Programming

Derangement is one well-known problem in the filed of probability theory. An in- stance of a derangement problem contains a finite collection C of n paired objects, C = {(x1 , y1 ), ..., (xn , yn )}. The derangement problem asks how many ways to gener- ate a new collection C′ ̸= C such that for each (xi,yj) ∈ C′,i ̸= j. We propose an efficient dynamic programming algorithm that divides an instance of the derangement problem into several subproblems. During a recursive process of unrolling a subproblem, there exists a repeated procedure that allows us to make a use of a subsolution that has already been computed. We present the methodology to formulate a concept of this subproblem as well as parts of designing and analyzing an efficiency of the proposed algorithm.