Reviewerly/exHarmony huggingface dataset:
Analytics of machine learning-based algorithms for text classification

Text classification is the most vital area in natural language processing in which text data is automatically sorted into a predefined set of classes. The application of text classification is wide in commercial works like spam filtering, decision making, extracting information from raw data, and many other applications. Text classification is more significant for many enterprises since it eliminates the need for manual data classification, a more expensive and time-consuming mechanism. In this paper, a comparative analysis of text classification is done in which the efficiency of different machine learning algorithms on different datasets is analyzed and compared. Support Vector Machine (SVM), k-Nearest Neighbor (k-NN), Logistic Regression (LR), Multinomial Naïve Bayes (MNB), and Random Forest (RF) are Machine Learning based algorithms used in this work. Two different datasets are used to make a comparative analysis of these algorithms. This paper further analyzes the machine learning techniques employed for text classification on the basis of performance metrics viz accuracy, precision, recall and f1- score. The resullltsss reveals that Logistic Regression and Support Vector Machine outperforms the other models in the IMDB dataset, and kNN outperforms the other models for the SPAM dataset as per the results obtained from the proposed system.
Evaluating Opportunities for Design Capture

A key motive for developing ways to structure and preserve records of decision making in software development is the belief that the high cost of system maintenance could be reduced by providing access to these records. Development projects vary widely. Tools and methods can be appropriate for one development environment but not for another. In this chapter, I describe four kinds of software development environments and outline the opportunities and obstacles for using design rationale in each. Some development projects have good reason to avoid investing resources early in development, even when great benefits may accrue later. Many off-the-shelf product development efforts are arguably of this nature. In other projects, such as customized software development, the value of upstream investment seems more compelling. I reflect on the difficulty of learning from failed projects and on the need for researchers to distinguish carefully between the requirements of science and those of engineering.
Symmetry-adapted representation learning

In this paper, we propose the use of data symmetries, in the sense of equivalences under signal transformations, as priors for learning symmetry-adapted data representations, i.e., representations that are equivariant to these transformations. We rely on a group-theoretic definition of equivariance and provide conditions for enforcing a learned representation, for example the weights in a neural network layer or the atoms in a dictionary, to have the structure of a group and specifically the group structure in the distribution of the input. By reducing the analysis of generic group symmetries to permutation symmetries, we devise a regularization scheme for representation learning algorithm, using an unlabeled training set. The proposed regularization is aimed to be a conceptual, theoretical and computational proof of concept for symmetry-adapted representation learning, where the learned data representations are equivariant or invariant to transformations, without explicit knowledge of the underlying symmetries in the data.
SemIndex+: A semantic indexing scheme for structured, unstructured, and partly structured data

While Information Retrieval (IR) systems have gained success in Web-style search engines in the past two decades, nonetheless, the DataBase (DB) paradigm remains prevalent in handling data in enterprise environments and digital libraries, and is gaining even more importance in the Semantic Web with the increasing need to handle partly structured (NoSQL) data. This paper describes SemIndex+, a semantic-aware indexing and querying framework that allows semantic search, result selection, and result ranking of structured (relational DB-style), unstructured (IR-style), and partly structured (NoSQL) data. Various weighting functions and a parallelized search algorithm have been developed for that purpose and are presented here. We provide a general keyword query model allowing the user to choose the results’ semantic coverage and expressiveness based on her needs. Different from alternative solutions involving query relaxation, query refinement, or query disambiguation, our approach incorporates semantics at the most basic data indexing level: providing more opportunities toward speedups and semantic coverage. An extensive experimental evaluation, comparing SemIndex+ with alternative methods, highlights our approach’s flexibility and effectiveness, which in turn impact efficiency (requiring less or more time following the user specified index and query semantic coverages).
Vision-based estimation of excavator manipulator pose for automated grading control

As an effective alternative to manual operation of hydraulic excavators, automated systems are being developed for common tasks that excavators routinely perform on typical job sites. An example is an automated ground grading system that can enhance the productivity of excavators by assisting the operator to perform ground grading in a fast and accurate manner. To develop an automated grading system, sensors are needed to measure the position of the hydraulic manipulator linkages for the feedback control system. In this paper, a novel method to estimate the pose of a hydraulic manipulator using a vision-based neural network system is presented. A webcam is used to capture images of a moving manipulator, and the captured images are used to train a neural network. Then, the trained neural network can be used to estimate the pose of the excavator manipulator for the feedback control system. A simulation study shows a stable grading performance when a PI controller is used to control the manipulator based on the estimated manipulator pose.
Mind the Gap - A Closer Look at the Security of Block Ciphers against Differential Cryptanalysis

Resistance against differential cryptanalysis is an important design criteria for any modern block cipher and most designs rely on finding some upper bound on probability of single differential characteristics. However, already at EUROCRYPT’91, Lai et al. comprehended that differential cryptanalysis rather uses differentials instead of single characteristics. In this paper, we consider exactly the gap between these two approaches and investigate this gap in the context of recent lightweight cryptographic primitives. This shows that for many recent designs like Midori, Skinny or Sparx one has to be careful as bounds from counting the number of active S-boxes only give an inaccurate evaluation of the best differential distinguishers. For several designs we found new differential distinguishers and show how this gap evolves. We found an 8-round differential distinguisher for Skinny-64 with a probability of $$2^{-56.93}$$ , while the best single characteristic only suggests a probability of $$2^{-72}$$ . Our approach is integrated into publicly available tools and can easily be used when developing new cryptographic primitives. Moreover, as differential cryptanalysis is critically dependent on the distribution over the keys for the probability of differentials, we provide experiments for some of these new differentials found, in order to confirm that our estimates for the probability are correct. While for Skinny-64 the distribution over the keys follows a Poisson distribution, as one would expect, we noticed that Speck-64 follows a bimodal distribution, and the distribution of Midori-64 suggests a large class of weak keys.
On the Assessment of a Numerical Weather Prediction Model for Solar Photovoltaic Power Forecasts in Cities

Recent developments in the weather research and forecasting (WRF) model have made it possible to accurately estimate incident solar radiation. This study couples the WRF-solar modifications with a multilayer urban canopy and building energy model (BEM) to create a unified WRF forecasting system called urban WRF–solar (uWRF-solar). This paper tests the integrated approach in the New York City (NYC) metro region as a sample case. Hourly forecasts are validated against ground station data collected at ten different sites in and around the city. Validation is carried out independently for clear, cloudy, and overcast sky conditions. Results indicate that the uWRF-solar model can forecast solar irradiance considerably well for the global horizontal irradiance (GHI) with an R2 value of 0.93 for clear sky conditions, 0.61 for cloudy sky conditions, and finally, 0.39 for overcast conditions. Results are further used to directly forecast solar power production in the region of interest, where evaluations of generation potential are done at the city scale. Outputs show a gradient of power generation produced by the potential available solar energy on the entire uWRF-solar grid. In total, the city has a city photovoltaic (PV) potential of 118 kWh/day/m2 and 3.65 MWh/month/m2.
Simultaneous Dimensionality Reduction and Classification via Dual Embedding Regularized Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is a well-known paradigm for data representation. Traditional NMF-based classification methods first perform NMF or one of its variants on input data samples to obtain their low-dimensional representations, which are successively classified by means of a typical classifier [e.g., k -nearest neighbors (KNN) and support vector machine (SVM)]. Such a stepwise manner may overlook the dependency between the two processes, resulting in the compromise of the classification accuracy. In this paper, we elegantly unify the two processes by formulating a novel constrained optimization model, namely dual embedding regularized NMF (DENMF), which is semi-supervised. Our DENMF solution simultaneously finds the low-dimensional representations and assignment matrix via joint optimization for better classification. Specifically, input data samples are projected onto a couple of low-dimensional spaces (i.e., feature and label spaces), and locally linear embedding is employed to preserve the identical local geometric structure in different spaces. Moreover, we propose an alternating iteration algorithm to solve the resulting DENMF, whose convergence is theoretically proven. Experimental results over five benchmark datasets demonstrate that DENMF can achieve higher classification accuracy than state-of-the-art algorithms.
Review and Analysis of Zero, One and Few Shot Learning Approaches

Machine Learning (ML) has come a long way with a neural networks based genre of ML algorithms, Deep Learning, that claims near-human performances in certain tasks in domains such as visual concept learning. While humans can efficiently learn new concepts with just one or few exemplars, most current generation ML algorithms need large datasets to train for effective concept learning. Visual concept learning is especially data hungry as computer vision is yet to mature in comparison with human vision. Human vision has far efficient concept learning even from fewer exemplars due to rich cognitive processing. ML Algorithms capable of learning from fewer examples has becoming pressing need as ML enters mainstream domains such as healthcare where it may be nearly impossible (for ex. rare disease prediction) or cost prohibitive to obtain a larger training data. Few-shot learning is desirable even when larger datasets are available as labeling data can be time consuming and training on larger data can be computationally expensive. There have been several approaches to learn with fewer labeled samples with diversity in the modeling (Shallow models, Bayesian networks and Neural Networks), in the training (domain adaptation to transfer learning, to associative memory based training), task domains (Visual concept learning, motor control tasks in robotics) and type of data (Symbolic images, real world images, Speech, etc.) This paper reviews the diverse approaches that effectively learn models for problems that lack larger training data. The approaches are broadly categorized into the data-bound approaches and learning-bound approaches for easier comprehension of current state of art. Approaches are categorized & compared for better analysis and to identify the future directions in few shot learning. This paper also intends to disambiguate several related terms in the context of few shot learning.
An Improved Collaborative Filtering Recommendation Algorithm and Recommendation Strategy

The e-commerce recommendation system mainly includes content recommendation technology, collaborative filtering recommendation technology, and hybrid recommendation technology. The collaborative filtering recommendation technology is a successful application of personalized recommendation technology. However, due to the sparse data and cold start problems of the collaborative recommendation technology and the continuous expansion of data scale in e-commerce, the e-commerce recommendation system also faces many challenges. This paper has conducted useful exploration and research on the collaborative recommendation technology. Firstly, this paper proposed an improved collaborative filtering algorithm. Secondly, the community detection algorithm is investigated, and two overlapping community detection algorithms based on the central node and k-based faction are proposed, which effectively mine the community in the network. Finally, we select a part of user communities from the user network projected by the user-item network as the candidate neighboring user set for the target user, thereby reducing calculation time and increasing recommendation speed and accuracy of the recommendation system. This paper has a perfect combination of social network technology and collaborative filtering technology, which can greatly increase recommendation system performance. This paper used the MovieLens dataset to test two performance indexes which include MAE and RMSE. The experimental results show that the improved collaborative filtering algorithm is superior to other two collaborative recommendation algorithms for MAE and RMSE performance.
Spatio-temporal solar exergoeconomic and exergoenvironmental maps for photovoltaic systems

This work introduces and develops the concepts of solar exergoeconomic and exergoenvironmental maps for photovoltaic systems. These maps are developed to complement the information provided by solar exergy maps for recognizing the most appropriate sites for hosting photovoltaic power plants. A detailed case study is carried out to demonstrate how the proposed approaches can be practically applied to promote solar photovoltaic systems. More specifically, the exergoeconomic and exergoenvironmental performances of a 100 kW grid-connected photovoltaic plant are assessed for several Iranian cities over the year based on actual climatic, economic, and environmental impact data. The spatio-temporal solar maps for unit exergoeconomic cost and unit exergoenvironmental impact of electricity generation are then developed for the investigated climatic conditions for the first time. In order to demonstrate the suitability and reliability of the developed maps, their interoperations are assessed and compared to those for exergy efficiency. The results show that solar exergoeconomic and exergoenvironmental maps provide more valuable information on the performance of solar photovoltaic systems than exergy efficiency maps, demonstrating their robustness for optimum site selection and plant sizing. In addition, the exergy efficiency maps may result in inappropriate site selection of photovoltaic solar farms. The central, south, southeast, and some west parts of Iran are the most suitable locations for implementing photovoltaic power fields from exergoeconomic and exergoenvironmental viewpoints. The developed approaches can also be used for locating and sizing other renewable energy systems under different climatic conditions.
One Size Does Not Fit All: Comparing

Nadir Durrani, Fahim Dalvi, Hassan Sajjad, Yonatan Belinkov, Preslav Nakov. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.
ARTHuS: Adaptive Real-Time Human Segmentation in Sports Through Online Distillation

Semantic segmentation can be regarded as a useful tool for global scene understanding in many areas, including sports, but has inherent difficulties, such as the need for pixel-wise annotated training data and the absence of well-performing real-time universal algorithms. To alleviate these issues, we sacrifice universality by developing a general method, named ARTHuS, that produces adaptive real-time match-specific networks for human segmentation in sports videos, without requiring any manual annotation. This is done by an online knowledge distillation process, in which a fast student network is trained to mimic the output of an existing slow but effective universal teacher network, while being periodically updated to adjust to the latest play conditions. As a result, ARTHuS allows to build highly effective real-time human segmentation networks that evolve through the match and that sometimes outperform their teacher. The usefulness of producing adaptive match-specific networks and their excellent performances are demonstrated quantitatively and qualitatively for soccer and basketball matches.
Color difference classification of solid color printing and dyeing products based on optimization of the extreme learning machine of the improved whale optimization algorithm

To mitigate the problem of low classification accuracy in solid color printing and dyeing, a color difference classification model based on the differential evolution (DE) improved whale optimization algorithm (WOA) for extreme learning machine (ELM) optimization, named the DE–WOA–ELM, was developed in this study. Considering that the initial population of the WOA has a significant influence on the solution speed and quality, DE was used to generate a more suitable initial population for the WOA by avoiding local optima, thereby improving the performance. The method used an excellent global search ability to improve the WOA for optimization and obtained an optimal parameter combination for the ELM. Thus, the problem of randomly initializing the input weight and the hidden layer bias of the ELM, which leads to a nonuniform training model and unstable algorithm, was solved. Finally, by optimizing the input weight and hidden layer bias, the color difference classification model of the ELM with a strong generalization ability was constructed. The results of the color difference classification experiments on fabric images collected under standard light sources show that the average classification accuracy for the dataset is increased by 2.15%, 11.06%, 12.11%, and 0.47% compared with those of the ELM, support vector machine, back propagation neural network, and kernel ELM, respectively.
VC Classes are Adversarially Robustly Learnable, but Only Improperly

We study the question of learning an adversarially robust predictor. We show that any hypothesis class $\mathcal{H}$ with finite VC dimension is robustly PAC learnable with an improper learning rule. The requirement of being improper is necessary as we exhibit examples of hypothesis classes $\mathcal{H}$ with finite VC dimension that are not robustly PAC learnable with any proper learning rule.
Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization

Optimization of Binarized Neural Networks (BNNs) currently relies on real-valued latent weights to accumulate small update steps. In this paper, we argue that these latent weights cannot be treated analogously to weights in real-valued networks. Instead their main role is to provide inertia during training. We interpret current methods in terms of inertia and provide novel insights into the optimization of BNNs. We subsequently introduce the first optimizer specifically designed for BNNs, Binary Optimizer (Bop), and demonstrate its performance on CIFAR-10 and ImageNet. Together, the redefinition of latent weights as inertia and the introduction of Bop enable a better understanding of BNN optimization and open up the way for further improvements in training methodologies for BNNs.
Bayesian Layers: A Module for Neural Network Uncertainty

We describe Bayesian Layers, a module designed for fast experimentation with neural network uncertainty. It extends neural network libraries with drop-in replacements for common layers. This enables composition via a unified abstraction over deterministic and stochastic functions and allows for scalability via the underlying system. These layers capture uncertainty over weights (Bayesian neural nets), pre-activation units (dropout), activations (``stochastic output layers''), or the function itself (Gaussian processes). They can also be reversible to propagate uncertainty from input to output. We include code examples for common architectures such as Bayesian LSTMs, deep GPs, and flow-based models. As demonstration, we fit a 5-billion parameter ``Bayesian Transformer'' on 512 TPUv2 cores for uncertainty in machine translation and a Bayesian dynamics model for model-based planning. Finally, we show how Bayesian Layers can be used within the Edward2 language for probabilistic programming with stochastic processes.
Fast Convergence of Natural Gradient Descent for Over-Parameterized Neural Networks

Natural gradient descent has proven very effective at mitigating the catastrophic effects of pathological curvature in the objective function, but little is known theoretically about its convergence properties, especially for \emph{non-linear} networks. In this work, we analyze for the first time the speed of convergence to global optimum for natural gradient descent on non-linear neural networks with the squared error loss. We identify two conditions which guarantee the global convergence: (1) the Jacobian matrix (of network's output for all training cases w.r.t the parameters) is full row rank and (2) the Jacobian matrix is stable for small perturbations around the initialization. For two-layer ReLU neural networks (i.e. with one hidden layer), we prove that these two conditions do hold throughout the training under the assumptions that the inputs do not degenerate and the network is over-parameterized. We further extend our analysis to more general loss function with similar convergence property. Lastly, we show that K-FAC, an approximate natural gradient descent method, also converges to global minima under the same assumptions.
A generalised fuzzy cognitive mapping approach for modelling complex systems

Fuzzy Cognitive Maps (FCMs) were developed as a tool for capturing and modelling the behaviour of qualitative system dynamics. However, several drawbacks have been identified that limit FCMs ability in simulating the behaviour of qualitative system. This paper addresses the limitations of FCMs in modelling complex qualitative system dynamics and proposes a generalised Fuzzy Cognitive Mapping (FCM) approach that is able to overcome those limitations. This approach uses fuzzy rules to represent the dynamics of concepts and relations, including time dynamics of relations and introduces a multistep simulation approach that can use several single layer perceptrons to simulate the dynamics of concepts and relations overtime. This approach also incorporates the fuzziness and ambiguity widely associated with expert knowledge when representing and simulating the dynamics of concepts and relations. In this paper, the design of the proposed generalised FCM approach is explained and demonstrated for a real-world case of the consequences of high intensity rainfall in Kampala City, Uganda. This generalised FCM approach creates a new perspective and an alternative approach to model the behaviour of complex qualitative system dynamics using FCMs.