Reviewerly/exHarmony huggingface dataset:
Carbon Emissions and Large Neural Network Training.

The computation demand for machine learning (ML) has grown rapidly recently, which comes with a number of costs. Estimating the energy cost helps measure its environmental impact and finding greener strategies, yet it is challenging without detailed information. We calculate the energy use and carbon footprint of several recent large models-T5, Meena, GShard, Switch Transformer, and GPT-3-and refine earlier estimates for the neural architecture search that found Evolved Transformer. We highlight the following opportunities to improve energy efficiency and CO2 equivalent emissions (CO2e): Large but sparsely activated DNNs can consume <1/10th the energy of large, dense DNNs without sacrificing accuracy despite using as many or even more parameters. Geographic location matters for ML workload scheduling since the fraction of carbon-free energy and resulting CO2e vary ~5X-10X, even within the same country and the same organization. We are now optimizing where and when large models are trained. Specific datacenter infrastructure matters, as Cloud datacenters can be ~1.4-2X more energy efficient than typical datacenters, and the ML-oriented accelerators inside them can be ~2-5X more effective than off-the-shelf systems. Remarkably, the choice of DNN, datacenter, and processor can reduce the carbon footprint up to ~100-1000X. These large factors also make retroactive estimates of energy cost difficult. To avoid miscalculations, we believe ML papers requiring large computational resources should make energy consumption and CO2e explicit when practical. We are working to be more transparent about energy use and CO2e in our future research. To help reduce the carbon footprint of ML, we believe energy usage and CO2e should be a key metric in evaluating models, and we are collaborating with MLPerf developers to include energy usage during training and inference in this industry standard benchmark.
The Syntax of Argument Structure

Bridging theoretical modelling and advanced empirical techniques is a central aim of current linguistic research. The progress in empirical methods contributes to the precise estimation of the properties of linguistic data and promises new ways for justifying theoretical models and testing their implications. The contributions to the present collective volume take up this challenge and focus on the relevance of empirical results achieved through up-to-date methodology for the theoretical analysis and modelling of argument structure. They tackle issues of argument structure from different perspectives addressing questions related to diverse verb types (unaccusatives, unergatives, (di)transitives, psych verbs), morpho-syntactic operations (prefixation, simple vs. particle verbs), case distinctions (dative vs. accusative, case vs. prepositions), argument and voice alternations (dative vs. benefactive alternation, active vs. passive), word order alternations and the impact of animacy, agentivity, and eventivity on argument structure. The volume will be of interest to theoretical linguists, psycholinguists, and corpus linguists interested in the syntax of argument structure and its modelling using precise empirical methods.
Machine learning methods for sign language recognition: A critical review and analysis

Sign language is an essential tool to bridge the communication gap between normal and hearing-impaired people. However, the diversity of over 7000 present-day sign languages with variability in motion position, hand shape, and position of body parts making automatic sign language recognition (ASLR) a complex system. In order to overcome such complexity, researchers are investigating better ways of developing ASLR systems to seek intelligent solutions and have demonstrated remarkable success. This paper aims to analyse the research published on intelligent systems in sign language recognition over the past two decades. A total of 649 publications related to decision support and intelligent systems on sign language recognition (SLR) are extracted from the Scopus database and analysed. The extracted publications are analysed using bibliometric VOSViewer software to (1) obtain the publications temporal and regional distributions, (2) create the cooperation networks between affiliations and authors and identify productive institutions in this context. Moreover, reviews of techniques for vision-based sign language recognition are presented. Various features extraction and classification techniques used in SLR to achieve good results are discussed. The literature review presented in this paper shows the importance of incorporating intelligent solutions into the sign language recognition systems and reveals that perfect intelligent systems for sign language recognition are still an open problem. Overall, it is expected that this study will facilitate knowledge accumulation and creation of intelligent-based SLR and provide readers, researchers, and practitioners a roadmap to guide future direction.
The Devil Is in the Details: Window-based Attention for Image Compression

Learned image compression methods have exhibited superior rate-distortion performance than classical image compression standards. Most existing learned image compression models are based on Convolutional Neural Networks (CNNs). Despite great contributions, a main drawback of CNN based model is that its structure is not designed for capturing local redundancy, especially the nonrepetitive textures, which severely affects the reconstruction quality. Therefore, how to make full use of both global structure and local texture becomes the core problem for learning-based image compression. Inspired by recent progresses of Vision Transformer (ViT) and Swin Transformer, we found that combining the local-aware attention mechanism with the global-related feature learning could meet the expectation in image compression. In this paper, we first extensively study the effects of multiple kinds of attention mechanisms for local features learning, then introduce a more straightforward yet effective window-based local attention block. The proposed window-based attention is very flexible which could work as a plug-and-play component to enhance CNN and Transformer models. Moreover, we propose a novel Symmetrical TransFormer (STF) framework with absolute transformer blocks in the down-sampling encoder and up-sampling decoder. Extensive experimental evaluations have shown that the proposed method is effective and outperforms the state-of-the-art methods. The code is publicly available at https://github.com/Googolxx/STF.
Detecting overreaction in the Bitcoin market: A quantile autoregression approach

We examine the persistence of returns on Bitcoin at different parts on the return distributions through the use of the quantile autoregressive (QAR) models. We find lower quantiles of the daily return distribution and upper quantiles of the weekly return distribution to exhibit positive dependence with past returns. The evidence points to overreaction in the Bitcoin market: investors overreact during days of sharp declines in the Bitcoin price and during weeks of market rallies.
ALIGN: Analyzing linguistic interactions with generalizable techNiques—A Python library.

Linguistic alignment (LA) is the tendency during a conversation to reuse each other's linguistic expressions, including lexical, conceptual, or syntactic structures. LA is often argued to be a crucial driver in reciprocal understanding and interpersonal rapport, though its precise dynamics and effects are still controversial. One barrier to more systematic investigation of these effects lies in the diversity in the methods employed to analyze LA, which makes it difficult to integrate and compare results of individual studies. To overcome this issue, we have developed ALIGN (Analyzing Linguistic Interactions with Generalizable techNiques), an open-source Python package to measure LA in conversation (https://pypi.python.org/pypi/align) along with in-depth open-source tutorials hosted on ALIGN's GitHub repository (https://github.com/nickduran/align-linguistic-alignment). Here, we first describe the challenges in the study of LA and outline how ALIGN can address them. We then demonstrate how our analytical protocol can be applied to theory-driven questions using a complex corpus of dialogue (the Devil's Advocate corpus; Duran & Fusaroli, 2017). We close by identifying further challenges and point to future developments of the field. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
Language-Capable Robots may Inadvertently Weaken Human Moral Norms

Previous research in moral psychology and human-robot interaction has shown that technology shapes human morality, and research in human-robot interaction has shown that humans naturally perceive robots as moral agents. Accordingly, we propose that language-capable autonomous robots are uniquely positioned among technologies to significantly impact human morality. We therefore argue that it is imperative that language-capable robots behave according to human moral norms and communicate in such a way that their intention to adhere to those norms is clear. Unfortunately, the design of current natural language oriented robot architectures enables certain architectural components to circumvent or preempt those architectures' moral reasoning capabilities. In this paper, we show how this may occur, using clarification request generation in current dialog systems as a motivating example. Furthermore, we present experimental evidence that the types of behavior exhibited by current approaches to clarification request generation can cause robots to (1) miscommunicate their moral intentions and (2) weaken humans' perceptions of moral norms within the current context. This work strengthens previous preliminary findings, and does so within an experimental paradigm that provides increased external and ecological validity over earlier approaches.
A game-theoretic method based on Q-learning to invalidate criminal smart contracts

Criminal smart contracts, severely threatening the security of cyberspace, allow criminals to maximize their utilities through illegal behaviors. The validity of criminal smart contracts is an indicator of criminals' success. While the validity of criminal smart contracts heavily hinges on parameters derived from data feed. Therefore, criminals have incentives to increase these contracts' validity by biasing the parameters therein. In this paper, we formalize data feed parameters by utilizing stochastic distributions, allowing us to analyze criminal smart contracts as state-based games and evaluate their validity through state-arrival probabilities. The main target of this paper is to decrease the validity of criminal smart contracts to prevent criminals' illegal behaviors. To this end, Q-learning is utilized to train distribution parameters so that criminals have a low probability of reaching their desirable state. This impairs the validity of criminal smart contracts and minimizes criminals' utilities. To the best of our knowledge, it's the first implementation of machine learning in the analysis of smart contracts. The experiments show that our method is at least an order of magnitude lower than previous works under the same settings with respect to the validity of criminal smart contracts.
A Feature Selection Method Based on Hybrid Improved Binary Quantum Particle Swarm Optimization

As the volume of data available for analysis grows, feature selection is becoming a vital part of ensuring accurate classification results.In classification problems, selecting a small number of features reduces computational complexity, but selecting the right features is important to maintain a high level of accuracy.In this paper, we present a feature selection method based on hybrid improved quantum-behavior particle swarm optimization, called HI-BQPSO.The HI-BQPSO combines a filtering method with an improved quantum-behavior particle swarm optimization algorithm to greatly reduce the dimensionality of the data so as to overcome some of the shortcomings of BQPSO.Tests were conducted on nine gene expression datasets and 36 UCI datasets to evaluate and compare the classification accuracy of the HI-BQPSO's selected feature subsets against four other algorithms.The results, using a variety of different classifiers, show that the HI-BQPSO significantly reduces the number of features required for classification while maintaining higher levels of accuracy in many cases.
Aspect-based sentiment analysis via fusing multiple sources of textual knowledge

The aim of aspect-based sentiment analysis (ABSA) is to predict sentiment polarity of text toward a specific aspect. Although existing neural network models show promising performances on ABSA, their capabilities can be unsatisfactory in cases where the amount of training data is limited. In this paper, we propose a unified model which exploits and incorporates multiple sources of knowledge to improve its ability on ABSA. Structure knowledge is extracted via clause recognition and fused in the model through the generation of multiple context representations to force the model to capture aspect-specific context information. Sentiment knowledge is exploited by means of training a general classification model with the sentiment labels of documents and fused through pretraining specific layers to extract contextual features and predict sentiment polarities more accurately. In addition, information of conjunctions is fused in our model to capture the relations between clauses and provide additional sentiment features. Experimental results on five publicly available ABSA datasets validate the effectiveness of our method and prove that multiple sources of knowledge can collaboratively enhance our model.
ASSESSING THE RELATIONSHIP BETWEEN INFORMATION TRANSPARENCY THROUGH SOCIAL MEDIA DISCLOSURE AND FIRM VALUE

Based on the signaling theory, managers disclose the firm's high performance to maintain their positions and receive rewards. On the other hand, users of financial information prefer the transparency of information rather than the quantity of disclosed information. Online financial disclosure as an output of advanced technology provides useful, timely and verifiable information for decision making. Nevertheless, the level of IFR by the Gulf Cooperation Council companies varies due to the lack of appropriate regulations. Therefore, this study investigates the association between online financial disclosure and performance in the Gulf Cooperation Council Countries.&#x0D; Extensive literature review was carried out and a checklist of 90 items (71 for content and 19 for presentation) was developed to measure the level of online financial disclosure for the companies that are listed in Gulf Cooperation Council bourses.&#x0D; The findings show that the overall online financial disclosure in Gulf Cooperation Council is 77% Nevertheless; the results indicate a positive association between OFD and performance. Accordingly, the study recommends that regulatory bodies should develop a guideline of disclosing information through the internet in order to enhance the corporate transparency and performance among the Gulf Cooperation Council listed companies leading to reasonable economic decision making.&#x0D; &#x0D; Keywords: Online Financial Disclosure; Performance; Voluntary Disclosure; GCC Countries.
Weight Agnostic Neural Networks

Not all neural network architectures are created equal, some perform much better than others for certain tasks. But how important are the weight parameters of a neural network compared to its architecture? In this work, we question to what extent neural network architectures alone, without learning any weight parameters, can encode solutions for a given task. We propose a search method for neural network architectures that can already perform a task without any explicit weight training. To evaluate these networks, we populate the connections with a single shared weight parameter sampled from a uniform random distribution, and measure the expected performance. We demonstrate that our method can find minimal neural network architectures that can perform several reinforcement learning tasks without weight training. On a supervised learning domain, we find network architectures that achieve much higher than chance accuracy on MNIST using random weights. Interactive version of this paper at https://weightagnostic.github.io/
Counting Cattle in UAV Images—Dealing with Clustered Animals and Animal/Background Contrast Changes

The management of livestock in extensive production systems may be challenging, especially in large areas. Using Unmanned Aerial Vehicles (UAVs) to collect images from the area of interest is quickly becoming a viable alternative, but suitable algorithms for extraction of relevant information from the images are still rare. This article proposes a method for counting cattle which combines a deep learning model for rough animal location, color space manipulation to increase contrast between animals and background, mathematical morphology to isolate the animals and infer the number of individuals in clustered groups, and image matching to take into account image overlap. Using Nelore and Canchim breeds as a case study, the proposed approach yields accuracies over 90% under a wide variety of conditions and backgrounds.
Multi-kernel fuzzy clustering based on auto-encoder for fMRI functional network

The existing clustering algorithms based on auto-encoder only use one layer of information. In this paper, we propose a new subspace clustering algorithm that uses multiple hidden layers of information from the stacked auto-encoder to construct different kernels. The proposed fuzzy multi-kernel clustering method based on auto-encoder is realized by updating the membership matrix and coefficients of these kernels until the value of objective function is iterated to the minimum error. Also, the proposed method combining auto-encoder achieves high dimension reduction of input data. In order to test the effectiveness of the algorithm, we first performed experiments on the published brain network dataset. Compared with MKFC, RMKKM and other algorithms, the proposed method can significantly improve accuracy. Specifically, the brain network has large dimensions when the functional magnetic resonance imaging (fMRI) data is preprocessed. Therefore, the method in this paper is applied to the Autism Spectrum Disorder (ASD) of the Autism Brain Imaging Data Exchange (ABIDE) database. The experiment results on the network dataset we constructed with high dimension are better than the present several clustering algorithms. The results show that the subspace information after dimensionality reduction is more conducive to clustering.
Towards federated unsupervised representation learning

Making deep learning models efficient at inferring nowadays requires training with an extensive number of labeled data that are gathered in a centralized system. However, gathering labeled data is an expensive and time-consuming process, centralized systems cannot aggregate an ever-increasing amount of data and aggregating user data is raising privacy concerns. Federated learning solves data volume and privacy issues by leaving user data on devices, but is limited to use cases where labeled data can be generated from user interaction. Unsupervised representation learning reduces the amount of labeled data required for model training, but previous work is limited to centralized systems. This work introduces federated unsupervised representation learning, a novel software architecture that uses unsupervised representation learning to pre-train deep neural networks using unlabeled data in a federated setting. Pre-trained networks can be used to extract discriminative features. The features help learn a down-stream task of interest with a reduced amount of labeled data. Based on representation performance experiments with human activity detection it is recommended to pre-train with unlabeled data originating from more users performing a bigger set of activities compared to data used with the down-stream task of interest. As a result, competitive or superior performance compared to supervised deep learning is achieved.
A novel interdependence based multilevel thresholding technique using adaptive equilibrium optimizer

The multilevel thresholding is one important operation in computer vision, which is a subfield of artificial intelligence (AI), used to understand and interpret the data in the real world. The existing entropic methods, based on the histogram of an image for the multilevel thresholding mostly deal with the maximization of the entropic information excluding the shred boundary, which reduces the accuracy. These problems lead to the poor thresholding accuracy and a lower speed. To address the problem, we propose a novel interdependence based technique that uses the shred boundary, which is a minimization problem. A firsthand objective function is investigated, which takes care of the shred boundary. The traditional multilevel thresholding techniques are computationally expensive due to the exhaustive search process, an alternate method is to use the evolutionary computation based on a nature-inspired algorithm. In this paper, a new optimizer called adaptive equilibrium optimizer (AEO) is also proposed for multilevel thresholding, an improvement over the basic equilibrium optimizer (EO) by implementing an adaptive decision making of dispersal for nonperformer search agents. The AEO performance is compared with state-of-the-art algorithms — equilibrium optimizer (EO), gray wolf optimizer (GWO), whale optimization algorithm (WOA), squirrel search algorithm (SSA) and the wind driven optimization (WDO) algorithm, using standard benchmark functions. Based on the qualitative and quantitative analysis, the AEO outperformed EO, GWO, WOA, SSA, and WDO. The optimal thresholds are obtained by minimizing the objective function using the AEO. For the experiment, 500 images of the BSDS 500 dataset are considered. Popular metrics such as the peak signal to noise ratio (PSNR), structural similarity index (SSIM), and the feature similarity index (FSIM) are considered for a quantitative analysis. Remarkable differences in the thresholding accuracy are observed with a simultaneously decreasing computational complexity. The merits of the paper are highlighted to ensure its future use in the world of engineering applications using soft computing, a subfield of the AI.
Mitigating Curtailment and Carbon Emissions through Load Migration between Data Centers

As the share of variable renewable energy (VRE) grows in the electric grid, so does the risk of curtailment. While energy storage and hydrogen production have been proposed as solutions to the curtailment problem, they often pose technological and economic challenges. Here, we analyze the potential of data center load migration for mitigating curtailment and greenhouse gas (GHG) emissions. Using historical hourly electricity generation, curtailment, and typical data center server utilization data, we simulate the effect of migrating data center workloads from the fossil-fuel-heavy PJM to the renewable-heavy CAISO. The results show that load migration within the existing data center capacity during the curtailment hours in CAISO has the potential to reduce 113–239 KtCO2e per year of GHG emissions and absorb up to 62% of the total curtailment with negative abatement costs in 2019. Our study demonstrates the overlooked role that data centers can play for VRE integration and GHG emissions mitigation.