Reviewerly/exHarmony huggingface dataset:
Activation functions: comparison of trends in practice and research for deep learning

Deep neural networks (DNN) have been successfully used in diverse emerging domains to solve real world complex problems with may more deep learning (DL) architectures, being developed to date. To achieve this state-of-the-art (SOTA) performances, the DL architectures use activation functions (AFs), to perform diverse computations between the hidden layers and the output layers of any given DL architecture. This paper presents a survey on the existing AFs used in deep learning applications and highlights the recent trends in the use of the AFs for DL applications. The novelty of this paper is that it compiles the majority of the AFs used in DL and outlines the current trends in the applications and usage of these functions in practical deep learning deployments against the SOTA research results. This compilation will aid in making effective decisions in the choice of the most suitable and appropriate AF for a given application, ready for deployment. This paper is timely because majority of the research papers on AF highlights similar works and results while this paper will be the first, to compile the trends in AF applications in practice against the research results from the literature, found in DL research to date.
Impact of Race and Gender on the SBA Paycheck Protection Program (PPP) Loan Amounts

Download This Paper Open PDF in Browser Add Paper to My Library Share: Permalink Using these links will ensure access to this page indefinitely Copy URL Copy DOI
Bayesian Deep Learning via Subnetwork Inference

The Bayesian paradigm has the potential to solve core issues of deep neural networks such as poor calibration and data inefficiency. Alas, scaling Bayesian inference to large weight spaces often requires restrictive approximations. In this work, we show that it suffices to perform inference over a small subset of model weights in order to obtain accurate predictive posteriors. The other weights are kept as point estimates. This subnetwork inference framework enables us to use expressive, otherwise intractable, posterior approximations over such subsets. In particular, we implement subnetwork linearized Laplace: We first obtain a MAP estimate of all weights and then infer a full-covariance Gaussian posterior over a subnetwork. We propose a subnetwork selection strategy that aims to maximally preserve the model's predictive uncertainty. Empirically, our approach is effective compared to ensembles and less expressive posterior approximations over full networks.
mmapx

Modern Systems-on-Chip (SoCs) are networks of heterogeneous cores, intelligent devices, and memory, connected through multiple configurable address translation and protection units like IOMMUs and System MMUs.
GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training

Normalization plays an important role in the optimization of deep neural networks. While there are standard normalization methods in computer vision and natural language processing, there is limited understanding of how to effectively normalize neural networks for graph representation learning. In this paper, we propose a principled normalization method, Graph Normalization (GraphNorm), where the key idea is to normalize the feature values across all nodes for each individual graph with a learnable shift. Theoretically, we show that GraphNorm serves as a preconditioner that smooths the distribution of the graph aggregation's spectrum, leading to faster optimization. Such an improvement cannot be well obtained if we use currently popular normalization methods, such as BatchNorm, which normalizes the nodes in a batch rather than in individual graphs, due to heavy batch noises. Moreover, we show that for some highly regular graphs, the mean of the feature values contains graph structural information, and directly subtracting the mean may lead to an expressiveness degradation. The learnable shift in GraphNorm enables the model to learn to avoid such degradation for those cases. Empirically, Graph neural networks (GNNs) with GraphNorm converge much faster compared to GNNs with other normalization methods, e.g., BatchNorm. GraphNorm also improves generalization of GNNs, achieving better performance on graph classification benchmarks.
A Systematic Mapping Study on Analysis of Code Repositories

Code repositories contain valuable information, which can be extracted, processed and synthesized into valuable information.It enabled developers to improve maintenance, increase code quality and understand software evolution, among other insights.Certain research has been made during the last years in this field.This paper presents a systematic mapping study to find, evaluate and investigate the mechanisms, methods and techniques used for the analysis of information from code repositories that allow the understanding of the evolution of software.Through this mapping study, we have identified the main information used as input for the analysis of code repositories (commit data and source code), as well as the most common methods and techniques of analysis (empirical/experimental and automatic).We believe the conducted research is useful for developers working on software development projects and seeking to improve maintenance and understand the evolution of software through the use and analysis of code repositories.
Web based Augmented Reality for Information Delivery Services A Performance Study

Augmented Reality (AR) is the overlapping of the real world and the virtual world. AR is an evolving technology, and its implementation opens up a new direction to multiple types of information access services rendered to the users. The digital library section of the central library IIT Kharagpur has a rich collection of several items like CD/DVD ROMs, Digital Scanners, and VR Devices. Library developed in-house WebAR programs using “ar.js” that support “three.js” and “A-Frame” for Augmented Reality rendering on the web. This study tried to evaluate web-based AR programs’ performance on mobile devices with low-end hardware configuration supporting WebGL and WebRTC. We conducted quantitative research to find the performance of the web-based AR applications using the four independent variables, namely frames per second (fps), request animation frame (raf), load time (lt), entity object (eo). The paper is also helpful to the academic librarians who think about implementing augmented reality library services with no cost involved. The AR-based information service is beneficial to library users with new COVID norms1, as the user does not need to touch anything to get the information. Instead, it gets into his/ her mobile device.
Deep Learning in Video Violence Detection

In recent years, in order to continuously promote the construction of a safe city, intelligent monitoring equipment has been widely used throughout the country. In the field of intelligent image analysis, behavioral intelligence analysis has received more and more attention. At present, behavior analysis methods focus on detecting simple actions, such as walking, jogging, or jumping; while there is less research on fighting or violent behaviors with complex actions. However, the effective intelligent analysis of violence in video surveillance is essential to maintaining social stability and protecting people's lives and property. Therefore, we summarize the latest methods of violence detection in existing video sequences.
Deductive Learning for Weakly-Supervised 3D Human Pose Estimation via Uncalibrated Cameras

Without prohibitive and laborious 3D annotations, weakly-supervised 3D human pose methods mainly employ the model regularization with geometric projection consistency or geometry estimation from multi-view images. Nevertheless, those approaches explicitly need known parameters of calibrated cameras, exhibiting a limited model generalization in various realistic scenarios. To mitigate this issue, in this paper, we propose a Deductive Weakly-Supervised Learning (DWSL) for 3D human pose machine. Our DWSL firstly learns latent representations on depth and camera pose for 3D pose reconstruction. Since weak supervision usually causes ill-conditioned learning or inferior estimation, our DWSL introduces deductive reasoning to make an inference for the human pose from a view to another and develops a reconstruction loss to demonstrate what the model learns and infers is reliable. This learning by deduction strategy employs the view-transform demonstration and structural rules derived from depth, geometry and angle constraints, which improves the reliability of the model training with weak supervision. On three 3D human pose benchmarks, we conduct extensive experiments to evaluate our proposed method, which achieves superior performance in comparison with state-of-the-art weak-supervised methods. Particularly, our model shows an appealing potential for learning from 2D data captured in dynamic outdoor scenes, which demonstrates promising robustness and generalization in realistic scenarios. Our code is publicly available at https://github.com/Xipeng-Chen/DWSL-3D-pose.
BIDIRECTIONAL MULTI-SCALE ATTENTION NETWORKS FOR SEMANTIC SEGMENTATION OF OBLIQUE UAV IMAGERY

Abstract. Semantic segmentation for aerial platforms has been one of the fundamental scene understanding task for the earth observation. Most of the semantic segmentation research focused on scenes captured in nadir view, in which objects have relatively smaller scale variation compared with scenes captured in oblique view. The huge scale variation of objects in oblique images limits the performance of deep neural networks (DNN) that process images in a single scale fashion. In order to tackle the scale variation issue, in this paper, we propose the novel bidirectional multi-scale attention networks, which fuse features from multiple scales bidirectionally for more adaptive and effective feature extraction. The experiments are conducted on the UAVid2020 dataset and have shown the effectiveness of our method. Our model achieved the state-of-the-art (SOTA) result with a mean intersection over union (mIoU) score of 70.80%.
CRMI: Confidence-Rich Mutual Information for Information-Theoretic Mapping

This letter focuses on information-theoretic active mapping and exploration with beam-based range sensing robots. Traditional works based on hand-engineered inverse sensor model (ISM) mapping or kernel inference methods lead to imbalanced accuracy and efficiency. This motivates us to propose a new approach to compute mutual information more accurately, based on the continuous belief distribution over the occupancy map and called confidence-rich mutual information (CRMI). Specifically, we explicitly model the measurement dependencies between grid cells within the same measurement cone at each time step and derive the CRMI for each cell on all beams by introducing a more general beam-based sensor cause model (SCM), rather than the customized ISM. The time efficiency for CRMI mapping allows for online implementation as well. Extensive simulations and experiments show the desired exploratory behavior to unexplored and obscured regions for CRMI-based robot controllers in the unstructured and cluttered scene, even in large scale environment.
Blockchain technology, macroeconomic uncertainty and investment efficiency

Purpose In addition to leading a new tide of global financial technology, blockchain delivers advantages in terms of risk control compared to traditional financial systems. By exploring the relationship between blockchain technology and macroeconomic uncertainty, this study aims to identify the hedge risk attribute of blockchain technology. Design/methodology/approach From a data set comprising 6,323 Chinese firms with A-shares listed on the Shenzhen and Shanghai Stock Exchanges in 2015–2018, the authors obtain the use of blockchain technology by listed companies on the basis of annual reports, news reports, search engines and prospectuses. These documents are then subjected to text analyses based on computer technology. Cross-sectional and propensity score matching analyses are used to ensure robustness. Findings The empirical results show that with an increase in macroeconomic uncertainty, blockchain technology can potentially enable companies to reduce their systemic risks and enhance their investment efficiency. Originality/value This study expands the literature on the application of blockchain technology, offers references for enterprises to address future risks based on specific macroeconomically uncertain environments and provides guidelines for governments to maintain financial market stability.
Psychological Difficulties during the Covid Lockdown: Video in Blended Digital Teaching Language, Literature, and Culture

The recent COVID-19 pandemic has caused an urgent necessity for higher education institutions and their teaching staff to move the educational process online in the shortest possible time. It should be admitted that higher education worldwide is changing, and institutions face challenges when adapting to the new COVID-19 reality – blended digital teaching and learning. They are tackling several issues connected with the essence of virtual education, its technical aspect, and students’ learning environment. During COVID-19 lockdown, teachers have to prepare and deliver their classes from home, simultaneously coping with numerous professional and technical challenges, often without any appropriate support. In addition to that, they were lack of technology literacy, professional knowledge, and experience needed for successful E-teaching. Besides, a new mode of blended digital teaching and learning also made them consider technical and administrative aspects of the current educational process, namely to use new educational platforms and tools, organize active workflows, and work out critical educational principles to design and facilitate practical online experiences. Along with the challenges that the teachers face in such conditions, they should be able to cope with the stress and psychological disorders of pupils and students. The article emphasizes the abnormal psychology of youngsters (pupils and students) as the result of stress and anxiety that appeared during Covid lockdown – the application of psychological science to understanding and treating mental disorders – and the use of video as a productive means of avoiding psychological disabilities. The research objective is to prove that video is an effective educational tool for avoiding psychological difficulties in blended digital learning during COVID-19 lockdown while teaching foreign languages, literature, and culture. Special attention is paid to psychological strategies of overcoming stress during COVID-19 lockdown, as well as to effective ways to students’ adapt to a virtual learning environment, development and implementation of anti-stress methods, which heighten their motivation. The article also heads on and explores potential solutions to educational problems that one can encounter in the new educational process at the post-COVID-19 epidemic period, such as the quality of education offered, its cost and availability, internationalization, and employability.
Artificial Fish Swarm Algorithm for Mining High Utility Itemsets

The discovery of high utility itemsets (HUIs) is an attractive topic in data mining. Because of its high computational cost, using heuristic methods is a promising approach to rapidly discovering sufficient HUIs. The artificial fish swarm algorithm is a heuristic method with many applications. Except the current position, artificial fish do not record additional previous information, as other related methods do. This is consistent with the HUI mining problem: that the results are not always distributed around a few extreme points. Thus, we study HUI mining from the perspective of the artificial fish swarm algorithm, and propose an HUI mining algorithm called HUIM-AF. We model the HUI mining problem with three behaviors of artificial fish: follow, swarm, and prey. We explain the HUIM-AF algorithm and compare it with two related algorithms on four publicly available datasets. The experimental results show that HUIM-AF can discover more HUIs than the existing algorithms, with comparable efficiency.
Towards Improved and Interpretable Action Quality Assessment with Self-Supervised Alignment

Action Quality Assessment (AQA) is a video understanding task aiming at the quantification of the execution quality of an action. One of the main challenges in relevant, deep learning-based approaches is the collection of training data annotated by experts. Current methods perform fine-tuning on pre-trained backbone models and aim to improve performance by modeling the subjects and the scene. In this work, we consider embeddings extracted using a self-supervised training method based on a differential cycle consistency loss between sequences of actions. These are shown to improve the state-of-the-art without the need for additional annotations or scene modeling. The same embeddings are also used to temporally align the sequences prior to quality assessment which further increases the accuracy, provides robustness to variance in execution speed and enables us to provide fine-grained interpretability of the assessment score. The experimental evaluation of the method on the MTL-AQA dataset demonstrates significant accuracy gain compared to the state-of-the-art baselines, which grows even more when the action execution sequences are not well aligned.
A Hierarchical Approach to Interpretability of TS Rule-Based Models

Interpretability of fuzzy rule-based models has always been of significant interest to the research community and the research in this area led to a number of far-reaching results. In this study, we briefly revisit the methodology and concepts of interpretability of Takagi–Sugeno (T-S) rule-based models and develop a conceptual framework involving several levels at which rules are interpreted. The layers at which interpretability is positioned are structured hierarchically by starting with the initial fuzzy set level (originating from the design of the rules), moving to information granules of finite support (where interval calculus is engaged) and finally ending up with symbols built at the higher level. As T-S rule-based models are endowed with local functions forming the conclusion parts of the rules, with the use of the principle of justifiable granularity, we develop a way of forming an interpretable conclusion in the form of information granule. To facilitate interpretability of conditions of the rules, multidimensional fuzzy sets (coming as a result of clustering) are decomposed into a Cartesian product of 1-D fuzzy sets and the quality of the resulting decomposition is evaluated. The quality of granular rules is assessed by analyzing the relationship between specificity of condition and conclusion information granules. The rules emerging at the level of symbols are further interpreted by engaging linguistic approximation, which helps approximate a collection of linguistic terms of subconditions producing a linguistic summarization in the form <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">τ</i> (inputs are <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</i> ) consisting of a certain linguistic quantifier <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">τ</i> . The performance of summarization is provided in the form of ranking of the relevance of the rules. Experimental studies using publicly available data are completed and analyzed.
Fast Flow Reconstruction via Robust Invertible n × n Convolution

Flow-based generative models have recently become one of the most efficient approaches to model data generation. Indeed, they are constructed with a sequence of invertible and tractable transformations. Glow first introduced a simple type of generative flow using an invertible 1×1 convolution. However, the 1×1 convolution suffers from limited flexibility compared to the standard convolutions. In this paper, we propose a novel invertible n×n convolution approach that overcomes the limitations of the invertible 1×1 convolution. In addition, our proposed network is not only tractable and invertible but also uses fewer parameters than standard convolutions. The experiments on CIFAR-10, ImageNet and Celeb-HQ datasets, have shown that our invertible n×n convolution helps to improve the performance of generative models significantly.
FaVIQ: FAct Verification from Information-seeking Questions

Despite significant interest in developing general purpose fact checking models, it is challenging to construct a large-scale fact verification dataset with realistic real-world claims.Existing claims are either authored by crowdworkers, thereby introducing subtle biases that are difficult to control for, or manually verified by professional fact checkers, causing them to be expensive and limited in scale.In this paper, we construct a large-scale challenging fact verification dataset called FAVIQ, consisting of 188k claims derived from an existing corpus of ambiguous information-seeking questions.The ambiguities in the questions enable automatically constructing true and false claims that reflect user confusions (e.g., the year of the movie being filmed vs. being released).Claims in FAVIQ are verified to be natural, contain little lexical bias, and require a complete understanding of the evidence for verification.Our experiments show that the stateof-the-art models are far from solving our new task.Moreover, training on our data helps in professional fact-checking, outperforming models trained on the widely used dataset FEVER or in-domain data by up to 17% absolute.Altogether, our data will serve as a challenging benchmark for natural language understanding and support future progress in professional fact checking.
Model‐agnostic online forecasting for PV power output

A reliable forecasting model is required for photovoltaic (PV) power output because solar energy is highly volatile. Another driver for the need of a reliable forecasting model is concept drift, which means that the statistical properties of the data change over time. In this paper, an online forecasting method to handle concept drift is proposed. First, the problem of forecasting in batch learning is transformed into a forecasting in online learning setting. Then, an online learning algorithm is applied, which is good for handling concept drift. Through experiments using the real-world data, it is shown that the method noticeably improves performance compared to the case where a trained model is used. Under various concept drift scenarios, the method improves performance by up to 87.3%. It is also shown that the re-training method (a representative existing method) has several limitations. This method requires several issues to be solved, such as selection of a proper window size, and this is evident through results showing different performance under different settings. In contrast, the method shows a reliable and desirable performance under various concept drift scenarios and thus outperforms the re-training method. The method improves performance by up to 79%.