Reviewerly/exHarmony huggingface dataset:
Deep Neural Network Architecture: Application for Facial Expression Recognition

There are great challenges to build a model or architecture in Deep Learning and integrate it into a real-time application. One of these challenges is the construction or acquisition of large quality datasets (thousands or millions of objects). Another one is to have great computing potential for the architecture learning process. Finally, efficient architectures are needed for the design of deep neural networks, which requires expertise, human experience and practical work. This work presents a deep neural network architecture to classify two feelings of facial expression (happy and sad). A set of data is also created that present great changes in: image environments, facial expression, pose, age, ethnicity and others. The evidence presented shows a competitive architecture and indicates an accuracy greater than 90% with noisy data. Finally, the implementation of a real-time application for facial expression recognition is shown.
Scene image representation by foreground, background and hybrid features

Previous methods for representing scene images based on deep learning primarily consider either the foreground or background information as the discriminating clues for the classification task. However, scene images also require additional information (hybrid) to cope with the inter-class similarity and intra-class variation problems. In this paper, we propose to use hybrid features in addition to foreground and background features to represent scene images. We suppose that these three types of information could jointly help to represent scene image more accurately. To this end, we adopt three VGG-16 architectures pre-trained on ImageNet, Places, and Hybrid (both ImageNet and Places) datasets for the corresponding extraction of foreground, background and hybrid information. All these three types of deep features are further aggregated to achieve our final features for the representation of scene images. Extensive experiments on two large benchmark scene datasets (MIT-67 and SUN-397) show that our method produces the state-of-the-art classification performance.
Approximation Capabilities of Neural ODEs and Invertible Residual Networks

Neural ODEs and i-ResNet are recently proposed methods for enforcing invertibility of residual neural models. Having a generic technique for constructing invertible models can open new avenues for advances in learning systems, but so far the question of whether Neural ODEs and i-ResNets can model any continuous invertible function remained unresolved. Here, we show that both of these models are limited in their approximation capabilities. We then prove that any homeomorphism on a $p$-dimensional Euclidean space can be approximated by a Neural ODE operating on a $2p$-dimensional Euclidean space, and a similar result for i-ResNets. We conclude by showing that capping a Neural ODE or an i-ResNet with a single linear layer is sufficient to turn the model into a universal approximator for non-invertible continuous functions.
Pedestrian Detection Algorithm for Intelligent Vehicles in Complex Scenarios

Pedestrian detection is an important aspect of the development of intelligent vehicles. To address problems in which traditional pedestrian detection is susceptible to environmental factors and are unable to meet the requirements of accuracy in real time, this study proposes a pedestrian detection algorithm for intelligent vehicles in complex scenarios. YOLOv3 is one of the deep learning-based object detection algorithms with good performance at present. In this article, the basic principle of YOLOv3 is elaborated and analyzed firstly to determine its limitations in pedestrian detection. Then, on the basis of the original YOLOv3 network model, many improvements are made, including modifying grid cell size, adopting improved k-means clustering algorithm, improving multi-scale bounding box prediction based on receptive field, and using Soft-NMS algorithm. Finally, based on INRIA person and PASCAL VOC 2012 datasets, pedestrian detection experiments are conducted to test the performance of the algorithm in various complex scenarios. The experimental results show that the mean Average Precision (mAP) value reaches 90.42%, and the average processing time of each frame is 9.6 ms. Compared with other detection algorithms, the proposed algorithm exhibits accuracy and real-time performance together, good robustness and anti-interference ability in complex scenarios, strong generalization ability, high network stability, and detection accuracy and detection speed have been markedly improved. Such improvements are significant in protecting the road safety of pedestrians and reducing traffic accidents, and are conducive to ensuring the steady development of the technological level of intelligent vehicle driving assistance.
Identifying and Correcting Label Bias in Machine Learning

The present disclosure is directed to systems and methods for identifying and correcting label bias in machine learning via intelligent re-weighting of training examples. In particular, aspects of the present disclosure leverage a problem formulation which assumes the existence of underlying, unknown, and unbiased labels which are overwritten by an agent who intends to provide accurate labels but may have biases towards certain groups. Despite the fact that a biased training dataset provides only observations of the biased labels, the systems and methods described herein can nevertheless correct the bias by re-weighting the data points without changing the labels.
Comparative Analysis of Ground-Mounted vs. Rooftop Photovoltaic Systems Optimized for Interrow Distance between Parallel Arrays

The aim of this research is to perform an in-depth performance comparison of ground-mounted and rooftop photovoltaic (PV) systems. The PV modules are tilted to receive maximum solar irradiance. The efficiency of the PV system decreases due to the mutual shading impact of parallel tilted PV modules. The mutual shading decreases with the increasing interrow distance of parallel PV modules, but a distance that is too large causes an increase in land cost in the case of ground-mounted configuration and a decrease in roof surface shading in the case of rooftop configuration, because larger sections of roof are exposed to sun radiation. Therefore, an optimized interrow distance for the two PV configurations is determined with the aim being to minimize the levelized cost of energy (LCoE) and maximize the energy yield. The model of the building is simulated in EnergyPlus software to determine the cooling load requirement and roof surface temperatures under different shading scenarios. The layout of the rooftop PV system is designed in Helioscope software. A detailed comparison of the two systems is carried out based on energy output, performance ratio, capacity utilization factor (CUF), energy yield, and LCoE. Compared to ground-mounted configuration, the rooftop PV configuration results in a 2.9% increase in CUF, and up to a 23.7% decrease in LCoE. The results of this research show that installing a PV system on a roof has many distinct advantages over ground-mounted PV systems such as the shading of the roof, which leads to the curtailment of the cooling energy requirements of the buildings in hot regions and land cost savings, especially for urban environments.
Orchestrating automation and sharing in DevOps teams: a revelatory case of job satisfaction factors, risk and work conditions

Agile can be seen as an accelerator to develop software, yet it suffers from bottlenecks between development and operations. DevOps can overcome bottlenecks between agile development teams and operations, notably by providing continuous integration, delivery and deployment, thus improving end-to-end processes between development and operations functions. However, literature on DevOps effects is scant. Using job design characteristics theory and Herzberg’s job satisfaction theory, this paper investigates perceptions of job (dis)satisfaction, risks and work conditions among 59 employees working within 12 agile and DevOps teams in the same firm. Findings show that DevOps provides greater job satisfaction than agile alone. However, this case study also unveiled an effect of risk amplification with DevOps, and the greater need to orchestrate automation and sharing, contingent on work conditions. We theorise that orchestration of interactions between automation and sharing and related risks must be managed to improve job satisfaction when agile teams transition to DevOps.
BERTMeSH: deep contextual representation learning for large-scale high-performance MeSH indexing with full text

Abstract Motivation With the rapid increase of biomedical articles, large-scale automatic Medical Subject Headings (MeSH) indexing has become increasingly important. FullMeSH, the only method for large-scale MeSH indexing with full text, suffers from three major drawbacks: FullMeSH (i) uses Learning To Rank, which is time-consuming, (ii) can capture some pre-defined sections only in full text and (iii) ignores the whole MEDLINE database. Results We propose a computationally lighter, full text and deep-learning-based MeSH indexing method, BERTMeSH, which is flexible for section organization in full text. BERTMeSH has two technologies: (i) the state-of-the-art pre-trained deep contextual representation, Bidirectional Encoder Representations from Transformers (BERT), which makes BERTMeSH capture deep semantics of full text. (ii) A transfer learning strategy for using both full text in PubMed Central (PMC) and title and abstract (only and no full text) in MEDLINE, to take advantages of both. In our experiments, BERTMeSH was pre-trained with 3 million MEDLINE citations and trained on ∼1.5 million full texts in PMC. BERTMeSH outperformed various cutting-edge baselines. For example, for 20 K test articles of PMC, BERTMeSH achieved a Micro F-measure of 69.2%, which was 6.3% higher than FullMeSH with the difference being statistically significant. Also prediction of 20 K test articles needed 5 min by BERTMeSH, while it took more than 10 h by FullMeSH, proving the computational efficiency of BERTMeSH. Supplementary information Supplementary data are available at Bioinformatics online
On Completeness-aware Concept-Based Explanations in Deep Neural Networks

Human explanations of high-level decisions are often expressed in terms of key concepts the decisions are based on. In this paper, we study such concept-based explainability for Deep Neural Networks (DNNs). First, we define the notion of completeness, which quantifies how sufficient a particular set of concepts is in explaining a model's prediction behavior based on the assumption that complete concept scores are sufficient statistics of the model prediction. Next, we propose a concept discovery method that aims to infer a complete set of concepts that are additionally encouraged to be interpretable, which addresses the limitations of existing methods on concept explanations. To define an importance score for each discovered concept, we adapt game-theoretic notions to aggregate over sets and propose ConceptSHAP. Via proposed metrics and user studies, on a synthetic dataset with apriori-known concept explanations, as well as on real-world image and language datasets, we validate the effectiveness of our method in finding concepts that are both complete in explaining the decisions and interpretable. (The code is released at this https URL)
Optimization Analysis and Implementation of Online Wisdom Teaching Mode in Cloud Classroom Based on Data Mining and Processing

The rapid development of Internet technology and information technology is rapidly changing the way people think, recognize, live, work and learn. In the context of Internet + education, the emerging learning form of a cloud classroom has emerged. Cloud classroom refers to the process in which learners use the network as a way to obtain learning objectives and learning resources, communicate with teachers and other learners through the net-work, and build their own knowledge structure. Because it breaks the boundaries of time and space, it has the characteristics of freedom, high effi-ciency and extensiveness, and is quickly accepted by learners of different ag-es and occupations. The traditional cloud classroom teaching mode has no personalized recommendation module and cannot solve an information over-load problem. Therefore, this paper proposes a cloud classroom online teach-ing system under the personalized recommendation system. The system adopts a collaborative filtering recommendation algorithm, which helps to mine the potential preferences of users and thus complete more accurate recommendations. It not only highlights the core position of personalized curriculum recommendation in the field of online education, but also makes the cloud classroom online teaching mode more intelligent and meets the needs of intelligent teaching.
Sentence-Level Classification Using Parallel Fuzzy Deep Learning Classifier

At present, with the growing number of Web 2.0 platforms such as Instagram, Facebook, and Twitter, users honestly communicate their opinions and ideas about events, services, and products. Owing to this rise in the number of social platforms and their extensive use by people, enormous amounts of data are produced hourly. However, sentiment analysis or opinion mining is considered as a useful tool that aims to extract the emotion and attitude from the user-posted data on social media platforms by using different computational methods to linguistic terms and various Natural Language Processing (NLP). Therefore, enhancing text sentiment classification accuracy has become feasible, and an interesting research area for many community researchers. In this study, a new Fuzzy Deep Learning Classifier (FDLC) is suggested for improving the performance of data-sentiment classification. Our proposed FDLC integrates Convolutional Neural Network (CNN) to build an effective automatic process for extracting the features from collected unstructured data and Feedforward Neural Network (FFNN) to compute both positive and negative sentimental scores. Then, we used the Mamdani Fuzzy System (MFS) as a fuzzy classifier to classify the outcomes of the two used deep (CNN+FFNN) learning models in three classes, which are: Neutral, Negative, and Positive. Also, to prevent the long execution time taking by our hybrid proposed FDLC, we have implemented our proposal under the Hadoop cluster. An experimental comparative study between our FDLC and some other suggestions from the literature is performed to demonstrate our offered classifier's effectiveness. The empirical result proved that our FDLC performs better than other classifiers in terms of true positive rate, true negative rate, false positive rate, false negative rate, error rate, precision, classification rate, kappa statistic, F1-score and time consumption, complexity, convergence, and stability.
Delving Deeper into Anti-Aliasing in ConvNets

Aliasing refers to the phenomenon that high frequency signals degenerate into completely different ones after sampling. It arises as a problem in the context of deep learning as downsampling layers are widely adopted in deep architectures to reduce parameters and computation. The standard solution is to apply a low-pass filter (e.g., Gaussian blur) before downsampling (Zhang in: ICML, 2020). However, it can be suboptimal to apply the same filter across the entire content, as the frequency of feature maps can vary across both spatial locations and feature channels. To tackle this, we propose an adaptive content-aware low-pass filtering layer, which predicts separate filter weights for each spatial location and channel group of the input feature maps. We investigate the effectiveness and generalization of the proposed method across multiple tasks, including image classification, semantic segmentation, instance segmentation, video instance segmentation, and image-to-image translation. Both qualitative and quantitative results demonstrate that our approach effectively adapts to the different feature frequencies to avoid aliasing while preserving useful information for recognition. Code is available at https://maureenzou.github.io/ddac/ .
Data and knowledge-driven named entity recognition for cyber security

Abstract Named Entity Recognition (NER) for cyber security aims to identify and classify cyber security terms from a large number of heterogeneous multisource cyber security texts. In the field of machine learning, deep neural networks automatically learn text features from a large number of datasets, but this data-driven method usually lacks the ability to deal with rare entities. Gasmi et al. proposed a deep learning method for named entity recognition in the field of cyber security, and achieved good results, reaching an F1 value of 82.8%. But it is difficult to accurately identify rare entities and complex words in the text.To cope with this challenge, this paper proposes a new model that combines data-driven deep learning methods with knowledge-driven dictionary methods to build dictionary features to assist in rare entity recognition. In addition, based on the data-driven deep learning model, an attention mechanism is adopted to enrich the local features of the text, better models the context, and improves the recognition effect of complex entities. Experimental results show that our method is better than the baseline model. Our model is more effective in identifying cyber security entities. The Precision, Recall and F1 value reached 90.19%, 86.60% and 88.36% respectively.
HTCInfoMax: A Global Model for Hierarchical Text Classification via Information Maximization

The current state-of-the-art model HiAGM for hierarchical text classification has two limitations.First, it correlates each text sample with all labels in the dataset which contains irrelevant information.Second, it does not consider any statistical constraint on the label representations learned by the structure encoder, while constraints for representation learning are proved to be helpful in previous work.In this paper, we propose HTCInfoMax to address these issues by introducing information maximization which includes two modules: text-label mutual information maximization and label prior matching.The first module can model the interaction between each text sample and its ground truth labels explicitly which filters out irrelevant information.The second one encourages the structure encoder to learn better representations with desired characteristics for all labels which can better handle label imbalance in hierarchical text classification.Experimental results on two benchmark datasets demonstrate the effectiveness of the proposed HTCInfoMax.
SDGNN: Learning Node Representation for Signed Directed Networks

Network embedding is aimed at mapping nodes in a network into low-dimensional vector representations. Graph Neural Networks (GNNs) have received widespread attention and lead to state-of-the-art performance in learning node representations. However, most GNNs only work in unsigned networks, where only positive links exist. It is not trivial to transfer these models to signed directed networks, which are widely observed in the real world yet less studied. In this paper, we first review two fundamental sociological theories (i.e., status theory and balance theory) and conduct empirical studies on real-world datasets to analyze the social mechanism in signed directed networks. Guided by related socio- logical theories, we propose a novel Signed Directed Graph Neural Networks model named SDGNN to learn node embeddings for signed directed networks. The proposed model simultaneously reconstructs link signs, link directions, and signed directed triangles. We validate our model’s effectiveness on five real-world datasets, which are commonly used as the benchmark for signed network embeddings. Experiments demonstrate the proposed model outperforms existing models, including feature-based methods, network embedding methods, and several GNN methods.
Discontinuous Named Entity Recognition as Maximal Clique Discovery

Named entity recognition (NER) remains challenging when entity mentions can be discontinuous.Existing methods break the recognition process into several sequential steps.In training, they predict conditioned on the golden intermediate results, while at inference relying on the model output of the previous steps, which introduces exposure bias.To solve this problem, we first construct a segment graph for each sentence, in which each node denotes a segment (a continuous entity on its own, or a part of discontinuous entities), and an edge links two nodes that belong to the same entity.The nodes and edges can be generated respectively in one stage with a grid tagging scheme and learned jointly using a novel architecture named Mac.Then discontinuous NER can be reformulated as a non-parametric process of discovering maximal cliques in the graph and concatenating the spans in each clique.Experiments on three benchmarks show that our method outperforms the state-ofthe-art (SOTA) results, with up to 3.5 percentage points improvement on F1, and achieves 5x speedup over the SOTA model.
Learning Graph Neural Networks with Positive and Unlabeled Nodes

Graph neural networks (GNNs) are important tools for transductive learning tasks, such as node classification in graphs, due to their expressive power in capturing complex interdependency between nodes. To enable GNN learning, existing works typically assume that labeled nodes, from two or multiple classes, are provided, so that a discriminative classifier can be learned from the labeled data. In reality, this assumption might be too restrictive for applications, as users may only provide labels of interest in a single class for a small number of nodes. In addition, most GNN models only aggregate information from short distances ( e.g. , 1-hop neighbors) in each round, and fail to capture long-distance relationship in graphs. In this article, we propose a novel GNN framework, long-short distance aggregation networks, to overcome these limitations. By generating multiple graphs at different distance levels, based on the adjacency matrix, we develop a long-short distance attention model to model these graphs. The direct neighbors are captured via a short-distance attention mechanism, and neighbors with long distance are captured by a long-distance attention mechanism. Two novel risk estimators are further employed to aggregate long-short-distance networks, for PU learning and the loss is back-propagated for model learning. Experimental results on real-world datasets demonstrate the effectiveness of our algorithm.
Review of dynamic gesture recognition

In recent years, gesture recognition has been widely used in the fields of intelligent driving, virtual reality, and human-computer interaction. With the development of artificial intelligence, deep learning has achieved remarkable success in computer vision. To help researchers better understanding the development status of gesture recognition in video, this article provides a detailed survey of the latest developments in gesture recognition technology for videos based on deep learning. The reviewed methods are broadly categorized into three groups based on the type of neural networks used for recognition: twostream convolutional neural networks, 3D convolutional neural networks, and Long-short Term Memory (LSTM) networks. In this review, we discuss the advantages and limitations of existing technologies, focusing on the feature extraction method of the spatiotemporal structure information in a video sequence, and consider future research directions.