Reviewerly/exHarmony huggingface dataset:
Intelligent human action recognition using an ensemble model of evolving deep networks with swarm-based optimization

Automatic interpretation of human actions from realistic videos attracts increasing research attention owing to its growing demand in real-world deployments such as biometrics, intelligent robotics, and surveillance. In this research, we propose an ensemble model of evolving deep networks comprising Convolutional Neural Networks (CNNs) and bidirectional Long Short-Term Memory (BLSTM) networks for human action recognition. A swarm intelligence (SI)-based algorithm is also proposed for identifying the optimal hyper-parameters of the deep networks. The SI algorithm plays a crucial role for determining the BLSTM network and learning configurations such as the learning and dropout rates and the number of hidden neurons, in order to establish effective deep features that accurately represent the temporal dynamics of human actions. The proposed SI algorithm incorporates hybrid crossover operators implemented by sine, cosine, and tanh functions for multiple elite offspring signal generation, as well as geometric search coefficients extracted from a three-dimensional super-ellipse surface. Moreover, it employs a versatile search process led by the yielded promising offspring solutions to overcome stagnation. Diverse CNN–BLSTM networks with distinctive hyper-parameter settings are devised. An ensemble model is subsequently constructed by aggregating a set of three optimized CNN–BLSTM​ networks based on the average prediction probabilities. Evaluated using several publicly available human action data sets, our evolving ensemble deep networks illustrate statistically significant superiority over those with default and optimal settings identified by other search methods. The proposed SI algorithm also shows great superiority over several other methods for solving diverse high-dimensional unimodal and multimodal optimization functions with artificial landscapes.
Hawk-Eye: An AI-Powered Threat Detector for Intelligent Surveillance Cameras

With recent advances in both AI and IoT capabilities, it is possible than ever to implement surveillance systems that can automatically identify people who might represent a potential security threat to the public in real-time. Imagine a surveillance camera system that can detect various on-body weapons, masked faces, suspicious objects and traffic. This system could transform surveillance cameras from passive sentries into active observers which would help in preventing a possible mass shooting in a school, stadium or mall. In this paper, we present a prototype implementation of such systems, Hawk-Eye, an AI-powered threat detector for smart surveillance cameras. Hawk-Eye can be deployed on centralized servers hosted in the cloud, as well as locally on the surveillance cameras at the network edge. Deploying AI-enabled surveillance applications at the edge enables the initial analysis of the captured images to take place on-site, which reduces the communication overheads and enables swift security actions. At the cloud side, we built a Mask R-CNN model that can detect suspicious objects in an image captured by a camera at the edge. The model can generate a high-quality segmentation mask for each object instance in the image, along with the confidence percentage and classification time. The camera side used a Raspberry Pi 3 device, Intel Neural Compute Stick 2 (NCS 2), and Logitech C920 webcam. At the camera side, we built a CNN model that can consume a stream of images directly from an on-site webcam, classify them, and displays the results to the user via a GUI-friendly interface. A motion detection module is developed to capture images automatically from the video when a new motion is detected. Finally, we evaluated our system using various performance metrics such as classification time and accuracy. Our experimental results showed an average overall prediction accuracy of 94% on our dataset.
An Enhanced Histopathology Analysis: An AI-Based System for Multiclass Grading of Oral Squamous Cell Carcinoma and Segmenting of Epithelial and Stromal Tissue

Oral squamous cell carcinoma is most frequent histological neoplasm of head and neck cancers, and although it is localized in a region that is accessible to see and can be detected very early, this usually does not occur. The standard procedure for the diagnosis of oral cancer is based on histopathological examination, however, the main problem in this kind of procedure is tumor heterogeneity where a subjective component of the examination could directly impact patient-specific treatment intervention. For this reason, artificial intelligence (AI) algorithms are widely used as computational aid in the diagnosis for classification and segmentation of tumors, in order to reduce inter- and intra-observer variability. In this research, a two-stage AI-based system for automatic multiclass grading (the first stage) and segmentation of the epithelial and stromal tissue (the second stage) from oral histopathological images is proposed in order to assist the clinician in oral squamous cell carcinoma diagnosis. The integration of Xception and SWT resulted in the highest classification value of 0.963 (σ = 0.042) AUCmacro and 0.966 (σ = 0.027) AUCmicro while using DeepLabv3+ along with Xception_65 as backbone and data preprocessing, semantic segmentation prediction resulted in 0.878 (σ = 0.027) mIOU and 0.955 (σ = 0.014) F1 score. Obtained results reveal that the proposed AI-based system has great potential in the diagnosis of OSCC.
Facebook's "Oversight Board:" Move Fast with Stable Infrastructure and Humility

Facebook’s proposed Oversight Board is one of the most ambitious constitution-making projects of the modern era. With pre-existing governance of tech platforms delegitimized by the ongoing “techlash,” this represents a pivotal moment when new constitutional forms can emerge that will shape the future of online discourse. For all the potential of the Facebook Oversight Board, there are many things it cannot be. It will not hear a sufficient amount of cases to be a meaningful response to calls for greater due process in individual content moderation decisions. Nor will it be able to become a font for global speech norms for the worldwide platform. The true value that the Board can bring to Facebook’s content moderation ecosystem lies between these two extremes of individual error correction and the settlement of globally applicable speech rules. The institutional offering of the Board should focus on two primary, but more modest, functions. First, it can help highlight weaknesses in the policy formation process at Facebook, removing blockages (such as blind spots and inertia) in the “legislative process” leading to the formulation of its Community Standards. Second, by providing an independent forum for the discussion of disputed content moderation decisions, the Board can be an exemplar of the public reasoning necessary for persons in a pluralistic community to come to accept the rules that govern them, even if they disagree with the substance of those rules. Understanding the institutional role of the Board in these terms provides useful insights into the institutional design that will best help it achieve these goals.
DI-Fusion: Online Implicit 3D Reconstruction with Deep Priors

Previous online 3D dense reconstruction methods struggle to achieve the balance between memory storage and surface quality, largely due to the usage of stagnant underlying geometry representation, such as TSDF (truncated signed distance functions) or surfels, without any knowledge of the scene priors. In this paper, we present DI-Fusion (Deep Implicit Fusion), based on a novel 3D representation, i.e. Probabilistic Local Implicit Voxels (PLIVoxs), for online 3D reconstruction with a commodity RGB-D camera. Our PLIVox encodes scene priors considering both the local geometry and uncertainty parameterized by a deep neural network. With such deep priors, we are able to perform online implicit 3D reconstruction achieving state-of-the-art camera trajectory estimation accuracy and mapping quality, while achieving better storage efficiency compared with previous online 3D reconstruction approaches. Our implementation is available at https://www.github.com/huangjh-pub/di-fusion.
Robust and Accurate Object Detection via Adversarial Learning

Data augmentation has become a de facto component for training high-performance deep image classifiers, but its potential is under-explored for object detection. Noting that most state-of-the-art object detectors benefit from fine-tuning a pre-trained classifier, we first study how the classifiers’ gains from various data augmentations transfer to object detection. The results are discouraging; the gains diminish after fine-tuning in terms of either accuracy or robustness. This work instead augments the fine-tuning stage for object detectors by exploring adversarial examples, which can be viewed as a model-dependent data augmentation. Our method dynamically selects the stronger adversarial images sourced from a detector’s classification and localization branches and evolves with the detector to ensure the augmentation policy stays current and relevant. This model-dependent augmentation generalizes to different object detectors better than AutoAugment, a model-agnostic augmentation policy searched based on one particular detector. Our approach boosts the performance of state-of-the-art EfficientDets by +1.1 mAP on the COCO object detection benchmark. It also improves the detectors’ robustness against natural distortions by +3.8 mAP and against domain shift by +1.3 mAP.
Educación superior e investigación en Latinoamérica: Transición al uso de tecnologías digitales por Covid-19/ Higher education and research in Latin America: Transition to the use of digital technologies by Covid-19

The development of initiatives to overcome barriers that affect the functional ecosystem of higher education and the sustainability of education in Latin America, leads to the analysis of higher education and research in Latin America considering the transition to the use of digital technologies by COVID-19.Through a computerized study on web content of 25 universities, selected considering the QS Latin America Ranking index and five Latin American countries in which mayor the contagion crisis prevails.Likewise, the contributions promoted by government agencies and universities are listed.The results determine that the information on the academic programs is accessed through digital environments, in terms of research, little projection prevails over the advances in times of pandemic, generating uncertainty in the face of the revitalization of the science-technology-humanities trilogy, in Latin America.It is concluded that the scope of technologies to overcome student desertion is weakly spread.In research, the visibility of strategies that guarantee their continuity is omitted, generating paralysis or delay, influencing scientific-technical progress.It is necessary to go beyond the exclusive use of technologies for teaching, and promote the dissemination of accessibility, visibility and interoperability of technological platforms that respond to the requirements of an educational environment violated by
Multi-view 3D Reconstruction with Transformers

Deep CNN-based methods have so far achieved the state of the art results in multi-view 3D object reconstruction. Despite the considerable progress, the two core modules of these methods - view feature extraction and multi-view fusion, are usually investigated separately, and the relations among multiple input views are rarely explored. Inspired by the recent great success in Transformer models, we reformulate the multi-view 3D reconstruction as a sequence-to-sequence prediction problem and propose a framework named 3D Volume Transformer. Unlike previous CNN-based methods using a separate design, we unify the feature extraction and view fusion in a single Transformer network. A natural advantage of our design lies in the exploration of view-to-view relationships using self-attention among multiple unordered inputs. On ShapeNet - a large-scale 3D reconstruction benchmark, our method achieves a new state-of-the-art accuracy in multi-view reconstruction with fewer parameters (70% less) than CNN-based methods. Experimental results also suggest the strong scaling capability of our method. Our code will be made publicly available.
Dynamic Spatial Propagation Network for Depth Completion

Image-guided depth completion aims to generate dense depth maps with sparse depth measurements and corresponding RGB images. Currently, spatial propagation networks (SPNs) are the most popular affinity-based methods in depth completion, but they still suffer from the representation limitation of the fixed affinity and the over smoothing during iterations. Our solution is to estimate independent affinity matrices in each SPN iteration, but it is over-parameterized and heavy calculation.This paper introduces an efficient model that learns the affinity among neighboring pixels with an attention-based, dynamic approach. Specifically, the Dynamic Spatial Propagation Network (DySPN) we proposed makes use of a non-linear propagation model (NLPM). It decouples the neighborhood into parts regarding to different distances and recursively generates independent attention maps to refine these parts into adaptive affinity matrices. Furthermore, we adopt a diffusion suppression (DS) operation so that the model converges at an early stage to prevent over-smoothing of dense depth. Finally, in order to decrease the computational cost required, we also introduce three variations that reduce the amount of neighbors and attentions needed while still retaining similar accuracy. In practice, our method requires less iteration to match the performance of other SPNs and yields better results overall. DySPN outperforms other state-of-the-art (SoTA) methods on KITTI Depth Completion (DC) evaluation by the time of submission and is able to yield SoTA performance in NYU Depth v2 dataset as well.
Hybrid flexible (HyFlex) teaching and learning: climbing the mountain of implementation challenges for synchronous online and face-to-face seminars during a pandemic

Abstract In 2020, King’s College London introduced HyFlex teaching as a means to supplement online and face-to-face teaching and to respond to Covid-19 restrictions. This enabled teaching to a mixed cohort of students (both online and on campus). This article provides an outline of how such an approach was conceptualized and implemented in a higher-education institution during an intense three-month period over that summer and prior to the limited re-opening of the university campus. This was a new approach that offers a number of pointers for reflection and provides key insights in on this novel learning environment and the physical and pedagogical contexts in which learning can occur. Technical implementation factors are detailed, along with both reflections on challenges and solutions. Pedagogical issues such as cognitive load, social presence, and resolving the issues of a cohort spread across two locations are discussed. While we should be mindful of the limitations of this relatively-specific research, and shouldn’t therefore over-extrapolate our findings, one key finding is that delivering Hyflex is associated with a higher cognitive load. Further, the audio quality of our implementation enhanced the feeling of presence in the learning environment. We recommend providing appropriate technical and pedagogical training, as well as audio-visual and digital education support.
Self-Supervised Attentive Generative Adversarial Networks for Video Anomaly Detection

Video anomaly detection (VAD) refers to the discrimination of unexpected events in videos. The deep generative model (DGM)-based method learns the regular patterns on normal videos and expects the learned model to yield larger generative errors for abnormal frames. However, DGM cannot always do so, since it usually captures the shared patterns between normal and abnormal events, which results in similar generative errors for them. In this article, we propose a novel self-supervised framework for unsupervised VAD to tackle the above-mentioned problem. To this end, we design a novel self-supervised attentive generative adversarial network (SSAGAN), which is composed of the self-attentive predictor, the vanilla discriminator, and the self-supervised discriminator. On the one hand, the self-attentive predictor can capture the long-term dependences for improving the prediction qualities of normal frames. On the other hand, the predicted frames are fed to the vanilla discriminator and self-supervised discriminator for performing true–false discrimination and self-supervised rotation detection, respectively. Essentially, the role of the self-supervised task is to enable the predictor to encode semantic information into the predicted normal frames via adversarial training, in order for the angles of rotated normal frames can be detected. As a result, our self-supervised framework lessens the generalization ability of the model to abnormal frames, resulting in larger detection errors for abnormal frames. Extensive experimental results indicate that SSAGAN outperforms other state-of-the-art methods, which demonstrates the validity and advancement of SSAGAN.
A novel explainable COVID-19 diagnosis method by integration of feature selection with random forest

Several Artificial Intelligence-based models have been developed for COVID-19 disease diagnosis. In spite of the promise of artificial intelligence, there are very few models which bridge the gap between traditional human-centered diagnosis and the potential future of machine-centered disease diagnosis. Under the concept of human-computer interaction design, this study proposes a new explainable artificial intelligence method that exploits graph analysis for feature visualization and optimization for the purpose of COVID-19 diagnosis from blood test samples. In this developed model, an explainable decision forest classifier is employed to COVID-19 classification based on routinely available patient blood test data. The approach enables the clinician to use the decision tree and feature visualization to guide the explainability and interpretability of the prediction model. By utilizing this novel feature selection phase, the proposed diagnosis model will not only improve diagnosis accuracy but decrease the execution time as well.
An Accurate UAV 3-D Path Planning Method for Disaster Emergency Response Based on an Improved Multiobjective Swarm Intelligence Algorithm

Planning a practical three-dimensional (3-D) flight path for unmanned aerial vehicles (UAVs) is a key challenge for the follow-up management and decision making in disaster emergency response. The ideal flight path is expected to balance the total flight path length and the terrain threat, to shorten the flight time and reduce the possibility of collision. However, in the traditional methods, the tradeoff between these concerns is difficult to achieve, and practical constraints are lacking in the optimized objective functions, which leads to inaccurate modeling. In addition, the traditional methods based on gradient optimization lack an accurate optimization capability in the complex multimodal objective space, resulting in a nonoptimal path. Thus, in this article, an accurate UAV 3-D path planning approach in accordance with an enhanced multiobjective swarm intelligence algorithm is proposed (APPMS). In the APPMS method, the path planning mission is converted into a multiobjective optimization task with multiple constraints, and the objectives based on the total flight path length and degree of terrain threat are simultaneously optimized. In addition, to obtain the optimal UAV 3-D flight path, an accurate swarm intelligence search approach based on improved ant colony optimization is introduced, which can improve the global and local search capabilities by using the preferred search direction and random neighborhood search mechanism. The effectiveness of the proposed APPMS method was demonstrated in three groups of simulated experiments with different degrees of terrain threat, and a real-data experiment with 3-D terrain data from an actual emergency situation.
Discrete Opinion Tree Induction for Aspect-based Sentiment Analysis

Dependency trees have been intensively used with graph neural networks for aspect-based sentiment classification.Though being effective, such methods rely on external dependency parsers, which can be unavailable for low-resource languages or perform worse in low-resource domains.In addition, dependency trees are also not optimized for aspect-based sentiment classification.In this paper, we propose an aspect-specific and language-agnostic discrete latent opinion tree model as an alternative structure to explicit dependency trees.To ease the learning of complicated structured latent variables, we build a connection between aspect-to-context attention scores and syntactic distances, inducing trees from the attention scores.Results on six English benchmarks, one Chinese dataset and one Korean dataset show that our model can achieve competitive performance and interpretability.
Pun Generation with Surprise

He He, Nanyun Peng, Percy Liang. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.
Knowledge Distillation Using Output Errors for Self-attention End-to-end Models

Most automatic speech recognition (ASR) neural network models are not suitable for mobile devices due to their large model sizes. Therefore, it is required to reduce the model size to meet the limited hardware resources. In this study, we investigate sequence-level knowledge distillation techniques of self-attention ASR models for model compression. In order to overcome the performance degradation of compressed models, our proposed method adds an exponential weight to the sequence-level knowledge distillation loss function, which reflects the word error rate of the output of the teacher model based on the ground-truth word sequences. Evaluated on LibriSpeech dataset, the proposed knowledge distillation method achieves significant improvements over the student baseline model.
How Do I Feel? Identifying Emotional Expressions on Facebook Reactions Using Clustering Mechanism

The recognition of emotions and feelings through computer technology and devices has been widely explored in recent years. Social networks have become a natural environment in which users express their feelings and opinions through social media, and this includes their Facebook reactions. The aim of this study was to investigate whether the emoticons have chosen by users in social network news actually express the emotions they wish to express, having as indicative, the polarity of the emotions, and the six basic emotions. The data collection was carried out following three courses of action: 1) survey of the posts with higher reactions rates of popular news pages; 2) selection of news by a panel of experts to verify its reliability; and 3) identification of reactions, polarity, and basic emotions flagged by Facebook users for each news item. Finally, an Expectation-Maximization algorithm was deployed to find the relationship between the reactions and the basic emotions signaled. The results made it possible to determine the polarity and the correlation of the reactions with the emotional expressions. This suggests that the use of reactions in feelings analysis algorithms can increase the confidence in determining the emotion that the content reflects and the emotional state of the social network users.