Reviewerly/exHarmony huggingface dataset:
Recognizing Unusual Activity with the Deep Learning Perspective in Crowd Segment

The data's unusual behavior suggests finding patterns in data that are not consistent with the expected behaviour. A developing prerequisite for more intelligent video vigilance of secure and open space utilizing shrewd vision frameworks can separate semantically significant behaviour towards the human spectator as typical behavior or anomalous behavior. Now presenting a novel-based energy approach for strange behavior recognition utilizing deep learning procedures. Utilize a versatile optical stream model to work on moving particles rather than articles and circuit highlights with data's shape and direction. We present and incorporate numerous behavioral models for precise irregular activity recognition in a complex crowd scene. We have to use the individual behavior model, yet also different social behavior models. The test results show that our proposed technique effectively recognizes the unusual behavior in a packed scene. This paper revolves around a bunch conducts assessment which can perceive regular conduct or abnormal conduct.
Image Deblurring based on Lightweight Multi-Information Fusion Network

Recently, deep learning based image deblurring has been well developed. However, exploiting the detailed image features in a deep learning framework always requires a mass of parameters, which inevitably makes the network suffer from high computational burden. To solve this problem, we propose a lightweight multi-information fusion network (LMFN) for image deblurring. The proposed LMFN is designed as an encoder-decoder architecture. In the encoding stage, the image feature is reduced to various small-scale spaces for multi-scale information extraction and fusion without a large amount of information loss. Then, a distillation network is used in the decoding stage, which allows the network benefit the most from residual learning while remaining sufficiently lightweight. Meanwhile, an information fusion strategy between distillation modules and feature channels is also carried out by attention mechanism. Through fusing different information in the proposed approach, our network can achieve state-of-the-art image deblurring result with smaller number of parameters and outperforms existing methods in model complexity.
Resource and Performance Estimation for CNN Models using Machine Learning

Field-Programmable Gate Array (FPGA) based hardware accelerators offer reconfigurability, performance, adaptability, and good energy efficiency. The majority of Convolutional Neural Network (CNN) based inference systems are initially developed using standardized frameworks like PyTorch, Tensor Flow, and more. These Python or Python-like models can be mapped on FPGAs to build accelerators. Mapping frameworks to port designs on an FPGA convert the CNN models to high-level languages like C/C++ or OpenCL so that standard tools like high-level synthesis can facilitate the mapping of models on an FPGA. The logic utilization and performance of FPGA-based accelerators are dependent on the CNN network parameters, architectural selection (data-flow, pipelined, etc.), and synthesis-based control of design generation. A scalable multi-layer CNN hardware accelerator is modeled in Vitis 2020 HLS tool. Early estimation of performance and hardware resources helps choose the best CNN network before those are executed for time-consuming high-level synthesis and physical design mapping for FPGAs. We present various Machine Learning (ML) models to estimate the Logic Utilization and Computation Time from the Python design descriptions of CNNs. Our results show a very successful and accurate estimation for performance and resource utilization over various multi-layer CNN networks in negligible time before running HLS synthesis.
A Dataset and System for Real-Time Gun Detection in Surveillance Video Using Deep Learning

Gun violence is a severe problem in the world, particularly in the United States. Deep learning methods have been studied to detect guns in surveillance video cameras or smart IP cameras and to send a real-time alert to security personals. One problem for the development of gun detection algorithms is the lack of large public datasets. In this work, we first publish a dataset with 51K annotated gun images for gun detection and other 51K cropped gun chip images for gun classification we collect from a few different sources. To our knowledge, this is the largest dataset for the study of gun detection. This dataset can be downloaded at www.linksprite.com/gundetection-datasets. We present a gun detection system using a smart IP camera as an embedded edge device and a cloud server as a manager for device, data, alert, and to further reduce the false positive rate. We study to find solutions for gun detection in an embedded device and for gun classification on the edge device and the cloud server. This edge/cloud framework makes the deployment of gun detection in the real world possible.
Robust Visual Object Tracking with Spatiotemporal Regularisation and Discriminative Occlusion Deformation

Spatiotemporal regularized Discriminative Correlation Filters (DCF) have been proposed recently for visual tracking, achieving state-of-the-art performance. However, the tracking performance of the online learning model used in this kind methods is highly dependent on the quality of the appearance feature of the target, and the target feature appearance could be heavily deformed due to the occlusion by other objects or the variations in their dynamic self-appearance. In this paper, we propose a new approach to mitigate these two kinds of appearance deformation. Firstly, we embed the occlusion perception block into the model update stage, then we adaptively adjust the model update according to the situation of occlusion. Secondly, we use the relatively stable colour statistics to deal with the appearance shape changes in large targets, and compute the histogram response scores as a complementary part of final correlation response. Extensive experiments are performed on four well-known datasets, i.e. OTB100, VOT-2018, UAV123, and TC128. The results show that the proposed approach outperforms the baseline DCF method, especially, on the TC128/UAV123 datasets, with a gain of over 4.05 %/2.43% in mean overlap precision. We will release our code at https://github.com/SYLan2019/STD0D.
Omnipotent Virtual Giant for Remote Human–Swarm Interaction

This paper proposes an intuitive human-swarm interaction framework inspired by our childhood memory in which we interacted with living ants by changing their positions and environments as if we were omnipotent relative to the ants. In virtual reality, analogously, we can be a super-powered virtual giant who can supervise a swarm of robots in a vast and remote environment by flying over or resizing the world, and coordinate them by picking and placing a robot or creating virtual walls. This work implements this idea by using Virtual Reality along with Leap Motion, which is then validated by proof-of-concept experiments using real and virtual mobile robots in mixed reality. We conduct a usability analysis to quantify the effectiveness of the overall system as well as the individual interfaces proposed in this work. The results reveal that the proposed method is intuitive and feasible for interaction with swarm robots, but may require appropriate training for the new end-user interface device.
An Efficient Enhanced Full Homomorphic Encryption for Securing Video in Cloud Environment

At present days, exponential growth in the transmission of multimedia data takes place due to a significant rise in network bandwidth and video image compression technologies. However, the transmission of videos over wireless channels often brings an unseen risk that sensitive video details might be corrupted and distributed in an illegal way. So, the security of video transmission has become a hot research topic. Several encryption models have been presented in the literature, yet, it is believed that the performance of encryption process can be further improved. In this perspective, an efficient novel video encryption technique is presented using an enhanced variant of Fully Homomorphic Encryption (FHE) model called as EFHE model. By the hybridization of Ducas and Micciancio (DM) with the FHE model presented by Gentry, Sahai, and Waters (GSW), matrix operations vector additions are properly employed in the proposed EFHE model. In addition, a new key generation scheme to increase the fastness of the encryption process. The EFHE model is designed and placed on a cloud environment which leads to reduced cloud user's communication and computation complexity. It is ensured that the presented EFHE model is highly efficient and secure over the compared methods.
Facial expression generation of 3D avatar based on semantic analysis

3D avatars are widely used in various fields of emerging technology, from augmented reality to social robots. To interact in a natural way with the user, they must be able to show at least some basic emotions. However, generating animation for these virtual avatars is a time-consuming task and a creative process. The main goal of this work is to facilitate a generation of facial animation of basic emotions on a 3D avatar. To this end, we developed and compared two approaches. The first method consists of the generation of animation using tuning Blendshape features of the 3D model, whereas the second method captures it from the real face and maps it on the model correspondingly. Additionally, the text, from which the emotion was estimated, was passed to lip synchronization software for generating realistic lip movements for the avatar. Then, animations of six basic emotions were shown in different variations in the survey and respondents were asked to guess the emotion shown in the video. Besides, such anthropomorphic features of the avatar as human-likeness, life-likeness and pleasantness were examined. In general, the analysis of the survey provided the following interesting findings: a) participants did not have significant differences in recognizing emotions based on the type of animation generation method; b) inclusion of voice significantly enhanced the recognition of emotion. In relation to participants’ accuracy of emotion recognition, Excitement and Happiness were mostly confused between each other more than any other two emotions, while Anger was the easiest emotion to recognize.
Gaussian Mixture Filter for Angles-Only Orbit Determination in Modified Equinoctial Elements

A Gaussian mixture orbit determination filter is developed using a state vector that consists of equinoctial-like osculating elements. This new filter seeks to reduce the number of Gaussian mixture elements that are needed in order to accurately model the Bayesian posterior distributions that apply to the angles-only orbit determination problem. The modified version of the equinoctial elements replaces the elements and , whose sum squared is bounded by 1. The two replacement elements are unbounded, and and can be determined from them. This modification allows the orbit determination filter to operate in an unbounded state space. The new state requires the development of a corresponding mixand covariance upper bound due to the particular type of Gaussian mixture filter that is implemented. The upper bound is used in a mixture resampling algorithm in a way which ensures that extended Kalman filter calculations will be sufficiently accurate for each mixand’s computations. The resulting filter is able to reduce the required number of mixands from 5000 to 500 for angles-only orbit determination of a geosynchronous spacecraft.
Benchmark of Bitrate Adaptation in Video Streaming

The HTTP adaptive streaming technique opened the door to cope with the fluctuating network conditions during the streaming process by dynamically adjusting the volume of the future chunks to be downloaded. The bitrate selection in this adjustment inevitably involves the task of predicting the future throughput of a video session, owing to which various heuristic solutions have been explored. The ultimate goal of the present work is to explore the theoretical upper bounds of the QoE that any ABR algorithm can possibly reach, therefore providing an essential step to benchmarking the performance evaluation of ABR algorithms. In our setting, the QoE is defined in terms of a linear combination of the average perceptual quality and the buffering ratio. The optimization problem is proven to be NP-hard when the perceptual quality is defined by chunk size and conditions are given under which the problem becomes polynomially solvable. Enriched by a global lower bound, a pseudo-polynomial time algorithm along the dynamic programming approach is presented. When the minimum buffering is given higher priority over higher perceptual quality, the problem is shown to be also NP-hard, and the above algorithm is simplified and enhanced by a sequence of lower bounds on the completion time of chunk downloading, which, according to our experiment, brings a 36.0% performance improvement in terms of computation time. To handle large amounts of data more efficiently, a polynomial-time algorithm is also introduced to approximate the optimal values when minimum buffering is prioritized. Besides its performance guarantee, this algorithm is shown to reach 99.938% close to the optimal results, while taking only 0.024% of the computation time compared to the exact algorithm in dynamic programming.
Hashtags Recommendations in Twitter based on Collaborative Filtering

Social networks generate each day a vast amount of heterogeneous information, which can be used for recommendation systems, in order to provide personalized service. Currently, these systems are being developed to help Twitter users find items that is of interest to them, as a way to solve the information overload problem. To take advantages provided by collaborative filtering recommendation systems, in this paper we implement the KNN algorithm for the hashtags recommendation in the Twitter social network, both users and hashtags are used to predict the missing values in the user-item matrix. As in this matrix the class is binary and imbalanced, we apply a sub-sampling technique to balance the class. A memory-based approach is used to generate the final recommendations. The experiments are carried out on a real dataset, extracted from Twitter social network interactions. The experimental results demonstrate an improvement in the quality of both predictions and recommendations using a balanced dataset.
Text To Image Synthesis With Erudite Generative Adversarial Networks

In this paper, an Erudite Generative Adversarial Networks (EruditeGAN) is proposed for the text-to-image synthesis task. By introducing additional image distribution related to the original image into the network structure, the entire network can learn more about the image distribution and become more knowledgeable. In this case, it can be more clear about the distribution of the image that needs to be synthesized and finally synthesize high-quality results. Experiments well validate our method’s effectiveness and demonstrate the different effects of different distribution situations on the final results. According to the quantitative results of Fréchet Inception Distance (FID) and R-precision, our method’s comprehensive score is the best, which reflects our results are closer to the real image effect.
Uncertainty‐aware network alignment

Network alignment (NA) aims to link common nodes across multiple networks and is an essential task in many graph mining applications. Despite the progress achieved by many recent works, several fundamental limitations have eluded the proper cohesive way of addressing, including matching confusion, lack of the formal treatment of uncertainty, and Point-to-Point (P2P) constraint. This study proposes a novel framework UANA (Uncertainty-Aware Network Alignment) to tackle the limitations of the existing works. By embedding nodes as Gaussian distributions rather than point vectors, UANA enables to capture the uncertainty of a node representation, while being able to discriminate the anchor nodes from the potentially confusing neighbors. We address the P2P matching constraint by introducing an adversarial learning paradigm, which relaxes the exact matching assumption during training with an across-domain generative procedure to reduce the matching errors on testing nodes. In the end, interpretability methods are included to explain the aligning results made by our UANA based on the robust statistics, which enables the explanation of the effect of individual training sample on the NA performance without the need of retraining the model. Extensive experiments conducted on real-world data sets demonstrate that UANA significantly outperforms existing state-of-the-art baselines while providing explainable results.
Efficient Image-Warping Framework for Content-Adaptive Superpixels Generation

We address the problem of efficient content-adaptive superpixel segmentation. Instead of adapting the size and/or amount of superpixels to the image content, we propose a warpingtransform that makes the image content more suitable to be segmented into regular superpixels. Regular superpixels in the warped image induce content-adaptive superpixels in the original image with improved segmentation accuracy. To efficiently compute the warping transform, we develop an iterative coarse-to-fine optimization procedure and employ a parallelization strategy allowing for a speedy GPU-based implementation. This solution works as a simple ‘add-on’ framework over an underlying segmentation algorithm and requires no additional parameters. Compared to the state-of-the-art methods, our approach provides competitive quality results and achieves a better time-accuracy trade-off. We further demonstrate the effectiveness of our method with an application to disparity estimation.
Understanding the Limitations of SfM-Based Camera Calibration on Multi-View Stereo Reconstruction

The recent success on convolutional neural networks (CNNs) research has triggered the interest to improve the stereo reconstruction. There have been many considerations to reconstruct unstructured general images. However, photos taken in an unstructured environment are not well applied to previous research techniques. This paper proposes a robust method in this special environment. First, camera parameters are extracted using two types of Structure-from-Motion (SfM). In general, intrinsic camera parameters are extracted via camera calibration, and extrinsic parameters are computed by SfM. After that, we put these parameters as an input into two types of multiview stereo methods and compare the results of each methods. Finally, the structured indoor dataset which is called ETRI 360° dataset has better reconstruction results compared with unstructured dataset by using specific SfM and MVS methods.
Effective information, political structure and economic growth

Abstract Digital transformation of information led us to reconsider Hayek’s (American Economic Review, 35, 519–530) insight on a fuller use of information and re‐classify political structures based on their information protection policies. This allows us to link the accumulation of information with the political structure to frame their joint impact on economic growth. We develop a model of ‘effective information’ beginning with information production and absorption and then allowing for its political propagation based on the degree of information protection. Using data from 40 countries, we find: (i) effective information and its spillovers contribute to an increase in productivity; and (ii) reductions in information protection bring larger increases in effective information as economies near an ‘information‐technology frontier’ contributing to economic growth divergence.
Interactive information bottleneck for high-dimensional co-occurrence data clustering

Clustering high-dimensional data is quite challenging due to lots of redundant and irrelevant information contained in features. Most existing methods sequentially or jointly perform the feature dimensionality reduction and data clustering on the low-dimensional representations. However, the relationships between the clustered data points and the dimension-reduced features, as well as the influence of the relationships on the low-dimensional feature subspace learning are neglected in these methods. In this paper, an embarrassingly simple yet effective interactive information bottleneck (IIB) method is proposed for high-dimensional co-occurrence data clustering by simultaneously performing data clustering and low-dimensional feature subspace learning. What is different from existing methods is that, we perform data clustering while maximally preserving the correlations between the data clusters and the learned dimension-reduced features, and simultaneously learn the low-dimensional feature subspace while maintaining the correlations with the data clustering results obtained in the previous iteration. Thus, the two stages are interactive and refined mutually. Finally, a new twin “draw-and-merge” method is designed for optimization. Experimental results on four high-dimensional datasets demonstrate the superiority and effectiveness of the proposed method.
Discrete light sheet microscopic segmentation of left ventricle using morphological tuning and active contours

Left ventricular segmentation using cardiovascular MR scan is required for the diagnosis and further cure of cardiac diseases. Automatic systems for left ventricle segmentation are being studied for attaining more accurate results in a shorter period of time. A novel algorithm introducing discrete segmentation of left ventricle achieves an independent processing of images swiftly. The workflow consists of four segments; first, automated localization is performed on the MR image. Second, performing preprocessing intimately improves and enhances the quality of image using mean contrast adjustment. Central segmentation of endocardium and epicardium layers includes novel MTAC (Morphological tuning using active contours) segmentation algorithm that provides a perfect combination of active contours and morphological tuning to bring an adequate and desirable segmentation. The prospective snake model is a restrained progression, which takes iterations for an impulse throughout the left ventricle contours. At the end, contrast based refining overcomes minor edge problems for both outer and inner boundaries. Proposed algorithm is evaluated via Sunnybrook cardiac MR images by producing an overall average perpendicular distance 2.45 mm, an average dice matrix (endo: 91.3%; epi: 92.16%) and 91.7% dice matrix of overall endocardium and epicardium contours from ground truth contours.