Reviewerly/exHarmony huggingface dataset:
3D CNNs with Adaptive Temporal Feature Resolutions

While state-of-the-art 3D Convolutional Neural Networks (CNN) achieve very good results on action recognition datasets, they are computationally very expensive and require many GFLOPs. While the GFLOPs of a 3D CNN can be decreased by reducing the temporal feature resolution within the network, there is no setting that is optimal for all input clips. In this work, we therefore introduce a differentiable Similarity Guided Sampling (SGS) module, which can be plugged into any existing 3D CNN architecture. SGS empowers 3D CNNs by learning the similarity of temporal features and grouping similar features together. As a result, the temporal feature resolution is not anymore static but it varies for each input video clip. By integrating SGS as an additional layer within current 3D CNNs, we can convert them into much more efficient 3D CNNs with adaptive temporal feature resolutions (ATFR). Our evaluations show that the proposed module improves the state-of-the-art by reducing the computational cost (GFLOPs) by half while preserving or even improving the accuracy. We evaluate our module by adding it to multiple state-of-the-art 3D CNNs on various datasets such as Kinetics-600, Kinetics-400, mini-Kinetics, Something-Something V2, UCF101, and HMDB51.
Touching text line segmentation combined local baseline and connected component for Uchen Tibetan historical documents

The text lines of ancient Tibetan books are skewed and distorted, strokes are broken, and complex adjacent text lines touch each other, which makes text line segmentation extremely challenging. In this paper, a text line segmentation method based on local baselines and connected component allocation is proposed. First, the pseudotext line is detected by analyzing the horizontal projection, straight line detection and the average character height information, and then the local baseline position is determined in the pseudotext line area by the projection method. Second, the adhesion area detection is performed, which mainly includes the adhesion between characters and the adhesion between characters and strokes. The position relationship between the connected components is used to complete the adhesion between characters. A convolutional neural network is used to complete the adhesion between characters and strokes. Then, the watershed algorithm is used to segment touching connected components. Finally, the broken strokes are assigned to the text lines in which they belong according to the characteristics of Tibetan character structures. Subsequently, the assigned strokes are postprocessed to complete stroke correction, and finally the line segmentation is completed. Experiments show that this method can effectively reduce the influence of text line distortion and skew on text line segmentation, has a high degree of robustness, and has good segmentation accuracy for image text lines in Tibetan documents with touching and broken strokes.
Heralded quantum gates for hybrid systems via waveguide-mediated photon scattering

Universal quantum gates play a critical role in quantum information processing. Here, based on the scattering property of photons off single emitters in one-dimensional waveguides, we present some heralded protocols for realizing controlled-not, Toffoli, and Fredkin gates on hybrid systems. The control qubit of our gates is encoded on a flying photon, and the target qubits are encoded in the degenerate ground states of the emitter. In our schemes, the faulty scattering processes between emitters and photons caused by system imperfections can be transformed into the detection of photon polarization. That is, our three quantum gates are accomplished in a heralded way. Moreover, the quantum circuits for the three gates are versatile and compact, and no additional qubits are required. With current technique on manipulating the emitter-waveguide system, our protocols may be experimentally feasible. This work provides an effective avenue for implementing quantum logic gates.
New Encryption Algorithm Using Bit-Level Permutation and Non-Invertible Chaotic Map

Recently, a little research into image encryption has been used on chaotic economic maps.The current paper suggests a bit-level permutation and a non-invertible chaotic economic map to encrypt an image.Firstly, the secret key generation is linked to the plain image.So, the suggested algorithm may resist both known-plaintext and chosen-plaintext attacks.Then a bit-level permutation is performed for all the binary bits of the plain image's pixels, using the logistic map (permutation stage).It is used to improve the algorithm's security.Then pixel diffusion is applied using the 2D non-invertible chaotic economic map and bit-wise XOR operations.It is used to change the pixels' values and make them highly random.The results of the experiments and the security analyses show that the given image encryption algorithm is efficient with higher security.Some comparisons showed that the proposed algorithm outperformed many recent algorithms.Finally, the proposed algorithm may be able to withstand a variety of attacks.
Adadb: Adaptive Diff-Batch Optimization Technique for Gradient Descent

Gradient descent is the workhorse of deep neural networks.Gradient descent has the disadvantage of slow convergence.The famous way to overcome slow convergence is to use momentum.Momentum effectively increases the learning factor of gradient descent.Recently, many approaches have been proposed to control the momentum for better optimization towards global minima, such as Adam, diffGrad, and AdaBelief.Adam decreases the momentum by dividing it with square root of moving averages of squared past gradients or second moment.The sudden decrease in the second moment often results in the overshoot of the gradient from the minima and then settle at the closest minima.DiffGrad decreases this problem by using a friction constant based on the difference of current gradient and immediate past gradient in Adam.The friction constant further decreases the momentum and results in slow convergence.AdaBelief adapts the step size according to the belief in the current gradient direction.Another famous way of fast convergence is to increase the batch size adaptively.This paper proposes a new optimization technique named adaptive diff-batch or adadb that removes the problem of overshooting gradient in Adam, slow convergence in diffGrad, and combines the methods with adaptive batch size for further increase in convergence rate.The proposed technique uses the friction constant based on the past three differences of gradients rather than one as in diffGrad and a condition to decide the use of friction constant.The proposed technique has outperformed the Adam, diffGrad, and AdaBelief optimizers on synthetic complex non-convex functions and real-world datasets.
Adaptive Hierarchical Graph Reasoning with Semantic Coherence for Video-and-Language Inference

Video-and-Language Inference is a recently proposed task for joint video-and-language understanding. This new task requires a model to draw inference on whether a natural language statement entails or contradicts a given video clip. In this paper, we study how to address three critical challenges for this task: judging the global correctness of the statement involved multiple semantic meanings, joint reasoning over video and subtitles, and modeling long-range relationships and complex social interactions. First, we propose an adaptive hierarchical graph network that achieves in-depth understanding of the video over complex interactions. Specifically, it performs joint reasoning over video and subtitles in three hierarchies, where the graph structure is adaptively adjusted according to the semantic structures of the statement. Secondly, we introduce semantic coherence learning to explicitly encourage the semantic coherence of the adaptive hierarchical graph network from three hierarchies. The semantic coherence learning can further improve the alignment between vision and linguistics, and the coherence across a sequence of video segments. Experimental results show that our method significantly outperforms the baseline by a large margin.
Real-Time Sphere Sweeping Stereo from Multiview Fisheye Images

A set of cameras with fisheye lenses have been used to capture a wide field of view. The traditional scan-line stereo algorithms based on epipolar geometry are directly inapplicable to this non-pinhole camera setup due to optical characteristics of fisheye lenses; hence, existing complete 360° RGB-D imaging systems have rarely achieved realtime performance yet. In this paper, we introduce an efficient sphere-sweeping stereo that can run directly on multiview fisheye images without requiring additional spherical rectification. Our main contributions are: First, we introduce an adaptive spherical matching method that accounts for each input fisheye camera’s resolving power concerning spherical distortion. Second, we propose a fast inter-scale bilateral cost volume filtering method that refines distance in noisy and textureless regions with optimal complexity of O(n). It enables real-time dense distance estimation while preserving edges. Lastly, the fisheye color and distance images are seamlessly combined into a complete 360° RGB-D image via fast inpainting of the dense distance map. We demonstrate an embedded 360° RGB-D imaging prototype composed of a mobile GPU and four fisheye cameras. Our prototype is capable of capturing complete 360° RGB-D videos with a resolution of two megapixels at 29 fps. Results demonstrate that our real-time method outperforms traditional omnidirectional stereo and learning-based omnidirectional stereo in terms of accuracy and performance.
Research on insulator defect detection algorithm of transmission line based on CenterNet

The reliability of the insulator has directly affected the stable operation of electric power system. The detection of defective insulators has always been an important issue in smart grid systems. However, the traditional transmission line detection method has low accuracy and poor real-time performance. We present an insulator defect detection method based on CenterNet. In order to improve detection efficiency, we simplified the backbone network. In addition, an attention mechanism is utilized to suppress useless information and improve the accuracy of network detection. In image preprocessing, the blurring of some detected images results in the samples being discarded, so we use super-resolution reconstruction algorithm to reconstruct the blurred images to enhance the dataset. The results show that the AP of the proposed method reaches 96.16% and the reasoning speed reaches 30FPS under the test condition of NVIDIA GTX 1080 test conditions. Compared with Faster R-CNN, YOLOV3, RetinaNet and FSAF, the detection accuracy of proposed method is greatly improved, which fully proves the effectiveness of the proposed method.
Direct Tomography of High-Dimensional Density Matrices for General Quantum States of Photons

Quantum state tomography is the conventional method used to characterize density matrices for general quantum states. However, the data acquisition time generally scales linearly with the dimension of the Hilbert space, hindering the possibility of dynamic monitoring of a high-dimensional quantum system. Here, we demonstrate a direct tomography protocol to measure density matrices of photons in the position basis through the use of a polarization-resolving camera, where the dimension of density matrices can be as large as 580$\times$580 in our experiment. The use of the polarization-resolving camera enables parallel measurements in the position and polarization basis and as a result, the data acquisition time of our protocol does not increase with the dimension of the Hilbert space and is solely determined by the camera exposure time (on the order of 10 ms). Our method is potentially useful for the real-time monitoring of the dynamics of quantum states and paves the way for the development of high-dimensional, time-efficient quantum metrology techniques.
A Fully Digital True Random Number Generator With Entropy Source Based in Frequency Collapse

All cryptography systems have a True Random Number Generator (TRNG). In the process of validating, these systems are necessary for prototyping in Field Programmable Gate Array (FPGA). However, TRNG uses an entropy source based on non-deterministic effects challenging to replicate in FPGA. This work shows the problems and solutions to implement an entropy source based on frequency collapse in multimodal Ring Oscillators (RO). The entropy source implemented in FPGA pass all SP800-90B tests from the National Institute of Standards and Technology (NIST) with a good entropy compared to related works. The TRNG passes all NIST SP800-22 with and without the post-processing stage. Besides, the TRNG and the post-processing stage pass all tests of Application notes and Interpretation of the Scheme (AIS31). The TRNG implementation on a Xilinx Artix-7 XC7A100TCSG324 FPGA occupies less than 1% of the resources. This work presents 0.62 μs up to 9.92 μs of sampling latency and 1.1 Mbps up to 9.1 Mbps of bit rate throughput.
On the efficacy of old features for the detection of new bots

For more than a decade now, academicians and online platform administrators have been studying solutions to the problem of bot detection. Bots are computer algorithms whose use is far from being benign: malicious bots are purposely created to distribute spam, sponsor public characters and, ultimately, induce a bias within the public opinion. To fight the bot invasion on our online ecosystem, several approaches have been implemented, mostly based on (supervised and unsupervised) classifiers, which adopt the most varied account features, from the simplest to the most expensive ones to be extracted from the raw data obtainable through the Twitter public APIs. In this exploratory study, using Twitter as a benchmark, we compare the performances of four state-of-art feature sets in detecting novel bots: one of the output scores of the popular bot detector Botometer, which considers more than 1,000 features of an account to take a decision; two feature sets based on the account profile and timeline; and the information about the Twitter client from which the user tweets. The results of our analysis, conducted on six recently released datasets of Twitter accounts, hint at the possible use of general-purpose classifiers and cheap-to-compute account features for the detection of evolved bots.
6-D pose estimation method for large gear structure assembly using monocular vision

Measuring the pose of large gear structures during the assembly process using large volume measurement systems (LVMSs) is difficult owing to the shielding between the output and input shafts. In addition, collision damage caused by assembly errors occurs during the assembly process, which affects the assembly accuracy and product reliability. In this study, a 6-D pose estimation method based on the monocular vision was proposed, which comprises a feature detection method and a pose calculation method, which is based on key features and is improved by using the weighting coefficient that was applied to the assembly of large gear structures. Finally, the assembly system of large gear structures based on the monocular vision was built. The system shows that this method has high precision and efficiency and can be used in the automatic assembly of large gear structures instead of traditional manual methods.
Weighted oversampling algorithms for imbalanced problems and application in prediction of streamflow

Synthetic Minority Oversampling Technique (SMOTE) is one of the most prevalent oversampling methods in imbalanced classification. Due to its intrinsic drawbacks of generating new samples blindly without distinguishing noisy samples, many variants of SMOTE have been developed to avoid selecting the label-noise samples as seed samples or to remove label-noise samples after being oversampled. However, these variants interpolate new samples linearly among the minority samples and their neighbors randomly without considering the relative chaotic level between each minority sample and its neighbors. In this paper, we propose a general weighting framework that carefully designates the interpolation location of each synthetic sample by computing the chaotic levels between the seed sample and its neighbors, placing it closer to a safe and clean sample and far away from a chaotic one. This general weighting framework can be easily combined with diversified SMOTE variants, thus we called it W-SMOTEs. Extensive experiments on synthetic, UCI, and industrial datasets with different levels of label noise demonstrate that the W-SMOTEs can effectively reduce the noisy samples produced and can enhance the separability between classes.
Pushing it out of the Way: Interactive Visual Navigation

We have observed significant progress in visual navigation for embodied agents. A common assumption in studying visual navigation is that the environments are static; this is a limiting assumption. Intelligent navigation may involve interacting with the environment beyond just moving forward/backward and turning left/right. Sometimes, the best way to navigate is to push something out of the way. In this paper, we study the problem of interactive navigation where agents learn to change the environment to navigate more efficiently to their goals. To this end, we introduce the Neural Interaction Engine (NIE) to explicitly predict the change in the environment caused by the agent’s actions. By modeling the changes while planning, we find that agents exhibit significant improvements in their navigational capabilities. More specifically, we consider two downstream tasks in the physics-enabled, visually rich, AI2-THOR environment: (1) reaching a target while the path to the target is blocked (2) moving an object to a target location by pushing it. For both tasks, agents equipped with an NIE significantly outperform agents without the understanding of the effect of the actions indicating the benefits of our approach. The code and dataset are available at github.com/KuoHaoZeng/Interactive_Visual_Navigation.
Coarse-to-fine

Facial structure editing of portrait images is challenging given the facial variety, the lack of ground-truth, the necessity of jointly adjusting color and shape, and the requirement of no visual artifacts. In this paper, we investigate how to perform chin editing as a case study of editing facial structures. We present a novel method that can automatically remove the double chin effect in portrait images. Our core idea is to train a fine classification boundary in the latent space of the portrait images. This can be used to edit the chin appearance by manipulating the latent code of the input portrait image while preserving the original portrait features. To achieve such a fine separation boundary, we employ a carefully designed training stage based on latent codes of paired synthetic images with and without a double chin. In the testing stage, our method can automatically handle portrait images with only a refinement to subtle misalignment before and after double chin editing. Our model enables alteration to the neck region of the input portrait image while keeping other regions unchanged, and guarantees the rationality of neck structure and the consistency of facial characteristics. To the best of our knowledge, this presents the first effort towards an effective application for editing double chins. We validate the efficacy and efficiency of our approach through extensive experiments and user studies.
Towards cloud-edge collaborative online video analytics with fine-grained serverless pipelines

The ever-growing deployment scale of surveillance cameras and the users' increasing appetite for real-time queries have urged online video analytics. Synergizing the virtually unlimited cloud resources with agile edge processing would deliver an ideal online video analytics system; yet, given the complex interaction and dependency within and across video query pipelines, it is easier said than done. This paper starts with a measurement study to acquire a deep understanding of video query pipelines on real-world camera streams. We identify the potentials and practical challenges towards cloud-edge collaborative video analytics. We then argue that the newly emerged serverless computing paradigm is the key to achieve fine-grained resource partitioning with minimum dependency. We accordingly propose CEVAS, a Cloud-Edge collaborative Video Analytics system empowered by fine-grained Serverless pipelines. It builds flexible serverless-based infrastructures to facilitate fine-grained and adaptive partitioning of cloud-edge workloads for multiple concurrent query pipelines. With the optimized design of individual modules and their integration, CEVAS achieves real-time responses to highly dynamic input workloads. We have developed a prototype of CEVAS over Amazon Web Services (AWS) and conducted extensive experiments with real-world video streams and queries. The results show that by judiciously coordinating the fine-grained serverless resources in the cloud and at the edge, CEVAS reduces 86.9% cloud expenditure and 74.4% data transfer overhead of a pure cloud scheme and improves the analysis throughput of a pure edge scheme by up to 20.6%. Thanks to the fine-grained video content-aware forecasting, CEVAS is also more adaptive than the state-of-the-art cloud-edge collaborative scheme.
Vehicle detection in severe weather based on pseudo-visual search and HOG–LBP feature fusion

Vehicle detection in severe weather has always been a difficult task in the environmental perception of intelligent vehicles. This paper proposes a vehicle detection method based on pseudo-visual search and the histogram of oriented gradients (HOG)–local binary pattern (LBP) feature fusion. Using radar detection information, this method can directly extract the region of interest (ROI) of vehicles from infrared images by imitating human vision. Unlike traditional methods, the pseudo-visual search mechanism is independent of complex image processing and environmental interferences, thereby significantly improving the speed and accuracy of ROI extraction. More notably, the ROI extraction process based on pseudo-visual search can reduce image processing by 40%–80%, with an ROI extraction time of only 4 ms, which is far lower than the traditional algorithms. In addition, we used the HOG–LBP fusion feature to train the vehicle classifier, which improves the extraction ability of local and global features of vehicles. The HOG–LBP fusion feature can improve vehicle detection accuracy by 6%–9%, compared to a single feature. Experimental results show that the accuracy of vehicle detection is 92.7%, and the detection speed is 31 fps, which validates the feasibility of the proposed method and effectively improve the vehicle detection performance in severe weather
Turning circle based trajectory planning method of an underactuated AUV for the mobile docking mission

In this paper, a turning circle based trajectory planning method (TCBTPM) is proposed to conduct trajectory planning of an underactuated autonomous underwater vehicle (AUV) for mobile docking mission. Firstly, the six degrees of freedom kinematics and dynamics model of underactuated AUV is acquired. Following this, the simulation of horizontal turning circle trial is studied and the overview of mobile docking mission is introduced. Furthermore, TCBTPM is used to predict the relative position of AUV and the mobile docking station (MDS) in advance and generate a feasible trajectory for the mobile docking mission. Finally, a mobile docking guidance system containing TCBTPM, pure pursuit (PP) guidance law and line-of-sight (LOS) guidance law is employed to obtain desired guidance commands and guides the underactuated AUV to dock efficiently and reliably. • A turning circle based trajectory planning method is proposed. • A new guidance system is used to generate guidance commands for mobile docking mission. • This method has low computational burden and is suitable for real-time applications.