Miracle-dz/UniFAR-FTUs huggingface dataset:
Improving Low-Resolution Image Classification by Super-Resolution with Enhancing High-Frequency Content

With the prosperous development of Convolutional Neural Networks, currently they can perform excellently on visual understanding tasks when the input images are high quality and common quality images. However, large degradation in performance always occur when the input images are low quality images. In this paper, we propose a new super-resolution method in order to improve the classification performance for low-resolution images. In an image, the regions in which pixel values vary dramatically contain more abundant high frequency contents compared to other parts. Based on this fact, we design a weight map and integrate it with a super-resolution CNN training framework. During the process of training, this weight map can find out positions of the high frequency pixels in ground truth high-resolution images. After that, the pixel-level loss function takes effect only at these found positions to minimize the difference between reconstructed high-resolution images and ground truth high-resolution images. Compared with other state-of-the-art super-resolution methods, the experiment results show that our method can recover more high frequency contents in high-resolution image reconstructing, and better improve the classification accuracy after low-resolution image preprocessing.
Derivation of Geometrically and Semantically Annotated UAV Datasets at Large Scales from 3D City Models

While in high demand for the development of deep learning approaches, extensive datasets of annotated unmanned aerial vehicle (UAV) imagery are still scarce today. Manual annotation, however, is time-consuming and, thus, has limited the potential for creating large-scale datasets. We tackle this challenge by presenting a procedure for the automatic creation of simulated UAV image sequences in urban areas and pixel-level annotations from publicly available data sources. We synthesize photo-realistic UAV imagery from Google Earth Studio and derive annotations from an open CityGML model that not only provides geometric but also semantic information. The first dataset we exemplarily created using our approach contains 144 000 images of Berlin, Germany, with four types of annotations, namely semantic labels as well as depth, surface normals, and edge maps. Based on this specific case, we demonstrate the entire pipeline and give a comprehensive overview of technical obstacles and solutions. In experiments, we evaluated the quality of our dataset and its potential application in monocular depth estimation and semantic segmentation using deep learning methods. To facilitate the creation of further datasets, we provide our source code along with the produced dataset at https://github.com/sian1995/largescaleuavdataset.
Real-Time Driver Drowsiness Detection using Facial Action Units

This paper presents a two-stage, vision-based pipeline for the real-time detection of driver drowsiness using Facial Action Units (FAUs). FAUs capture movements in groups of muscles in the face like widening of the eyes or dropping of the jaw. The first stage of the pipeline employs a Convolutional Neural Network (CNN) trained to detect FAUs. The output of this network's penultimate layer serves as an image embedding that captures features relevant to FAU detection. These embeddings are then used to predict drowsiness using an Extreme Gradient Boosting (XGBoost) classifier. A separate X G Boost model is trained for each user of the system so that behavior specific to each user can be modeled into the drowsiness classifier. We show that user-specific classifiers require very little data and short training time to yield high prediction accuracies in real-time.
State Estimation for HALE UAVs With Deep-Learning-Aided Virtual AOA/SSA Sensors for Analytical Redundancy

High-altitudelong-endurance (HALE) unmanned aerial vehicles (UAVs) are employed in a variety of fields because of their ability to fly for a long time at high altitudes, even in the stratosphere. Two paramount concerns exist: enhancing their safety during long-term flight and reducing their weight as much as possible to increase their energy efficiency based on analytical redundancy approaches. In this letter, a novel deep-learning-aided navigation filter is proposed, which consists of two parts: an end-to-end mapping-based synthetic sensor measurement model that utilizes long short-term memory (LSTM) networks to estimate the angle of attack (AOA) and sideslip angle (SSA) and an unscented Kalman filter for state estimation. Our proposed method can not only reduce the weight of HALE UAVs but also ensure their safety by means of an analytical redundancy approach. In contrast to conventional approaches, our LSTM-based method achieves better estimation by virtue of its nonlinear mapping capability.
EDD-Net: An Efficient Defect Detection Network

As the most commonly used communication tool, the mobile phone has become an indispensable part of our daily life. The surface of the mobile phone as the main window of human-phone interaction directly affects the user experience. It is necessary to detect surface defects on the production line in order to ensure the high quality of the mobile phone. However, the existing mobile phone surface defect detection is mainly done manually. Currently, there are few automatic defect detection methods to replace human eyes. How to quickly and accurately detect the surface defects of the mobile phone is an urgent problem to be solved. Hence, an efficient defect detection network (EDD-Net) is proposed. Firstly, EfficientNet is used as the backbone network. Then, according to the small-scale of mobile phone surface defects, a feature pyramid module named GCSA-BiFPN is proposed to obtain more discriminative features. Finally, the box/class prediction network is used to achieve effective defect detection. We also build a mobile phone surface oil stain defect (MPSOSD) dataset to alleviate the lack of dataset in this field. The performance on the relevant datasets shows that the proposed network is effective and has practical significance for industrial production.
Extracting and Interpreting Unknown Factors with Classifier for Foot Strike Types in Running

This paper proposes a method that can classify foot strike types using a deep learning model and can extract unknown factors, which enables to evaluate running motions without being influenced by biases of sports experts, using the contribution degree of input values (CDIV). Accelerometers are attached to the runner's body, and when the runner runs, a fixed camera observes the runner and acquires a video sequence synchronously with the accelerometers. To train a deep learning model for classifying foot strikes, we annotate foot strike acceleration data for RFS (Rearfoot strike) or non-RFS objectively by watching the video. To interpret the unknown factors extracted from the learned model, we calculate two CDIVs: the contributions of the resampling time and the accelerometer value to the output (foot strike type). Experiments on classifying unknown runners' foot strikes were conducted. As a common result to sport science, it is confirmed that the CDIVs contribute highly at the time of the right foot strike, and the sensor values corresponding to the right and left tibias contribute highly to classifying the foot strikes. Experimental results show the right tibia is important for classifying foot strikes. This is because many of the training data represent difference between the two foot strikes in the right tibia. As a conclusion, our proposed method could extract unknown factors from the classifier and could interpret the factors that contain similar knowledge to the prior knowledge of experts, as well as new findings that are not included in conventional knowledge.
Conditional Multi-Task learning for Plant Disease Identification

Several recent studies have proposed an automatic plant disease identification system based on deep learning. However, these approaches are generally based on learned classification models with target classes of joint host species-disease pairs that may not allow optimal use of the available information. This is because these approaches require distinguishing between similar host species or diseases, and more importantly, they have limited scalability due to the size of a network gradually increases as new classes are added, despite the fact that information on host species or diseases is already available. This constraint is all the more important as it can be difficult to collect/establish a specific list of all diseases for each host plant species in an actual application. In this paper, we address the problems by proposing a new conditional multi-task learning (CMTL) approach which allows the distribution of host species and disease characteristics learned simultaneously with a conditional link between them. This conditioning is formed in such a way that the knowledge to infer the prediction of one concept (the diseases) depends on the other concept (the host species), which corresponds to the way plant pathologists used to infer the diseases of the host species. We show that our approach can improve the performance of plant disease identification compared to the usual species-disease pair modeling in the previous studies. Meanwhile, we also compose a new dataset on plant disease identification that could serve as an important benchmark in this field.
Minority Class Oriented Active Learning for Imbalanced Datasets

Active learning aims to optimize the dataset annotation process when resources are constrained. Most existing methods are designed for balanced datasets. Their practical applicability is limited by the fact that a majority of real-life datasets are actually imbalanced. Here, we introduce a new active learning method which is designed for imbalanced datasets. It favors samples likely to be in minority classes so as to reduce the imbalance of the labeled subset and create a better representation for these classes. We also compare two training schemes for active learning: (1) the one commonly deployed in deep active learning using model fine tuning for each iteration and (2) a scheme which is inspired by transfer learning and exploits generic pre-trained models and train shallow classifiers for each iteration. Evaluation is run with three imbalanced image datasets. Results show that the proposed active learning method outperforms competitive baselines. Equally interesting, they also indicate that the transfer learning training scheme outperforms model fine tuning if features are transferable from the generic dataset to the unlabeled one. This last result is surprising and should encourage the community to explore the design of deep active learning methods.
Objectivity and Subjectivity in Games: Understanding Engagement and Addiction Mechanism

The gaming industry had rapidly been expanding globally, where it encompasses more than the purpose of recreational and increasingly becomes more immersive and engaging, and potentially leads to pathological gaming behaviors that lead to addiction. Such experience of engagement and addiction involves understanding the fundamental functions of the human mind’s dynamic state. This study uncovers the mind’s underlying physics via the analogy of motion (i.e., mass, velocity, etc.) using games as the source of information. This study also conjectures that the law of conservation in mind occurred in games where momentum and energy were conserved over time, where game-playing experience relative to the gambling psychology and perceptive force were identified from the objective and subjective perspectives. It was found that momentum conservation provides new engagement measures while energy conservation, considering several other factors, provides the necessary components of understanding the addiction mechanism in the game-playing context. This measurement is examined in various domains, such as popular boards and sports games, and public gambling, where its effectiveness is determined.
Understanding the Role of Defects in Silicon Nitride-Based Resistive Switching Memories Through Oxygen Doping

Resistive memories are promising candidates for replacing current nonvolatile memories and realize storage class memories. Moreover, they have memristive properties, with many discrete resistance levels and implement artificial synapses. The last years researchers have demonstrated RRAM chips used as accelerators in computing, following the new in-memory and neuromorphic computational approaches. Many different metal oxides have been used as resistance switching materials in MIM structures. Understanding of the switching mechanism is very critical for the modeling and the use of memristors in different applications. Here, we demonstrate the bipolar resistance switching of silicon nitride thin films using heavily doped Si and Cu as bottom and top-electrodes respectively. Next, we dope nitride with oxygen in order to introduce and modify the intrinsic nitride defects. Analysis of the current-voltage characteristics reveal that under space-charge limited conditions and by setting the appropriate current compliance, the operation condition of the RRAM cells can be tuned. Furthermore, resistance change can be obtained using appropriate SET/RESET pulsing sequences allowing the use of the devices in computing acceleration application. Impedance spectroscopy measurements clarify the presence of different mechanisms during SET and RESET. We prove through a customized measurement set-up and the appropriate control software that the initial charge-storage in the intrinsic nitride traps governs the resistance change.
Extending Single Beam Lidar To Full Resolution By Fusing with Single Image Depth Estimation

Depth estimation is playing an important role in indoor and outdoor scene understanding, autonomous driving, augmented reality and many other tasks. Vehicles and robotics are able to use active illumination sensors such as LIDAR to receive high precision depth estimation. However, high-resolution LIDARs are usually too expensive, which limits its massive production on various applications. Though single beam LIDAR enjoys the benefits of low cost, one beam depth sensing is not usually sufficient to perceive the surrounding environment in many scenarios. In this paper, we propose a deep learning based framework to explore to replicate similar or even higher performance as costly LIDARs with our designed self-supervised network and a low-cost single-beam LIDAR. After the accurate calibration with a visible camera, the single beam LIDAR can adjust the scale uncertainty of the depth map estimated by the visible camera. The adjusted depth map enjoys the benefits of high resolution and sensing accuracy as high beam LIDAR and maintains low-cost as single beam LIDAR. Thus we can achieve similar sensing effect of high beam LIDAR with more than a 30–100 times cheaper price (e.g., $80000 Velodyne HDL-64E LIDAR v.s. $2000 SICK TIM-781 2D LIDAR and normal camera). The proposed approach is verified on our collected dataset and public dataset with superior depth-sensing performance.
Dimensionality Reduction for Data Visualization and Linear Classification, and the Trade-off between Robustness and Classification Accuracy

This paper has three intertwined goals. The first is to introduce a new similarity measure for scatter plots. It uses Delaunay triangulations to compare two scatter plots regarding their relative positioning of clusters. The second is to apply this measure for the robustness assessment of a recent deep neural network (DNN) approach to dimensionality reduction (DR) for data visualization. It uses a nonlinear generalization of Fisher's linear discriminant analysis (LDA) as the encoder network of a deep autoencoder (DAE). The DAE's decoder network acts as a regularizer. The third goal is to look at different variants of the DNN: ones that promise robustness and ones that promise high classification accuracies. This is to study the trade-off between these two objectives – our results support the recent claim that robustness may be at odds with accuracy; however, results that are balanced regarding both objectives are achievable. We see a restricted Boltzmann machine (RBM) pretraining and the DAE based regularization as important building blocks for achieving balanced results. As a means of assessing the robustness of DR methods, we propose a measure that is based on our similarity measure for scatter plots. The robustness measure comes with a superimposition view of Delaunay triangulations that enables a fast comparison of results from multiple DR methods.
A Novel Disaster Image Data-set and Characteristics Analysis using Attention Model

The advancement of deep learning technology has enabled us to develop systems that outperform any other classification technique. However, success of any empirical system depends on the quality and diversity of the data available to train the proposed system. In this research, we have carefully accumulated a relatively challenging dataset that contains images collected from various sources for three different disasters: fire, water and land. Besides this, we have also collected images for various damaged infrastructure due to natural or man made calamities and damaged human due to war or accidents. We have also accumulated image data for a class named non-damage that contains images with no such disaster or sign of damage in them. There are 13,720 manually annotated images in this dataset, each image is annotated by three individuals. We are also providing discriminating image class information annotated manually with bounding box for a set of 200 test images. Images are collected from different news portals, social media, and standard datasets made available by other researchers. A three layer attention model (TLAM) is trained and average five fold validation accuracy of 95.88% is achieved. Moreover, on the 200 unseen test images this accuracy is 96.48%. We also generate and compare attention maps for these test images to determine the characteristics of the trained attention model.
Disentangled Representation Learning for Controllable Image Synthesis: An Information-Theoretic Perspective

In this paper, we look into the problem of disentangled representation learning and controllable image synthesis in a deep generative model. We develop an encoder-decoder architecture for a variant of the Variational Auto-Encoder (VAE) with two latent codes $z_{1}$ and $z_{2}$. Our framework uses $z_{2}$ to capture specified factors of variation while $z_{1}$ captures the complementary factors of variation. To this end, we analyze the learning problem from the perspective of multivariate mutual information, derive optimizable lower bounds of the conditional mutual information in the image synthesis processes and incorporate them into the training objective. We validate our method empirically on the Color MNIST dataset and the CelebA dataset by showing controllable image syntheses. Our proposed paradigm is simple yet effective and is applicable to many situations, including those where there is not an explicit factorization of features available, or where the features are non-categorical.
Can data placement be effective for Neural Networks classification tasks? Introducing the Orthogonal Loss

Traditionally, a Neural Network classification training loss function follows the same principle: minimizing the distance between samples that belong to the same class, while maximizing the distance to the other classes. There are no restrictions on the spatial placement of deep features (last layer input). This paper addresses this issue when dealing with Neural Networks, providing a set of loss functions that are able to train a classifier by forcing the deep features to be projected over a predefined orthogonal basis. Experimental results shows that these ‘data placement’ functions can overcome the training accuracy provided by the classic cross-entropy loss function.
An Effective Hybrid Atrous Convolutional Network for Pixel-Level Crack Detection

Automated pixel-level crack detection is one of the essential tasks in the field of defect inspection. Deep convolutional neural networks, typically using encoder–decoder architectures, have been successfully applied to many crack detection scenes in recent works. However, encoder–decoder networks commonly rely on downsampling and upsampling operations and have a large number of parameters, which may influence the accuracy of crack prediction due to the cracks usually have long, narrow sizes, and the labeled training set is always limited. To address these issues, we propose a simple and effective hybrid atrous convolutional network (HACNet). HACNet maintains the same spatial resolution throughout the whole architecture. It can retain more spatial precision in prediction. HACNet uses atrous convolutions with the proper dilation rates to enlarge the receptive field and a hybrid approach connecting these convolutions to aggregate multiscale features. The resulting architecture can achieve accurate segmentation with relatively few parameters. Evaluations on the public CFD data set, CrackTree206 data set, Deepcrack data set (DCD), and Yang et al. Crack data set (YCD) demonstrate that our method can obtain promising results, compared with other recent approaches. Evaluation on self-collected images and SDNET2018 data set illustrates the good potential of HACNet for practical applications.
A Programmable-Metasurface-Based TDMA Fast Beam Switching Communication System at 28 GHz

A programmable metasurface (PM) is a promising active antenna array approach for beamforming purposes. In comparison with a phased array, our interested PM achieves beamforming by electronically control the states of Pin diodes instead of phase shifters, which highly reduces the system price and power consumption. In this letter, a novel wireless communication system with an ultrafast beam steering ability using a PM at 28 GHz in a time-division multiple access (TDMA) scenario is constructed. Realization of the entire system dealing with technical specifications is introduced in detail. System performances with different beam switching times, modulation schemes, and frame structures are analyzed. As a conclusion, we prove that a PM system is a possible candidate for fast beam switching with low cost and complexity.
Prediction of Obstructive Coronary Artery Disease from Myocardial Perfusion Scintigraphy using Deep Neural Networks

For diagnosis and risk assessment in patients with stable ischemic heart disease, myocardial perfusion scintigraphy is one of the most common cardiological examinations performed today. There are however many motivations for why an artificial intelligence algorithm would provide useful input to this task. For example to reduce the subjectiveness and save time for the nuclear medicine physicians working with this time consuming task. In this work we have developed a deep learning algorithm for multi-label classification based on a convolutional neural network to estimate the probability of obstructive coronary artery disease in the left anterior artery, left circumflex artery and right coronary artery. The prediction is based on data from myocardial perfusion scintigraphy studies conducted in a dedicated Cadmium-Zinc-Telluride cardio camera (D-SPECT Spectrum Dynamics). Data from 588 patients was available, with stress images in both upright and supine position, as well as a number of auxiliary parameters such as angina symptoms and age. The data was used to train and evaluate the algorithm using 5-fold cross-validation. We achieve state-of-the-art results for this task with an area under the receiver operating characteristics curve of 0.89 as average on per-vessel level and 0.95 on per-patient level.