Reviewerly/exHarmony huggingface dataset:
Fast Deep Learning Training through Intelligently Freezing Layers

As the complexity of deep learning models grows, the difficulty and time of training them also increases. Depending on the task, the complexity of the model, and the hardware resources available, the amount of training time could take hours, weeks or even months. To decrease the training time, we propose a method to intelligently freeze layers during the training process. Our method involves designing a formula to calculate normalized gradient differences for all layers with weights in the model, and then use the calculated values to decide how many layers should be frozen. We implemented our method on top of stochastic gradient descent, and performed experiments on standard image classification dataset CIFAR-10. Results show that our method can accelerate training on VGG nets, ResNets, and DenseNets while having similar test accuracy.
TOT-Net: An Endeavor Toward Optimizing Ternary Neural Networks

High computation demands and big memory resources are the major implementation challenges of Convolutional Neural Networks (CNNs) especially for low-power and resource-limited embedded devices. Many binarized neural networks are recently proposed to address these issues. Although they have significantly decreased computation and memory footprint, they have suffered from accuracy loss especially for large datasets. In this paper, we propose TOT-Net, a ternarized neural network with [-1, 0, 1] values for both weights and activation functions that has simultaneously achieved a higher level of accuracy and less computational load. In fact, first, TOT-Net introduces a simple bitwise logic for convolution computations to reduce the cost of multiply operations. To improve the accuracy, selecting proper activation function and learning rate are influential, but also difficult. As the second contribution, we propose a novel piece-wise activation function, and optimized learning rate for different datasets. Our findings first reveal that 0.01 is a preferable learning rate for the studied datasets. Third, by using an evolutionary optimization approach, we found novel piece-wise activation functions customized for TOT-Net. According to the experimental results, TOT-Net achieves 2.15%, 8.77%, and 5.7/5.52% better accuracy compared to XNOR-Net on CIFAR-10, CIFAR-100, and ImageNet top-5/top-1 datasets, respectively.
Semi-Supervised Skin Detection by Network With Mutual Guidance

We present a new data-driven method for robust skin detection from a single human portrait image. Unlike previous methods, we incorporate human body as a weak semantic guidance into this task, considering acquiring large-scale of human labeled skin data is commonly expensive and time-consuming. To be specific, we propose a dual-task neural network for joint detection of skin and body via a semi-supervised learning strategy. The dual-task network contains a shared encoder but two decoders for skin and body separately. For each decoder, its output also serves as a guidance for its counterpart, making both decoders mutually guided. Extensive experiments were conducted to demonstrate the effectiveness of our network with mutual guidance, and experimental results show our network outperforms the state-of-the-art in skin detection.
Workshop with 335 primary school children in The Netherlands: What is needed to improve the IEQ in their classrooms?

To identify current problems in the classroom and to conceptualize design solutions by primary school children to solve these problems, 335 children from seven primary schools participated in a workshop held in the Experience room of the SenseLab, comprising of two parts. In part 1, the children were asked to think about their own classroom at school and to choose an IEQ-problem in their own classroom that they are bothered with. In part 2 of the workshop, the children were asked to imagine they are an inventor or scientist in 2040 with all resources available and to make a design for the future. The content analysis of the problems and solutions appearing in the drawings and the written text resulted in 5 themes (light, noise, temperature, air and other than IEQ) and 16 sub-themes (11 for the problems and 5 for the solutions). Noise-related problems were most frequently reported (58%), followed by temperature (53%), air (22%), and light (16%). Girls reported more problems than boys, which is possibly related to a better recollection of negative feelings towards those problems in their classrooms. 47% of the children proposed solutions related to more than one IEQ-problem. Solutions ranged from existing solutions, for example headphones to protect against noise to far-fetched solutions such as send noisy children away by means of a rocket. The outcome showed that children can be valuable contributors in co-designing ‘new’ or ‘adapted’ classroom environments.
Detecting Saturation Attacks in SDN via Machine Learning

Software Defined Networking (SDN) is a new network paradigm that facilitates network management by separating the control plane from the data plane. Studies have shown that an SDN may experience a high packet loss rate and a long delay in forwarding messages when the OpenFlow channel is overwhelmed by a saturation attack. The existing approaches have focused on the detection of saturation attacks caused by TCP-SYN flooding through periodic analysis of network traffic. However, there are two issues. First, previous approaches are incapable of detecting other types, especially unknown types, of saturation attacks. Second, they rely on predetermined time-window of network traffic and thus are unable to determine what time window of traffic data would be appropriate for effective attack detection. To tackle these problems, this paper first investigates the impact of different time-windows of OpenFlow traffic on the detection performance of three classification algorithms: the Support Vector Machine (SVM), the Naïve Bayes (NB) classifier, and the K-Nearest Neighbors (K-NN) classifier. We have built and analyzed a total of 150 models on OpenFlow traffic datasets generated from both physical and simulated SDN environments. The experiment results show that the chosen time-interval of OpenFlow traffic heavily influences the detection performance - larger time-windows may result in decreased detection performance. In addition, we were able to achieve reasonable accuracy on detection of unknown attacks by applying proper time-windows of OpenFlow traffic.
RF-DNA Fingerprint Classification of OFDM Signals Using a Rayleigh Fading Channel Model

The Internet of Things is a collection of Internet connected devices capable of interacting with the physical world and computer-based systems and is estimated to consist of 20 to 50 billion devices by the year 2020. Due to these numbers and the fact that 70% of the edge devices have no or poor encryption, there is a need for mechanisms by which to secure these devices and associated networks. Specific Emitter Identification (SEI) is one proposed mechanism for IoT security; however, performance within multipath environments has received little attention. This work presents the integration of a novel, Nelder-Mead (N-M)-based channel estimator within the Radio Frequency-Distinct Native Attributes (RF-DNA) fingerprinting process to facilitate serial number discrimination when IoT devices are operating within a multipath environment and degrading Signal-to-Noise Ratio. Percent correct classification performance is used to assess the developed RF-DNA fingerprinting process. Two additional SEI approaches are assessed to facilitate comparative analysis of the serial number discrimination of four IEEE 802.11a Wi-Fi radios using a Rayleigh fading channel. Relative to the Adaptive Compensator (A-C) and N-M SEI approaches, RF-DNA fingerprinting provides the best means for achieving reliable (better than 95%) identification of all radios within a L=2 Rayleigh fading channel at SNR ≥ 21 dB.
Iterative Deep Neural Network Quantization With Lipschitz Constraint

Network quantization offers an effective solution to deep neural network compression for practical usage. Existing network quantization methods cannot theoretically guarantee the convergence. This paper proposes a novel iterative framework for network quantization with arbitrary bit-widths. We present two Lipschitz constraint based quantization strategies, namely width-level network quantization (WLQ) and multi-level network quantization (MLQ), for high-bit and extremely low-bit (ternary) quantization, respectively. In WLQ, Lipschitz based partition is developed to divide parameters in each layer into two groups: one for quantization and the other for re-training to eliminate the quantization loss. WLQ is further extended to MLQ by introducing layer partition to suppress the quantization loss for extremely low bit-widths. The Lipschitz based partition is proven to guarantee the convergence of the quantized networks. Moreover, the proposed framework is complementary to network compression methods such as activation quantization, pruning and efficient network architectures. The proposed framework is evaluated over extensive state-of-the-art deep neural networks, i.e., AlexNet, VGG-16, GoogleNet and ResNet18. Experimental results show that the proposed framework improves the performance of tasks like classification, object detection and semantic segmentation.
Understanding what the users say in chatbots: A case study for the Vietnamese language

This paper1 presents a study on understanding what the users say in chatbot systems: the situation where users input utterances bots would hopefully (1) detect intents and (2) recognize corresponding contexts implied by utterances. This helps bots better understand what users are saying, and act upon a much wider range of actions. To this end, we propose a framework which models the first task as a classification problem and the second one as a two-layer sequence labeling problem. The framework explores deep neural networks to automatically learn useful features at both character and word levels. We apply this framework to building a chatbot in a Vietnamese e-commerce domain to help retail brands better communicate with their customers. Experimental results on four newly-built datasets demonstrate that deep neural networks could be able to outperform strong conventional machine-learning methods. In detecting intents, we achieve the best F-measure of 82.32%. In extracting contexts, the proposed method yields promising F-measures ranging from 78% to 91% depending on specific types of contexts.
OGRU: An Optimized Gated Recurrent Unit Neural Network

Abstract Due to the structural problem, the traditional neural network models are prone to problems such as gradient explosion and over-fitting, while the deep GRU neural network model has low update efficiency and poor information processing capability among multiple hidden layers. Based on this, this paper proposes an optimized gated recurrent unit(OGRU) neural network.The OGRU neural network model proposed in this paper improves information processing capability and learning efficiency by optimizing the unit structure and learning mechanism of GRU, and avoids the update gate being interfered by the current forgetting information. The experiment uses Tensorflow framework to establish prediction models for LSTM neural network, GRU neural network and OGRU neural network respectively, and compare the prediction accuracy. The results show that the OGRU model has the highest learning efficiency and better prediction accuracy.
Efficient Publicly Verifiable 2PC over a Blockchain with Applications to Financially-Secure Computations

We present a new efficient two-party secure computation protocol which allows the honest party to catch dishonest behavior (if any) with a publicly-verifiable, non-repudiable proof without sacrificing the honest party's secret. Comparing to the best existing protocol of its kind, ours requires a substantially simpler judge algorithm and is able to process circuit evaluator's input-wires two orders of magnitude faster. Further, we propose an automated, decentralized judge implemented as a blockchain smart-contract. As a killer application of combining our two-party PVC protocol with our decentralized judge, we proposed the concept of financially-secure computation, which can be useful in many practical scenarios where it suffices to consider rational adversaries. We experimentally evaluated our prototype implementation, demonstrated the 2PC protocol is highly efficient and the judge is very affordable to protect users against rational attackers.
Fault location estimation for series-compensated double-circuit transmission line using EWT and weighted RVFLN

In this paper, empirical Wavelet transform (EWT), Hilbert transform (HT) and weighted random vector functional link network (WRVFLN) are integrated for fault detection, classification, and location estimation in a series capacitor compensated double circuit transmission line (SCCDCTL). The full cycle current signals from the point of fault inception are decomposed using EWT to extract three band-limited modes (BLMs). The four efficacious instantaneous features namely energy, Shannon entropy, the standard deviation of the magnitude, and crest factor are computed from the Hilbert transformed array of the BLMs to construct the feature vector. A diagonal matrix W is computed from the zero sequence current of original six current signals as a weighting factor to categorize the ground fault accurately. Numerous faults are generated with a wide variation of the system conditions such as fault resistance, fault inception angle, fault distance, percentage compensation level, source impedance, line parameters, load angle, and inter-circuit fault in MATLAB/Simulink environments. An efficient WRVFLN computational intelligence technique is proposed to recognize and estimate the location of the faults by taking the extracted suitable feature vector with weight factor as an input. The performances of WRVFLN are compared with the recently developed advanced classifiers such as least-square support vector machine (LSSVM) and extreme learning machine (ELM) in the MATLAB interface. The lesser computational complexity, faster learning speed, superior classification accuracy, accurate fault location estimation, and short event detection time prove that the proposed EWTHT–WRVFLN method can be implemented in the real power system for online fault diagnosis. Finally, the developed system architecture is implemented on the reconfigurable digital field programmable gate array (FPGA) in ISE design suite 14.5 environments to verify the cogency of the proposed method in real-time. The feasibility of the proposed method is tested and validated by using the fast FPGA digital circuitry in a loop.
Fast initial value estimation in digital image correlation for large rotation measurement

Digital image correlation (DIC) is widely used in macroscopic and mesoscopic mechanical tests because of its advantages of non-contact, high precision, full-field measurement and simple experimental equipment. The application of various nonlinear optimisation algorithms greatly reduces the computation time of the DIC iteration process. Thus, efficiently obtaining a reliable initial value is crucial. Particularly, in cases when the surface of the test object is substantially rotated or the deformation involves a large rotation, the initial value estimation can have a major influence on the execution speed of the algorithm. Some scholars have proposed initial value estimation methods for large-rotation objects, but they all sacrificed the speed of calculation. This study deals with improving the efficiency of the initial value estimation algorithm for large-rotation objects from two aspects. Firstly, we decomposed deformed and reference images into multiresolution layers to create wavelet pyramids, and the correlation coefficients between the two compared images were calculated at the low-resolution layer. The multiresolution processing of the image data provides an efficient method for registering large image data sets because the full-size data sets does not require matching. Secondly, we propose a local ring pattern (LRP), which is invariant to object rotation, to convert the 2D template into a 1D grey value sequence for calculating the correlation coefficients. The advantages of the LRP feature include the characterisation of its rotation invariance and the reduction of computational complexity. Experimental results indicate that the proposed method can be used to estimate an initial value where reference and deformed subsets are related by translational and rotational motions and the speed of the proposed method is higher than that of the traditional method.
The application of day and night time ASTER satellite imagery for geothermal and mineral mapping in East Africa

There is considerable interest in optimizing geothermal exploration techniques via the mapping of alteration and evaporate mineralisation, as well as of thermal emissions associated with geothermally active areas on the Earth's surface. Optical and thermal satellite sensor technologies, improvements in processing algorithms and the means for large scale (e.g. 1:250,000) spatial data distribution are required for detecting both these attributes. The extensive visible, -near, -shortwave and thermal infrared (VNIR-SWIR-TIR) data archive acquired by the multi-spectral Advanced Spaceborne Thermal Emission Reflectance Radiometer (ASTER) provides a rich source of geoscience related imagery for geothermal exploration. Examples of generating large scale mosaicked ASTER imagery to provide province to continental mineral mapping have been undertaken in areas including such as Australia, western USA, Namibia and Zagros Mountains Iran. In addition, ASTER's thermal infrared imagery also provides night time land surface temperature (LST) estimates relevant for detecting possible geothermal related anomalies. This study outlines existing methods for the application of ASTER data for geothermal exploration in East Africa. The study area encompasses a section of the East African Rift System across the Tanzanian and Kenyan border. The area includes rugged volcanic terrain which has had geological mapping of limited coverage at detailed scales, from various heritages and mapping agencies. This study summarizes the technology, the processing methodology and initial results in applying ASTER imagery for such compositional and thermal anomaly mapping related to geothermal activity. Fields observations have been used from the geothermal springs of Lake Natron, Tanzania, and compared with ASTER derived spectral composition and land surface temperature results. Published geothermal fields within the Kenyan portion of the study area have also been incorporated into this study.
De-smokeGCN: Generative Cooperative Networks for Joint Surgical Smoke Detection and Removal

Surgical smoke removal algorithms can improve the quality of intra-operative imaging and reduce hazards in image-guided surgery, a highly desirable post-process for many clinical applications. These algorithms also enable effective computer vision tasks for future robotic surgery. In this article, we present a new unsupervised learning framework for high-quality pixel-wise smoke detection and removal. One of the well recognized grand challenges in using convolutional neural networks (CNNs) for medical image processing is to obtain intra-operative medical imaging datasets for network training and validation, but availability and quality of these datasets are scarce. Our novel training framework does not require ground-truth image pairs. Instead, it learns purely from computer-generated simulation images. This approach opens up new avenues and bridges a substantial gap between conventional non-learning based methods and which requiring prior knowledge gained from extensive training datasets. Inspired by the Generative Adversarial Network (GAN), we have developed a novel generative-collaborative learning scheme that decomposes the de-smoke process into two separate tasks: smoke detection and smoke removal. The detection network is used as prior knowledge, and also as a loss function to maximize its support for training of the smoke removal network. Quantitative and qualitative studies show that the proposed training framework outperforms the state-of-the-art de-smoking approaches including the latest GAN framework (such as PIX2PIX). Although trained on synthetic images, experimental results on clinical images have proved the effectiveness of the proposed network for detecting and removing surgical smoke on both simulated and real-world laparoscopic images.
Conceptualizing Blockchain based Energy Market for Self Sustainable Community

With a spectacular boom in blockchain technologies at various verticals and with the increase in renewable energy penetration, these could solve one of the major issues in Power Systems - Energy Demand Management by the utilities. Blockchain being a distributed architecture, it could fit into the deregulated energy market to enable energy trading in a self sustainable energy community and manage the power demands among the community members. As proof of concept, the blockchain technology is introduced in a smart community where consumers take part in energy trading among themselves as well as with the utility. In this paper, three use cases for P2P energy trading in a private blockchain network is discussed. The blockchain architecture is developed using the Hyperledger framework and the smart contracts are defined in the chaincode. The performance of the blockchain network on resource constrained IoT system based on Raspberry Pi is also evaluated.
Scaling the Cascades: Interconnect-Aware FPGA Implementation of Machine Learning Problems

DSP48s, BRAMs and URAMs in the Xilinx Ultra-scale+ family support dedicated cascade interconnect for high frequency, nearest-neighbor data movement using hard wiring resources. We demonstrate how to leverage these interconnect structures to effectively support data movement requirements of dense machine learning (ML) workloads at URAM-limited 650MHz frequency (714MHz reported by Vivado). We refor-mulate convolution and matrix-vector multiplication operations to make effective use of cascade interconnect (1) in DSP48s for supporting the common multiply-accumulate chains, and (2) in BRAMs, and URAMs to exploit the data movement and reuse patterns of ML workloads. The use of these dedicated cascade interconnect are an alternative to Versal AI cores that throw away FPGA flexibility in favor of rigid ASIC components with unproven long-term value. Our 650MHz operation on the Xilinx VU37P UltraScale+ FPGA is competitive with the 720 MHz state-of-the-art Xilinx SuperTile design. We use 100% URAM288s, 95% DSP48s, and 77% BRAM in contrast to the 100% URAM288s, 56% DSP48, and 40% BRAM usage of the Xilinx SuperTile array. As a result, we deliver a ?7× superior GoogLeNet inference latency while sacrificing 30% of inference throughput than their design. For MLPerf benchmarks we note inference latencies between 2?s-1.54 ms with corresponding throughputs between 645-456K inf/s.