Reviewerly/exHarmony huggingface dataset:
Dynamic Probabilistic Pruning: Training sparse networks based on stochastic and dynamic masking

Deep Learning (DL) models are known to be heavily over-parametrized, resulting in a large memory footprint and power consumption. This hampers the use of such models in hardware-constrained edge technologies such as wearables and mobile devices. Model compression during training can be achieved by promoting sparse network structures both through weight regularization and by leveraging dynamic pruning methods. State-of-the-art pruning methods are however mostly magnitude-based which impedes their use in e.g. binary settings. Importantly, most of the pruning methods do not provide a structural sparsity, resulting in an inefficient memory allocation and access for hardware implementations. In this paper, we propose a novel dynamic pruning solution that we term Dynamic Probabilistic Pruning (DPP). DPP leverages Gumbel top-K sampling to select subsets of weights during training, which enables exploring which weights are most relevant. Our approach allows for setting an explicit per-neuron layer-wise sparsity level and structural pruning across weights and feature maps, without relying on weight magnitude heuristics. Relevantly, our method generates a hardware-oriented structural sparsity for fully-connected and convolutional layers that facilitates memory allocation and access, in contrast with conventional unstructured pruning. We show that DPP achieves competitive sparsity levels and classification accuracy on MNIST and CIFAR-10, CIFAR-100 datasets compared to a state-of-the-art baseline for various DL architectures, while respecting per-neuron sparsity constraints.
Verification of Neural Networks for Safety Critical Applications

In recent years, Neural Networks (NNs) have been widely adopted in engineering automated driving systems with examples in perception, decision-making, or even end-to-end scenarios. As these systems are safety-critical in nature, they are too complex and hard to verify. For using neural networks in safety-critical domains, it is important to know if a decision made by a neural network is supported by prior similarities in the training process. Verifying a trained neural network is to measure the extent of the decisions made by the neural network, which are based on prior similarities in the training process. In this paper, we propose a runtime monitoring that can measure the reliability of the neural network trained to classify a new input based on prior similarities in the training set. In the training process, the runtime monitor stores the values of the neurons of certain layers, which represent the neurons activation pattern for each example in the training data. We use the Binary Decision Diagrams (BDDs) formal technique to store the neuron activation patterns in binary form. In the inference process, a classification decision measured by a hamming distance is made to any new input by examining if the runtime monitor contains a similar neurons activation pattern. If the runtime monitor does not contain any similar activation pattern, it generates a warning that the decision is not based on prior similarities in the training data. Unlike previous work, we monitored more layers to allow for more neurons activation pattern of each input. We demonstrate our approach using the MNIST benchmark set. Our experimental results show that by adjusting the hamming distance, 75.63% of the misclassified labels are unseen activation patterns, which are not similar to any stored activation patterns from the training time.
Anonymous Asynchronous Ratchet Tree Protocol for Group Messaging

Signal is the first application that applies the double ratchet for its end-to-end encryption protocol. The core of the double ratchet protocol is then applied in WhatsApp, the most popular messaging application around the world. Asynchronous Ratchet Tree (ART) is extended from ratchet and Diffie-Hellman tree. It is the first group protocol that applies Forward Secrecy (FS) with Post-Compromised Security (PCS). However, it does not consider protecting the privacy of user identity. Therefore, it makes sense to provide anonymous features in the conditions of FS and PCS. In this paper, the concepts of Internal Group Anonymity (IGA) and External Group Anonymity (EGA) are formalized. On the basis of IGA and EGA, we develop the “Anonymous Asynchronous Ratchet Tree (AART)” to realize anonymity while preserving FS and PCS. Then, we prove that our AART meets the requirements of IGA and EGA as well as FS and PCS. Finally, the performance and related issues of AART are discussed.
Design and Accomplishment of Non-intrusive Web Camera SCADA in Power Station

Considering the high safety hazard and high labor cost of manual patrol inspection and the immature technology of power station intelligent robot at the present stage as well as the more stringent operating conditions, this paper puts forward the meter identification monitoring using non-intrusive web camera SCADA. The designed system relies on monitoring and wireless network implementation to read and sort out the effective information of the inspected equipment of the power station, generate information reports, reduce the workload and risk coefficient of manual inspection, facilitate relevant personnel to judge the condition of the equipment through the generated reports, and take quick countermeasures when necessary. Finally, a prototype demonstration system based on network is designed and built according to the background of power station.
Real-Time News Grouping: Detecting the Same-Content News on Turkish News Stream

The increasing number of news sources makes analysis of the news difficult and increase the need for automated systems. This paper presents a system that clusters news with similar content in real time. The system uses the Apache Solr database to capture texts from the news source and its MoreLikeThis (MLT) search component to extract the 5 most similar news from thousands of previously recorded news. The new news will be included in the cluster with the most similar of the 5 news obtained by pre-filtering. Therefore, the main problem sought in this study is finding the news that most closely resembles the new news. For this, a 2-step approach has been proposed. The majority of the news published in different sources in the media is created by the same or broadening/shortening of the same text. For this reason, the ‘citation rate’ was calculated primarily among news pairs. If there are news pairs that exceed the citation threshold, the pair with the highest citation rate is included in the same cluster. Otherwise, the numerical representations of the texts at different levels were used in order to determine the similarity semantically. The results of the study show that the proposed 2-stage approach reduces the sensitivity of embeddings at different levels to text lengths. Thus, it achieved up to 7.6% improvement compared to clustering approach only with embeddings. The system proposed in these study has a structure that can be used in real life applications in terms of real-time clustering with a high F-score rate of over 90%.
Practice of Education Program for Necessary Engineering Skill in Product Development

Many university, technical high school and technical college have been working on extracurricular activities in engineering fields. One of these activities, electric car project, as a good opportunity to experience of engineering design and manufacturing process. Kumamoto University has participated in the race as activity named ECR project. The student project was started up in 2008 for the purpose of enlightenment of engineering design and manufacturing activity. At the starting of project, machine was designed by graduate students because it is difficult for undergraduates. On the other hand, teacher and technical staff supported the team as educational program development project of active learning. The program was named “education program for necessary engineering skill in product development”, and the educational effect was confirmed by good performance in the race by undergraduate students as subject target of the program.
Dynamic Neural Network to Enable Run-Time Trade-off between Accuracy and Latency

To deploy powerful deep neural network (DNN) into smart, but resource limited IoT devices, many prior works have been proposed to compress DNN to reduce the network size and computation complexity with negligible accuracy degradation, such as weight quantization, network pruning, convolution decomposition, etc. However, by utilizing conventional DNN compression methods, a smaller, but fixed, network is generated from a relative large background model to achieve resource limited hardware acceleration. However, such optimization lacks the ability to adjust its structure in real-time to adapt for a dynamic computing hardware resource allocation and workloads. In this paper, we mainly review our two prior works [13, 15] to tackle this challenge, discussing how to construct a dynamic DNN by means of either uniform or non-uniform sub-nets generation methods. Moreover, to generate multiple nonuniform sub-nets, [15] needs to fully retrain the background model for each sub-net individually, named as multi-path method. To reduce the training cost, in this work, we further propose a single-path sub-nets generation method that can sample multiple sub-nets in different epochs within one training round. The constructed dynamic DNN, consisting of multiple sub-nets, provides the ability to run-time trade-off the inference accuracy and latency according to hardware resources and environment requirements. In the end, we study the the dynamic DNNs with different sub-nets generation methods on both CIFAR-10 and ImageNet dataset. We also present the run-time tuning of accuracy and latency on both GPU and CPU.
Founder of Nobelistics - Professor V. M. Tyutyunnik

The purpose of the study is to shed light on the periods marking formation of academic career of the professor V. M. Tyutyunnik, founder of Nobelistics. Close attention is paid to the work of the LLC “International Information Nobel Centre” (IINC), which has been instrumental in the professor’s advancement in the field of science studies, informatics and Nobelistics. Scientific novelty of the research lies in describing to the fullest extent the IINC’s activities and V. M. Tyutyunnik’s biography considered in connection with sociocultural and scientific problems. Relevant research findings amount to providing more comprehensive understanding of genesis of the academic activity undertaken by the IINC and the head of the organisation and characterising periods of the considered activity, which allows filling in a number of gaps in history of the International Nobel Movement and creation of Nobelistics as a science.
Navigation in desktop 3D games with gamepads

Common approaches for gamepad-based navigation in desktop 3D games are often not suited for inexperienced players. In particular, novices have difficulties in simultaneously controlling movement and viewing direction in a virtual 3D scene. Here, we compare the currently predominant 3D navigation approach with an alternative approach that is inspired by early 3D games. 20 test users with little or no 3D gaming experience used both approaches to walk along a test course in a virtual 3D scene containing different obstacles. Using the alternative navigation approach, the users finished the course faster and with fewer errors than with the predominant navigation approach. In addition, the pragmatic and the hedonic quality of the alternative approach were rated higher. As a conclusion, the presented alternative approach should be given consideration when designing 3D (serious) games or similar applications for inexperienced players, not only for gamepad-based navigation, but also for comparable interfaces like the typical combination of keyboard and mouse control.
Automated Thalamus Segmentation in MR Images Using Convolutional Networks

Thalamus segmentation in Magnetic Resonance Image is a challenging task because of the high inter-patient anatomical variability in both shape and size between patients. In this work, we proposed a fast and robust segmentation of thalamus using convolutional neural networks. First, cropped image which included the thalamus are extracted. Convolutional neural network (CNN) is then trained to segment the whole thalamus region. Intensity constrained model is constructed to obtain the final segmentation. Our method performs better than other state-of-the-art methods. The experiment results show that our approach achieves the highest dice scores than other methods.
Fixed Point Attack on Lin et al.’s Modified Hash Function Scheme based on SMALLPRESENT-[8] Algorithm

Lin et al.'s scheme is a hash function Message Authentication Codes (MAC) block cipher based scheme that's composed of the compression function. Fixed point messages have been found on SMALLPRESENT-[s] algorithm. The vulnerability of block cipher algorithm against fixed point attacks can affect the vulnerability of block cipher based hash function schemes. This paper applies fixed point attack against Lin et al.'s modified scheme based on SMALLPRESENT-[8] algorithm. Fixed point attack was done using fixed point message from SMALLPRESENT-[8] algorithm which used as Initial Value (IV) on the scheme branch. The attack result shows that eight fixed point messages are successfully discovered on the B <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> branch. The fixed point messages discovery on B <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> and B <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> branches form 18 fixed point messages on Lin et al.'s modified scheme with different IVs and keys. The discovery of fixed point messages shows that Lin et al.'s modified scheme is vulnerable to fixed point attack.
Power Enterprise Asset Estimation Algorithm Based on Improved GoogLeNet

In order to promote the reform of power market, it is necessary to accurately evaluate the data assets of power enterprises. Considering the complex and massive power data, a improved GoogLeNet deep learning algorithm for power enterprises asset evaluation is proposed. According to the characteristics of input data, this paper uses two Inception modules as feature extraction network, which are obtained by improving the Inception of GoogLeNet. And then, the global average pooling is replaced by the full connection layer to gets the evaluation value. This paper uses a large number of convolution kernels of 1×2 and 2×1 to lower parameters and establishes more nonlinear transformations, which can enhance the learning ability of the network. Finally, the experiment proves that the algorithm in this paper can realize the accurate evaluation for the assets of power enterprises.
Analysis of Unintelligible Speech for MLLR and MAP-Based Speaker Adaptation

Speech Recognition is the process of translating human voice into textual form, which in turn drives many applications including HCI (Human Computer Interaction). A recognizer uses the acoustic model to define rules for mapping sound signals to phonemes. This article brings out a combined method of applying Maximum Likelihood Linear Regression (MLLR) and Maximum A Posteriori (MAP) techniques to the acoustic model of a generic speech recognizer, so that it can accept data of people with speech impairments and transcribe the same. In the first phase, MLLR technique was applied to alter the acoustic model of a generic speech recognizer, with the feature vectors generated from the training data set. In the second phase, parameters of the updated model were used as informative priors to MAP adaptation. This combined algorithm produced better results than a Speaker Independent (SI) recognizer and was less effortful for training compared to a Speaker Dependent (SD) recognizer. Testing of the system was conducted with the UA-Speech Database and the combined algorithm produced improvements in recognition accuracy from 43% to 90% for medium to highly impaired speakers revealing its applicability for speakers with higher degrees of speech disorders.
Frontal Vehicle Body Damage Assessment in Surveillance Camera Images Using Deep Learning Techniques

Vehicle body damage detection from still images has received a considerable interest in traffic safety and insurance industries for the past few years. Existing vehicle body damage detection studies are typically proposed towards solving the claim leakage problem that is commonly encountered in the auto insurance claims. However, these studies use images taken from a short proximity (<3 m) to the vehicle or to the damaged region of vehicle in their decision making process. In this study, we investigate the vehicle frontal body damage detection problem in images captured from roadway surveillance cameras that are installed on fixed platforms. Proposed method utilizes ubiquitously used deep learning based object detection, feature representation and image classification methods to determine damage status for a given vehicle image. In particular, vehicle frontal region symmetry property and deep feature representations are utilized in the decision making stage as explained in the methodology section. Experimental results show that proposed method achieves 91% accuracy on a test dataset that includes a broad variety of damage types.
Intelligent Fault Diagnosis and Response Method Based on GIS and IoT in Power Grid Planning

The intelligent fault diagnoses and responses of power grid planning are very important parts of the future power grid. This paper proposes an intelligent fault diagnosis and method based on GIS map and IoT. The method mainly includes the stage of automatic fault diagnosis and data analysis based on IoT sensor data, and the process of panoramic display and response optimization based on GIS. The system includes response management and control module, business development monitoring module and power outage analysis module. It can be used for panoramic map display of current response to resource distribution. Besides, it also can support automatic topology tracing and fault analysis to copy with trajectory real-time update and optimal scheduling.
Multi-scale surface flatness evaluation method based on the idea of two-dimensional convolution

This paper proposes a multi-scale surface flatness evaluation method, where the surface flatness of objects is measured by using laser point clouds. This method is based on the idea of two-dimensional convolution, and the natural neighbor interpolation (NNI) is used to construct convolution kernels and establish an evaluation function. By changing the size of the convolution kernel, it is easier to reflect the situation of plane flatness at different scales. At the same time, according to the convolution results, non-flat regions and their sizes can be quickly extracted. Finally, the effectiveness of the evaluation method is verified by simulations and experiments.
Overparameterization and Generalization Error: Weighted Trigonometric Interpolation

Motivated by surprisingly good generalization properties of learned deep neural networks in overparameterized scenarios and by the related double descent phenomenon, this paper analyzes the relation between smoothness and low generalization error in an overparameterized linear learning problem. We study a random Fourier series model, where the task is to estimate the unknown Fourier coefficients from equidistant samples. We derive exact expressions for the generalization error of both plain and weighted least squares estimators. We show precisely how a bias toward smooth interpolants, in the form of weighted trigonometric interpolation, can lead to smaller generalization error in the overparameterized regime compared to the underparameterized regime. This provides insight into the power of overparameterization, which is common in modern machine learning.
Interinstitutional Research Team Formation Based on Bibliographic Network Embedding

This study aims at forming research teams for interinstitutional collaborations. Research institutes have their own purposes and topics of interest. Thus, supporting joint research between multiple institutes, we have to consider not only synergies between scholars but also purposes of the institutes. To solve this problem, we propose a bibliographic network embedding method that can learn characteristics of institutes, not only of each scholar. First, we compose a bibliographic network that consists of scholars, publications, venues, research projects, and institutes. Collaboration styles and research topics of institutes and scholars are extracted by mining subgraphs from the bibliographic network. Then, vector representations of network nodes are learned based on occurrences of subgraphs on the nodes and neighborhoods of the nodes. Based on the vector representations, we train multilayer perceptrons (MLP) to assess collaboration probability between scholars affiliated in different institutes. For training the MLP, we suggest three strategies: (i) considering every collaboration, (ii) focusing on interinstitutional collaborations, and (iii) focusing on collaboration outcomes. To evaluate the proposed methods, we have analyzed research collaborations of POSTECH (Pohang University of Science and Technology) and RIST (Research Institute of Industrial Science and Technology) from 2011 to 2020. Then, we conducted the research team formation for joint research of the two institutes according to two purposes: pure research and commercialization research.
Detection on Difficult Small Objects Using Layer-wise Training Strategy

Training strategy is a common condition which has considerable impact on the performance of deep learning networks. In this work, we propose a layer-wise training strategy which contains layers freezing, hard examples training and loss changing during training. Training networks using layer-wise training strategy can transfer pre-trained weights to similar tasks more efficiently. We detail about the way of freezing different layers in different epochs, which is associated with better stability. We also use small objects as training dataset, and it improves detection capabilities of networks on difficult small objects. The study offers some important insights into neural networks training. We further analyze the advantage of layer-wise training strategy and prove it efficient in experiments. Our method achieves superior accuracy over Fast R-CNN and Mask R-CNN on MS COCO dataset. We also detect photos contains difficult small objects using different models, and models trained by our strategy prove more efficient.
A Socio-Economic-Political Dimension of Employment in Sub-Saharan Africa

The paper adopts a broader perspective in studying the determinants of employment. The main objective of the study is to assess the impact of social and political variables on employment levels after accounting for macroeconomic factors. In addition, the paper adopts a gendered dimension to the labour market, as relatively more women tend to be unemployed or have vulnerable jobs in the informal sector. The Fully Modified Ordinary Least Squares is applied to analyse the drivers of employment. Data for Sub-Saharan Africa is collected from 1991 to 2016. The findings indicate that higher economic growth; more access to finance and education, low corruption as well as political stability positively affect employment. There is thus evidence to extend existing employment analysis and policy making beyond economic factors to embrace social and political constraints.