Reviewerly/exHarmony huggingface dataset:
SHAT: A Novel Asynchronous Training Algorithm That Provides Fast Model Convergence in Distributed Deep Learning

The recent unprecedented success of deep learning (DL) in various fields is underlied by its use of large-scale data and models. Training a large-scale deep neural network (DNN) model with large-scale data, however, is time-consuming. To speed up the training of massive DNN models, data-parallel distributed training based on the parameter server (PS) has been widely applied. In general, a synchronous PS-based training suffers from the synchronization overhead, especially in heterogeneous environments. To reduce the synchronization overhead, asynchronous PS-based training employs the asynchronous communication between PS and workers so that PS processes the request of each worker independently without waiting. Despite the performance improvement of asynchronous training, however, it inevitably incurs the difference among the local models of workers, where such a difference among workers may cause slower model convergence. Fro addressing this problem, in this work, we propose a novel asynchronous PS-based training algorithm, SHAT that considers (1) the scale of distributed training and (2) the heterogeneity among workers for successfully reducing the difference among the local models of workers. The extensive empirical evaluation demonstrates that (1) the model trained by SHAT converges to the higher accuracy up to 5.22% than state-of-the-art algorithms, and (2) the model convergence of SHAT is robust under various heterogeneous environments.
Analyzing OpenAPI Specifications for Security Design Issues

Modern web and mobile applications rely on an ever increasing set of services defined by their respective API (Application Programming Interface) specifications. The complexity of today&#x2019;s APIs, in terms of scale and inter-dependency, poses a challenge for security analyses as it requires much manual effort to conduct a check for design flaws. In this work, we leverage the standardized OpenAPI specification as input and propose a semi-automatic approach to infer various key information about that API specification&#x2019;s security issues. Our case study based on the OpenAPI specification of the Open Bank Project (consisting of 304 API calls and 402 data fields) shows that our approach can: 1) identify sensitive and insensitive data fields, 2) identify insecure or high-risk API calls that may leak sensitive data, and 3) calculate the exposure level of each data field and API call. In particular, we identified 31 sensitive data fields, 29 insufficiently protected API calls that access a subset of those sensitive data, and 34 high-risk API calls that may result in sensitive data exposure. Furthermore, our exposure level calculation shows that transactions-related fields generally have higher exposure level, hence requiring more scrutiny.
Deep Transfer Learning for Biology Cross-Domain Image Classification

Automatic biology image classification is essential for biodiversity conservation and ecological study. Recently, due to the record-shattering performance, deep convolutional neural networks (DCNNs) have been used more often in biology image classification. However, training DCNNs requires a large amount of labeled data, which may be difficult to collect for some organisms. This study was carried out to exploit cross-domain transfer learning for DCNNs with limited data. According to the literature, previous studies mainly focus on transferring from ImageNet to a specific domain or transferring between two closely related domains. While this study explores deep transfer learning between species from different domains and analyzes the situation when there is a huge difference between the source domain and the target domain. Inspired by the analysis of previous studies, the effect of biology cross-domain image classification in transfer learning is proposed. In this work, the multiple transfer learning scheme is designed to exploit deep transfer learning on several biology image datasets from different domains. There may be a huge difference between the source domain and the target domain, causing poor performance on transfer learning. To address this problem, multistage transfer learning is proposed by introducing an intermediate domain. The experimental results show the effectiveness of cross-domain transfer learning and the importance of data amount and validate the potential of multistage transfer learning.
A Supervised Learning Method for Improving the Generalization of Speaker Verification Systems by Learning Metrics from a Mean Teacher

The majority of recent speaker verification tasks are studied under open-set evaluation scenarios considering real-world conditions. The characteristics of these tasks imply that the generalization towards unseen speakers is a critical capability. Thus, this study aims to improve the generalization of the system for the performance enhancement of speaker verification. To achieve this goal, we propose a novel supervised-learning-method-based speaker verification system using the mean teacher framework. The mean teacher network refers to the temporal averaging of deep neural network parameters, which can produce a more accurate, stable representations than fixed weights at the end of training and is conventionally used for semi-supervised learning. Leveraging the success of the mean teacher framework in many studies, the proposed supervised learning method exploits the mean teacher network as an auxiliary model for better training of the main model, the student network. By learning the reliable intermediate representations derived from the mean teacher network as well as one-hot speaker labels, the student network is encouraged to explore more discriminative embedding spaces. The experimental results demonstrate that the proposed method relatively reduces the equal error rate by 11.61%, compared to the baseline system.
Wavelets as activation functions in Neural Networks

Traditionally, a few activation functions have been considered in neural networks, including bounded functions such as threshold, sigmoidal and hyperbolic-tangent, as well as unbounded ReLU, GELU, and Soft-plus, among other functions for deep learning, but the search for new activation functions still being an open research area. In this paper, wavelets are reconsidered as activation functions in neural networks and the performance of Gaussian family wavelets (first, second and third derivatives) are studied together with other functions available in Keras-Tensorflow. Experimental results show how the combination of these activation functions can improve the performance and supports the idea of extending the list of activation functions to wavelets which can be available in high performance platforms.
Detecting Web-Based Attacks with SHAP and Tree Ensemble Machine Learning Methods

Attacks using Uniform Resource Locators (URLs) and their JavaScript (JS) code content to perpetrate malicious activities on the Internet are rampant and continuously evolving. Methods such as blocklisting, client honeypots, domain reputation inspection, and heuristic and signature-based systems are used to detect these malicious activities. Recently, machine learning approaches have been proposed; however, challenges still exist. First, blocklist systems are easily evaded by new URLs and JS code content, obfuscation, fast-flux, cloaking, and URL shortening. Second, heuristic and signature-based systems do not generalize well to zero-day attacks. Third, the Domain Name System allows cybercriminals to easily migrate their malicious servers to hide their Internet protocol addresses behind domain names. Finally, crafting fully representative features is challenging, even for domain experts. This study proposes a feature selection and classification approach for malicious JS code content using Shapley additive explanations and tree ensemble methods. The JS code features are obtained from the Abstract Syntax Tree form of the JS code, sample JS attack codes, and association rule mining. The malicious and benign JS code datasets obtained from Hynek Petrak and the Majestic Million Service were used for performance evaluation. We compared the performance of the proposed method to those of other feature selection methods in the task of malicious JS code content detection. With a recall of 0.9989, our experimental results show that the proposed approach is a better prediction model.
Real-Time Evolutionary Music Composition Using JFUGUE and Genetic Algorithm

Computer musicology is the research from the interdisciplinary computer science and music fields to study music but not limited to these aforementioned disciplines. Composing a musical piece is a tedious work, time consuming and sometimes it is frustrating to obtain the same repetitive and unintentionally composed the copyrighted similar patterns from other artists. Therefore, automated computer music composition that is utilizing the computing to music essentially one of the methods is evolutionary music using evolutionary algorithms such as Genetic Algorithm (GA). This focal of the research project is the study of evolutionary music, and focuses on the development of an algorithmic music composer using the Java programming language. The system shall be able generated music based on a genetic algorithm, where users can define basic parameters including the instrument used to play the music, tempo, number of generated notes, and the number of measures to generate. In addition, the system also utilizes JavaFx and JFUGUE for its graphical user interface and music programming respectively. Besides generating music, the developed system includes various functionalities including saving the music into an audio file, editing the generated music, playing music, as well as loading and viewing the music previously generated and saved by the users. The music produced through this system are relatively simple and innovative, and is suited to be applied in personal projects which require such audio accompaniment. Lastly, the deliverable of this project will be a desktop application for users to compose and generate music in real-time with minimal effort.
GAN based Deep Learning Model for Detecting Damage and Displacement of Cultural Asset

The following paper proposes F-AnoGan model for anomaly detection regardless of the types of damage or displacement in wooden cultural assets image. Firstly, by training WGAN’s Generator model and Discriminator model using Normal images, we can get the value of parameter that is good at creating fake normal images. After the WGAN’s parameter is fixed, the Encoder is trained by the normal image that was used to train. Then, we input the new image data of wooden cultural assets to well-trained WGAN and Encoder and calculate Anomaly Score to discriminate and detect area where anomaly exists. The virtual dataset is established with normal data of cultural assets collected and abnormal data that contains anomaly made using Photoshop. The final experimental results confirmed that model cannot detect minute abnormal regions of abnormal images but accurately determines whether it is normal or abnormal. The proposed method from these results show that it is suitable for future anomaly detection of damage and displacement in wooden cultural assets.
Intent-Based Adaptation Coordination of Highly Decentralized Networked Self-adaptive Systems

One challenge, that information technology faces, is how to cope with continuous contextual changes and the need for adjustments of the underlying networked system during runtime. In application domains like the Internet of Things (IoT) among others, system adjustments can't be performed in a device-by-device manner due to a large number of spatially distributed autonomous nodes. As a consequence, innovative approaches for autonomic management are required for such systems. A promising solution for network infrastructures are intent-based approaches. With the idea of declaratively specified goals for network functions, operators can holistically manage the network infrastructure at a high level of abstraction. Anyway, concrete design and implementation concepts are required to enforce intents within a networked system of autonomous nodes in a coordinated manner to ensure consistent behavior of the system according to the declared intent. In this chapter, concepts and approaches of self-adaptive systems are explored as a promising solution to serve as a basis for designing and implementing intent-based networked systems. The authors focus particularly on role concepts that allow a continuous design and implementation of system variability at run time. An example scenario from the IoT domain is used to continuously illustrate concepts and to demonstrate how networked self-adaptive systems can benefit from the introduced role-based concepts.
A mechanical true random number generator

Abstract Random number generation has become an indispensable part of information processing: it is essential for many numerical algorithms, security applications, and in securing fairness in everyday life. Random number generators (RNGs) find application in many devices, ranging from dice and roulette wheels, via computer algorithms, lasers to quantum systems, which inevitably capitalize on their physical dynamics at respective spatio-temporal scales. Herein, to the best of our knowledge, we propose the first mathematically proven true RNG (TRNG) based on a mechanical system, particularly the triple linkage of Thurston and Weeks. By using certain parameters, its free motion has been proven to be an Anosov flow, from which we can show that it has an exponential mixing property and structural stability. We contend that this mechanical Anosov flow can be used as a TRNG, which requires that the random number should be unpredictable, irreproducible, robust against the inevitable noise seen in physical implementations, and the resulting distribution’s controllability (an important consideration in practice). We investigate the proposed system’s properties both theoretically and numerically based on the above four perspectives. Further, we confirm that the random bits numerically generated pass the standard statistical tests for random bits.
Deterministic and Statistical Strategies to Protect ANNs against Fault Injection Attacks

Attificial neural networks are currently used for many tasks, including safety critical ones such as automated driving. Hence, it is very important to protect them against faults and fault attacks. In this work, we propose two fault injection attack detection mechanisms: one based on using output labels for a reference input, and the other on the activations of neurons. First, we calibrate our detectors during normal conditions. Thereafter, we verify them to maximize fault detection performance. To prove the effectiveness of our solution, we consider highly employed neural networks (AlexNet, GoogleNet, and VGG) with their associated dataset ImageNet. Our results show that for both detectors we are able to obtain a high rate of coverage against faults, typically above 96%. Moreover, the hardware and software implementations of our detector indicate an extremely low area and time overhead.
An On-chip Reconfigurable Multi-layer Perceptron and Recurrent Neural Network Processor for Speech Recognition

Modern voice-based applications, such as speech recognition and classification, mainly exploit the neural network-based algorithms (i.e., Multilayer Perceptron, Recurrent Neural Network, LSTM), and they have established extraordinary promising results in terms of accuracy, especially in sequential data recognition. However, such algorithms involve massive Recurrent Neural Network (RNN) related computations, which are extremely difficult to manage with traditional general-purpose processors. To solve this issue, a Lookup Tables (LUT) based multiplier architecture of accelerators have merged, which comprises a collection of several hundred processing elements (PEs). By exploiting the phenomena of parallel computations, these accelerators accomplish excellent results and high throughput in general. However, already proposed architectures of PEs do not optimize the weight calculation and other parameters that impact the silicon implementation. To this end, we propose an on-chip architecture for the MLP-RNN processor to speed up the matrix multiplication while optimizing the performance, cost, and power consumption with LUT-powered multiplication. Theoretically, we observed a 75% reduction in weight and 5x improvement in power consumption in simulation on 65nm at 200MHz clock 1.2V CMOS process technology
Aerial Small Object Tracking with Transformers

Due to the long distance of UAV aerial photography and the small proportion of objects, small object tracking represented by UAV aerial photography has always been a challenging part in the tracking field. Through experiments, we found that such challenges are strongly correlated with attributes such as occlusion, out-of-view, and drift. In this paper, we applied trackers with transformer and template update to aerial small object tracking, and applied them to the UAV aerial photography datasets and the small object datasets to propose corresponding small object trackers. The attention module in the transformer can provide a global response between template frame and search frame, which can effectively supplement contextual information for small objects, and solve the problem that small objects have weak feature expression ability. Using the template update strategy, the update of the template can be controlled under the conditions of occlusion, out-of-view, and drift that small objects are prone to appear, so the performance of the trackers on the small objects can be improved. Experiments show that these two small object trackers can achieve state-of-the-art performance on the corresponding datasets, and the improvements in attributes occlusion, out-of-view, and drift are particularly obvious.
Boundary-based Real-time Text Detection on Container Code

Scene text detection is attracting more and more attention in machine learning region. Automatic container code recognition is very important for modern container intelligent management system. However, there is no text detection network for container code. This article proposes a real-time container code text segmentation network based on boundary, which can accurately locate the text in real time. Specifically, we simultaneously predict the text and the boundary of the text, and fuse the features of the two branches to improve the accuracy of segmentation. In the post processing stage, the final result is gotten by text segmentation map minus the boundary segmentation map. We achieve a competitive F-measure of 96.5% at 70 FPS on container code datasets.
Technical note: The effect of image annotation with minimal manual interaction for semiautomatic prostate segmentation in CT images using fully convolutional neural networks

The goal is to study the performance improvement of a deep learning algorithm in three-dimensional (3D) image segmentation through incorporating minimal user interaction into a fully convolutional neural network (CNN).A U-Net CNN was trained and tested for 3D prostate segmentation in computed tomography (CT) images. To improve the segmentation accuracy, the CNN's input images were annotated with a set of border landmarks to supervise the network for segmenting the prostate. The network was trained and tested again with annotated images after 5, 10, 15, 20, or 30 landmark points were used.Compared to fully automatic segmentation, the Dice similarity coefficient increased up to 9% when 5-30 sparse landmark points were involved, with the segmentation accuracy improving as more border landmarks were used.When a limited number of sparse border landmarks are used on the input image, the CNN performance approaches the interexpert observer difference observed in manual segmentation.
Review of Modern Vulnerabilities in Blockchain Systems

Blockchain technology has recently entered the life of a modern person. This technology is quite young. It is developing very rapidly. More and more industries are adopting this technology. However, technical regulation procedures lag behind the development of the technology itself. There are a number of issues related to the security of personal data. This article examines the current vulnerabilities of modern blockchain systems. An attempt was made to systematize vulnerabilities depending on their nature of occurrence. The reasons for the emergence of such vulnerabilities are identified, primarily related to the lack of international standards for the development of such systems.
Causal Intervention for Object Detection

We present a causal intervention module (CIM) to improve object detection methods. State-of-the-art object detectors learn the association between image pixels and bounding boxes with labels, which implicitly use contextual information in the backbone. Intuitively, context is such a rich source of information that an improvement due to contextual information is relatively modest. Inspired by this, we use the context explicitly in a novel framework of causal intervention for object detection. Specifically, we use a structural causal model to reveal how context confounders affect the object detection model, and adopt causal intervention to deal with the effect. The proposed CIM is applied to a two-stage object detection baseline, and extensive experiments show its effectiveness.
Enterprise Architecture based Representation of Architecture and Design Patterns for Machine Learning Systems

In this study, we consider projects for the development of machine learning (ML) service systems that apply ML techniques to enterprise functions, and propose a method of representing the architecture and design patterns for ML service systems. Based on the proposed method, we represent the items described in the pattern documents as elements in the enterprise architecture modeling and derived a generic model for ML architecture and designed patterns. By applying the proposed method and the generic pattern model, we analyze an existing ML design pattern and represent it as a model. Through modeling practice, we confirm that an effective use scenario occurs when using the represented model during the project activities, and we can revise or enhance the pattern documents consistently by applying the model.
Translation system from Tunisian Dialect to Modern Standard Arabic

Summary Nowadays, Tunisian Dialect (TD) is widely used in social media such as Facebook and Twitter to communicate and express opinions. However, the absence of a TD translator prevents comprehension and communication for non‐Tunisian users. Thus, Tunisian translator becomes an important task to overcome these problems. In this article, our main objective is to create a translator from TD to Modern Standard Arabic (MSA). In this context, we are carrying out an in‐depth linguistic study in order to better choose the translation rules which are applicable on the TD sentences. This study guarantees the best quality to obtain MSA sentences. Indeed, our proposed method is based on a linguistic approach. In addition, the method consists of the elaboration of a set of dictionaries and the construction of inflectional, morphological, and syntactic grammars using finite‐state transducers. Then, our method is implemented and tested with new technologies provided by the NooJ linguistic platform. To evaluate our constructed tool, we apply it to two different test corpora containing more than 15,000 words. The obtained results are promising and do highlight our proposed method.