Reviewerly/exHarmony huggingface dataset:
Optimization Effects for Word Representations with L2-Regularized Non-Parametric Model for Contrasting Epochs

The high dimensional sparse matrix word representations applied with supervised non-parametric learning models can have some infractions from training the model and result in errors known as loss. This loss of information can be retained using mathematical functions known as Optimizers which calculate the partial derivatives of the model parameters to yield better results. In this paper we have presented a result oriented experimentation of loss convergence performed on 5000 dimensional sparse matrix word representation with L2 regularized non-parametric supervised text classification model. We have presented results over 10, 50 and 100 epochs of training and traced the optimization prowess of varied loss optimizers that use different convergence approaches.
MLCNN: Cross-Layer Cooperative Optimization and Accelerator Architecture for Speeding Up Deep Learning Applications

The ever-increasing number of layers, millions of parameters, and large data volume make deep learning workloads resource-intensive and power-hungry. In this paper, we develop a convolutional neural network (CNN) acceleration framework, named MLCNN, which explores algorithm-hardware co-design to achieve cross-layer cooperative optimization and acceleration. MLCNN dramatically reduces computation and on-off chip communication, improving CNN's performance. To achieve this, MLCNN reorders the position of nonlinear activation layers and pooling layers, which we prove results in a negligible accuracy loss; then the convolutional layer and pooling layer are co-optimized by means of redundant multiplication elimination, local addition reuse, and global addition reuse. To the best of our knowledge, MLCNN is the first of its kind that incorporates cooperative optimization across convolutional, activation, and pooling layers. We further customize the MLCNN accelerator to take full advantage of cross-layer CNN optimization to reduce both computation and on-off chip communication. Our analysis shows that MLCNN can significantly reduce (up to 98%) multiplications and additions. We have implemented a prototype of MLCNN and evaluated its performance on several widely used CNN models using both an accelerator-level cycle and energy model and RTL implementation. Experimental results show that MLCNN achieves 3.2x speedup and 2.9x energy efficiency compared with dense CNNs. MLCNN's optimization methods are orthogonal to other CNN acceleration techniques, such as quantization and pruning. Combined with quantization, our quantized MLCNN gains a 12.8x speedup and 11.3x energy efficiency compared with DCNN.
Parallel Vertex Cover Algorithms on GPUs

Finding small vertex covers in a graph has applications in numerous domains such as scheduling, computational biology, telecommunication networks, artificial intelligence, social science, and many more. Two common formulations of the problem include: Minimum Vertex Cover (MVC), which finds the smallest vertex cover in a graph, and Parameterized Vertex Cover (PVC), which finds a vertex cover whose size is less than or equal to some parameter <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$k$</tex> . Algorithms for both formulations involve traversing a search tree, which grows exponentially with the size of the graph or the value of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$k$</tex> . Parallelizing the traversal of the vertex cover search tree on GPUs is challenging for multiple reasons. First, the search tree is a narrow binary tree which makes it difficult to extract enough sub-trees to process in parallel to fully utilize the GPU's massively parallel execution resources. Second, the search tree is highly imbalanced which makes load balancing across a massive number of parallel GPU workers especially challenging. Third, keeping around all the intermediate state needed to traverse many sub-trees in parallel puts high pressure on the GPU's memory resources and may act as a limiting factor to parallelism. To address these challenges, we propose an approach to traverse the vertex cover search tree in parallel using GPUs while handling dynamic load balancing. Each thread block traverses a different sub-tree using a local stack, however, we use a global worklist to balance the load to ensure that all blocks remain busy. Blocks contribute branches of their sub-trees to the global worklist on an as-needed basis, while blocks that finish their sub-trees pick up new ones from the global worklist. We use degree arrays to represent intermediate graphs so that the representation is compact in memory to avoid limiting parallelism, but self-contained which is necessary for the load balancing process. Our evaluation shows that compared to approaches used in prior work, our hybrid approach of using local stacks and a global worklist substantially improves performance and reduces load imbalance, especially on difficult instances of the problem. Our implementations have been open sourced to enable further research on parallel solutions to the vertex cover problem and other similar problems involving parallel traversal of narrow and highly imbalanced search trees.
CSC: Collaborative System Configuration for I/O-Intensive Applications in Multi-Tenant Clouds

I/O-intensive applications are important workloads of public clouds. Multiple cloud applications co-run on the same physical machine in different virtual machines (VMs), and the shared resources (e.g., disk bandwidth) are often isolated for fairness. Our investigation shows that the performance of an I/O-intensive application is impacted by both disk bandwidth allocation and the page cache settings in the guest operating system. However, none of prior work considers adjusting the page cache settings for better performance, when the disk bandwidth allocation is adjusted. We therefore propose CSC, a system that collaboratively identifies the appropriate disk bandwidth allocation and page cache settings in the guest operating system of each VM. CSC aims to improve the system-wide I/O throughput of the physical machine, while also improve the I/O throughput of each individual I/O-intensive application in VMs. CSC comprises an online disk bandwidth allocator and an adaptive dirty page setting optimizer. The bandwidth allocator monitors the disk bandwidth utilization and re-allocates some bandwidth from free VMs to busy VMs periodically. After the re-allocation, the opti-mizer identifies the appropriate dirty page settings in the guest operating system of the VMs using Bayesian Optimization. The experimental results show that CSC improves the performance of I/O-intensive applications by 9.5 % on average (up to 17.29 %) when 5 VMs are co-located while fairness is guaranteed.
PARSEC: PARallel Subgraph Enumeration in CUDA

Subgraph enumeration is an important problem in the field of Graph Analytics with numerous applications. The problem is provably NP-complete and requires sophisticated heuristics and highly efficient implementations to be feasible on problem sizes of realistic scales. Parallel solutions have shown a lot of promise on CPUs and distributed environments. Recently, GPU-based parallel solutions have also been proposed to take advantage of the massive execution resources in modern GPUs. Subgraph enumeration involves traversing a search tree for each vertex of the data graph to find matches of a query in a graph. Most GPU-based solutions traverse the tree in breadth-first manner that exploits parallelism at the cost of high memory requirement and presents a formidable challenge for processing large graphs with high-degree vertices since the memory capacity of GPUs is significantly lower than that of CPUs. In this work, we propose a novel GPU solution based on a hybrid BFS and DFS approach where the top level(s) of the search trees are traversed in a fully parallel, breadth-first manner while each subtree is traversed in a more space-efficient, depth-first manner. The depth-first traversal of subtrees requires less memory but presents more challenges for parallel execution. To overcome the less parallel nature of depth-first traversal, we exploit fine-grained parallelism in each step of the depth-first traversal of sub-trees. We further identify and implement various optimizations to efficiently utilize memory and compute resources of the GPUs. We evaluate our performance in comparison with the state-of-the-art GPU and CPU implementations. We outperform the GPU and CPU implementations with a geometric mean speedup of 9.47× (up to 92.01×) and 2.37× (up to 12.70×), respectively. We also show that the proposed approach can efficiently process the graphs that previously cannot be processed by the state-of-the-art GPU solutions due to their excessive memory requirement.
The use of random forests to support the decision-making process for sustainable manufacturing

Abstract Today’s companies need to change the way they produce and consume goods to meet sustainability requirements. These companies are striving to increase production efficiency, while reducing the use of raw materials, reducing costs and reducing their impact on the environment. Intelligent decision support systems are an inseparable element supporting such action. The presented work the use the data mining method (random forests) to identify the parameters affecting the condition of the cutting blade is presented. The developed model was evaluated using indicators to assess the quality of classifications. The obtained results confirm the high quality of the predictive model. The proposed model can be used to support the decision-making process in determining the life of a cutting tool. This will reduce the consumption of raw materials and production waste, and thus reduce the company’s environmental impact, which is consistent with the assumptions of sustainable production.
Development and future of information hiding in image transformation domain: A literature review

Information hiding technology is a technique to hide meaningful information in the public carrier information. When data elements are becoming more and more important, information hiding technology has a better performance than traditional encryption and decryption technologies such as single table substitution and Virginia cipher. Since it contains redundant information, image is a common carrier for information hiding among many carrier types. As one of the primary means of information hiding technology, image transform-domain information hiding technology has been widely studied and used in academic and industrial fields. Information hiding in the image transform domain improves security and robustness effectively. This paper mainly introduces the concepts, principles, processes, mainstream algorithms and applications of image transform-domain information hiding techniques. A possible general future development of it is also depicted.
DnT: Learning Unsupervised Denoising Transformer from Single Noisy Image

In the last few years, a myriad of Transformer based methods have drawn considerable attention due to their outstanding performance on various computer vision tasks. However, most image denoising methods are based on convolutional neural networks (CNNs), few attempts have been made with Transformer, especially in self-supervised and unsupervised methods. In this paper, we propose a novel and good performance unsupervised image Denoising Transformer (DnT) which is just trained by the single input noisy image. Our network combines Transformer and CNN to predict the counterpart clean target, the training loss was measured by pairs of noisy independent images constructed from the input image. The dropout-based ensemble is used to get the final denoised result by averaging multiple predictions generated by the trained model. Experiments show that the proposed method not only has superior performance over the state-of-the-art single noisy image denoiser on additive white Gaussian noise (AWGN) removal but also achieves good results on real-world image denoising.
Masked Face Recognition with 3D Facial Geometric Attributes

During the coronavirus pandemic, the demand for contactless biometrics technology has promoted the development of masked face recognition. Training a masked face recognition model needs to address two crucial issues: a lack of large-scale realistic masked face datasets and the difficulty of obtaining robust face representations due to the huge difference between complete faces and masked faces. To tackle with the first issue, this paper proposes to train a 3D masked face recognition network with non-masked face images. For the second issue, this paper utilizes the geometric features of 3D face, namely depth, azimuth, and elevation, to represent the face. The inherent advantages of 3D face enhance the stability and practicability of 3D masked face recognition network. In addition, a facial geometry extractor is proposed to highlight discriminative facial geometric features so that the 3D masked face recognition network can take full advantage of the depth, azimuth and elevation information in distinguishing face identities. The experimental results on four public 3D face datasets show that the proposed 3D masked face recognition network improves the accuracy of the masked face recognition, which verifies the feasibility of training the masked face recognition model with non-masked face images.
Image Processing based Scoring System for Small Arms Firing in the Military Domain

Firing and its overall conduct of scoring plays a very vital role in the military organizations. Currently, the evaluation of such a system is partially automated that still needs manual works. A fully automatic calculation of target scoring is time demanding and can add a great value in minimizing human error. This paper puts focus on designing and developing the process of scoring a fired target automatically based on image processing techniques in the context of military domain. Along with conventional image processing techniques, this work has used structural similarity indexing algorithms to detect the bullet holes from images. The experiment has been conducted on real-time scenarios considering the military field and shows a promising result to calculate the scores automatically.
Resdense-Focal-Deeplabv3+ Enabled Litchi Branch Semantic Segmentation for Robotic Harvesting

Robotic harvesting of litchi fruit should be achieved by clamping and shearing the bearing branches because the litchi fruits are small and can be easily damaged. Therefore, accurate segmentation of litchi branches is particularly important. In this study, the generic DeepLabv3+ model was initially used to semantically segment litchi branches. To further optimize its segmentation performance, a novel backbone of ResDense was proposed to replace the original backbone of DeepLabv3+ by integrating ResNet and DenseNet networks. Moreover, the cross-entropy loss function in DeepLabv3+ was replaced by an improved focal loss, thus constructing an improved ResDense-focal-DeepLabv3+ (or RDf-DeepLabv3+) network for accurate segmentation of litchi branches in orchard environments. To investigate the capability of feature extraction using the proposed RDf-DeepLabv3+, the dataset was further divided into three levels of complexity (i.e., simple, medium, and complex) to train, validate, and test the model. In addition, to prove the superiority of the proposed model, the other three powerful models were utilized for a comprehensive comparison, including ResNet-CE, DenseNet, and Xception. The results showed that our RDf-DeepLabv3+ model outperformed others. More specifically, the proposed model overall achieved a mean Intersection over Union (mIoU) value of 0.797, which was 0.9%, 1.8%, and 12.9% higher than ResNet-CE, DenseNet and Xception, respectively. When targeting “simple” level images, the mIoU of 0.848 was achieved by the proposed model with only slight improvements compared to others. While targeting “medium” level images, the mIoU of the proposed model was 0.811, which was 0.7%, 2.7%, and 14.8% better than the same ones, respectively. When considering “complex” level images, the mIoU of 0.770 was achieved by our model, improving by 0.6%, 1.3%, and 15.3%, respectively. Moreover, it was found that both training and testing speed of the proposed model are about 7.7% faster than those of Xception. Overall, the RDf-DeepLabv3+ developed in this study achieved the optimal semantic segmentation result on litchi branches.
An Efficient Gaussian Filter Based on Gaussian Symmetric Markov Random Field

This article presents a new image denoising algorithm that uses Gaussian Symmetric Markov random fields based on maximum a posteriori estimation.First, an image denoising model based on Gaussian Symmetric Markov random fields is built, and the image denoising problem was converted to a maximum a posteriori estimation problem.The prior probability of an image can be estimated using the Gibbs distribution, which is equivalent to Markov random fields.Second, the maximum a posteriori estimation is calculated using the expectation-maximization algorithm and conjugate gradient method, where the expectation-maximization algorithm is used to estimate Gaussian Symmetric Markov random field hyper-parameters and the conjugate gradient method is used to calculate the criterion function.The experimental results for the synthetic images and the standard Berkeley segmentation datasets demonstrate the success of the proposed Gaussian denoising filter, as compared with the state-of-the-art methods such as BM3D, WNNM, SGWD-HMMs, SSLBD, DnCNN and BUIFD.
Chinese Spelling Error Detection and Correction Based on Knowledge Graph

Spelling error correction is a task in which errors in a natural language sentence can be detected and corrected. In this paper, we consider Chinese spelling error correction (CSC) for generality. A previous state-of-the-art method for this task connects a detection network with a correction network based on BERT by soft masking. This method does solve the problem that BERT has the insufficient capability to detect the position of errors. However, we find that it still lacks sufficient inference ability and world knowledge by analyzing its results. To solve this issue, we propose a novel correction approach based on knowledge graphs (KGs), which queries triples from KGs and injects them into the sentences as domain knowledge. Moreover, we leverage MLM as correction to improve the inference ability of BERT and adopt a denoising filter to increase the accuracy of results. Experimental results on the SIGHAN dataset verify that the performance of our approach is better than state-of-the-art methods.
Model-Driven Development of Distributed Ledger Applications

The Distributed Ledger Technology (DLT) is one of the most durable results of virtual currencies, which goes beyond the financial sector and impacts business applications in general. Developers can empower their solutions with DLT capabilities to attain such benefits as decentralization, transparency, non-repudiability of actions and security and immutability of data assets, to the price of integrating a distributed ledger framework into their software architecture. Model-Driven Development (MDD) is the discipline that advocates the use of abstract models and of code generation to reduce the application development and integration effort by delegating repetitive coding to an automated model-to-code transformation engine. In this paper, we explore the suitability of MDD to support the development of hybrid applications that integrate centralized database and distributed ledger architectures and describe a prototypical tool capable of generating the implementation artefacts starting from a high level model of the application and of its architecture.
Hardware-agnostic computation for large-scale knowledge graph embeddings

Knowledge graph embedding research has mainly focused on learning continuous representations of knowledge graphs towards the link prediction problem. Recently developed frameworks can be effectively applied in research related applications. Yet, these frameworks do not fulfill many requirements of real-world applications. As the size of the knowledge graph grows, moving computation from a commodity computer to a cluster of computers in these frameworks becomes more challenging. Finding suitable hyperparameter settings w.r.t. time and computational budgets are left to practitioners. In addition, the continual learning aspect in knowledge graph embedding frameworks is often ignored, although continual learning plays an important role in many real-world (deep) learning-driven applications. Arguably, these limitations explain the lack of publicly available knowledge graph embedding models for large knowledge graphs. We developed a framework based on the frameworks DASK, Pytorch Lightning and Hugging Face to compute embeddings for large-scale knowledge graphs in a hardware-agnostic manner, which is able to address real-world challenges pertaining to the scale of real application. We provide an open-source version of our framework along with a hub of pre-trained models having more than 11.4 B parameters. 1https://github.com/dice-group/dice-embeddings.1https://github.com/dice-group/dice-embeddings.
A Proposal for Enhancing Agile Requirements Engineering with Prototyping and Enriched User Stories

Agile software development is the most developed and widely used family of software development processes. Agile Requirement Engineering (ARE) faces many challenges, such as managing with very little documentation and specifications, ensuring that the development team understands the requirements, and understanding all client needs. In agile methodologies, user stories are the primary means for capturing requirements. But most of the time, user stories are not enough to describe the requirements to the development team to the required level of details. Moreover, sometimes, the client does not have a clear vision of the system features from the beginning. S/he discovers them during or even after delivery, leading to an increase in the amount of rework. Hence, some researchers have suggested solutions to these challenges by using prototypes to support the user stories in explaining the requirements. Others suggested using enriched user stories to get more details about how to implement the requirements. In this paper, we discuss the challenges facing ARE, the role that each of prototyping and enriched user stories can play in improving ARE, and the limitations of each one if used alone. We propose a roadmap to improve ARE by a hybrid process that merges prototyping and enriched user stories along with extra validation steps.