Reviewerly/exHarmony huggingface dataset:
In-Memory Indexed Caching for Distributed Data Processing

Powerful abstractions such as dataframes are only as efficient as their underlying runtime system. The de-facto distributed data processing framework, Apache Spark, is poorly suited for the modern cloud-based data-science workloads due to its outdated assumptions: static datasets analyzed using coarse-grained transformations. In this paper, we introduce the Indexed DataFrame, an in-memory cache that supports a dataframe abstraction which incorporates indexing capabilities to support fast lookup and join operations. Moreover, it supports appends with multi-version concurrency control. We implement the Indexed DataFrame as a lightweight, standalone library which can be integrated with minimum effort in existing Spark programs. We analyze the performance of the Indexed DataFrame in cluster and cloud deployments with real-world datasets and benchmarks using both Apache Spark and Databricks Runtime. In our evaluation, we show that the Indexed DataFrame significantly speeds-up query execution when compared to a non-indexed dataframe, incurring modest memory overhead.
Communication-efficient Massively Distributed Connected Components

Finding the connected components of an undirected graph is one of the most fundamental graph problems. Connected components are used in a wide spectrum of applications including VLSI design, machine learning and image analysis. Sequentially, one can easily find all connected components in linear time using breadth-first traversal. However, in a massively distributed setting, finding connected components in a scalable way becomes much harder due to data irregularities and the overhead associated with the increased need for communication. In this work, we present a communication-efficient distributed graph algorithm for finding connected components that scales to massively parallel machines. Our algorithm is based on a recent linear-work shared-memory parallel algorithm by Blelloch et al. [1] and refines it for a distributed memory setting. This includes a communication-efficient graph contraction procedure, as well as a distributed variant of the low diameter decomposition by Miller et al. [2]. We tackle the data irregularities introduced by high degree vertices by using an efficient procedure for distributing their incident edges. Our experimental evaluation on up to 16384 cores indicates a good weak scaling behavior that outperforms current state-of-the-art algorithms.
An Efficient Block Validation Mechanism for UTXO-based Blockchains

It has been recognized that one of the bottlenecks in the UTXO-based blockchain systems is the slow block validation - the process of validating a newly-received block by a node before locally storing it and further broadcasting it. As a block contains multiple inputs, the block validation mainly involves checking the inputs against the status data, which is also known as the Unspent Transaction Outputs (UTXO) set. As time goes by, the UTXO set becomes more and more expansive, most of which can only be stored on disks. This considerably slows down the input checking and thus block validation, which can potentially compromise system security. To deal with the above problem, we disassemble the function of input checking into three parts: existence validation (EV), unspent validation (UV), and script validation (SV). Based on the disassembly, we propose EBV, an efficient block validation mechanism to speed up EV, UV, and SV individually. First, EBV changes the representation of status data, from UTXO set to a bit-vector set, which drastically reduces its size. The smaller status data can be entirely maintained in memory, thereby accelerating UV and also block validation. Second, EBV requires each transaction to carry the proof data, which enables EV and SV without accessing the disks. Furthermore, we also cope with two challenges in the design of EBV, namely transaction inflation and fake positions. To evaluate the EBV mechanism, we implement a prototype on top of Bitcoin, the most widely known UTXO-based blockchain, and conduct extensive experiments to compare EBV and Bitcoin. The experimental results demonstrate that EBV successfully reduces the memory requirement by 93.1 % and the block validation time by up to 93.5%.
Booster: An Accelerator for Gradient Boosting Decision Trees Training and Inference

Recent breakthroughs in machine learning (ML) have sparked hardware innovation for efficient execution of the emerging ML workloads. For instance, due to recent refine-ments and high-performance implementations, well-established gradient boosting decision tree (GBT) models (e.g., XGBoost) have demonstrated their dominance in commercially-important contexts, such as table-based datasets (e.g., relational databases and spreadsheets). Unfortunately, GBT training and inference are time-consuming (e.g., several hours of training for large datasets). Despite their importance, GBTs have not been targeted for hardware acceleration as much as neural networks. We propose Booster, a novel accelerator for GBTs based on their unique characteristics. We observe that the dominant steps of GBT training and inference (accounting for 90-98% of time) involve simple, fine-grained, independent operations on small-footprint data structures (e.g., histograms and shallow trees) - i.e., GBT is on-chip memory bandwidth-bound. Unfortunately, existing multicores and GPUs do not support massively-parallel data structure accesses that are irregular and data-dependent. By employing a scalable sea-of-small-SRAMs approach and an SRAM bandwidth-preserving mapping of data record fields to the SRAMs called group-by-field mapping, Booster achieves significantly more parallelism (e.g., 3200-way parallelism) than multicores and GPUs. In addition, Booster employs a redun-dant data representation that significantly lowers the memory bandwidth demand. Our simulations reveal that Booster achieves 11.4x and 6.4x speedups for training, and 45x and 22x (21x and 11x) speedups for offline (online) inference, over an ideal 32-core multicore and an ideal GPU, respectively. Based on ASIC synthesis of FPGA-validated RTL using 45 nm technology, we estimate a Booster chip to occupy 60 mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> of area and dissipate 23 W when operating at 1-G Hz clock speed.
DEAN: A Lightweight and Resource-efficient Blockchain Protocol for Reliable Edge Computing

Edge computing draws a lot of recent research interests because of the performance improvement by offloading many workloads from the remote data center to nearby edge nodes. Nonetheless, one open challenge of this emerging paradigm lies in the potential security issues on edge nodes. This paper proposes a cooperative protocol, namely DEAN, equipped with a unique resource-efficient quorum building mechanism to adopt blockchain seamlessly in an edge computing infrastructure to prevent data manipulation and allow fair data sharing with quick recovery under resource constraints of limited storage, computing, and network capacity. Specifically, DEAN leverages a parallel mechanism equipped with three independent core components, effectively achieving low resource consumption while allowing secured parallel block processing on edge nodes. We have implemented a system prototype based on DEAN and experimentally verified its effectiveness with a comparison with four popular blockchain implementations: Ethereum, Parity, IOTA, and Hyperledger Fabric. Experimental results show that the system prototype exhibits high resilience to arbitrary failures. Performance-wise, DEAN-based blockchain implementation out-performs the state-of-the-art blockchain systems with up to 88.6 x higher throughput and 26 x lower latency.
Falcon: A Timestamp-based Protocol to Maximize the Cache Efficiency in the Distributed Shared Memory

Distributed shared memory (DSM) systems can handle data-intensive applications and recently receiving more attention. A majority of existing DSM implementations are based on write-invalidation (WI) protocols, which achieve sub-optimal performance when the cache size is small. Specifically, the vast majority of invalidation messages become useless when evictions are frequent. The problem is troublesome regarding scarce memory resources in data centers. To this end, we propose a self-invalidation protocol Falcon to eliminate invalidation messages. It relies on per-operation timestamps to achieve the global memory order required by sequential consistency (SC). Furthermore, we conduct a comprehensive discussion on the two protocols with an emphasis on the cache size impact. We also implement both protocols atop a recent DSM system, Grappa. The evaluation shows that the optimal protocol can improve the performance of a KV database by 27% and a graph processing application by 71.4% against the vanilla cache-free scheme.
Center Dropout: A Simple Method for Speed and Fairness in Federated Learning

This work tackles the Federated Training Task put forth by FeTS 2021. The goal of the task is to manipulate the aggregation algorithm, including aspects like client selection and training-per-round, in order to improve the generalizability of the segmentation algorithm as well as the communication cost. Our work proposes Center Dropout, a simple modification to the vanilla algorithm that does not train on the entire consortium on every round, but selects a random assortment of the total collaborators and proportionally increases the amount of local learning. This way, underrepresented centers are not consistently overwhelmed by the bigger ones during aggregation steps and communication costs are cut down. We also test another modification that smooths the weights of the aggregation algorithm to improve fairness across centers. We find that both techniques show benefits in terms of generalization performance, and using several variants of the Center Dropout method, we also achieve a significant boost in speed.
An Efficient Data Distribution Strategy for Distributed Graph Processing System

Big data applications like social networks, biological networks, etc. are often realized on graphs. Graph processing, if done on a single node, increases time complexity. Partitioning of graphs has been proved to be useful towards handle this well-known issue. There are several partitioning algorithms that are used to partition a graph. Each partition is assigned to a node within a cluster. However, the storage capacity of a node is limited. Therefore, an effective data distribution mechanism is required. This work aims to propose a novel strategy that would define an efficient distribution of graphs into nodes using genetic algorithms. The proposed data distribution strategy, when applied on two benchmark data set, shows improved data availability without increasing the number of replicas. It has also observed that the execution time will almost became half after applying the proposed method.
Software Product Maintenance: A Case Study

Maintenance is the most important part of software product development. It is a major component of all software development life cycle models (SDLC). Several variables are being used as intermediaries for software systems' maintainability throughout software development. The goal of this study is to see whether these variables are consistent between themselves and how well they anticipate maintenance efforts at the system stage. The maintenance of software products includes error checking, proper functionality of the system, improving performance, making the system adaptive, and follow up the change request that come from the client side. To perform all these types of actions to enhance the product quality 4 types of maintenance are normally followed Corrective, Preventive, Adaptive, and Perfective maintenance. The maintenance activity is the most time taking activity when change requests come from the client. This paper is explaining the concept of maintenance with an iterative model and its 4 types. Case studies are also discussed in the paper to explain the maintenance activities performed in software organizations and their impact on the product. The performance measures are not aligned with one another. Only the size of the system and its lack of consistency are found to be highly linked to greater maintenance effort. Maintainability measurements, aside from size, may well not accurately indicate additional maintenance effort. Parts of the project must be assessed in the situations where they will be deployed. Conventional factors are being used to indicate problematic places in the code, but improving the worst parts may mistakenly cause further problems throughout the system. Our findings show that local changes must be supported by a system-wide examination.
Automatic Brain Tumor Segmentation with a Bridge-Unet Deeply Supervised Enhanced with Downsampling Pooling Combination, Atrous Spatial Pyramid Pooling, Squeeze-and-Excitation and EvoNorm

Segmentation of brain tumors is a critical task for patient disease management. Since this task is time-consuming and subject to inter-expert delineation variation, automatic methods are of significant interest. The Multimodal Brain Tumor Segmentation Challenge (BraTS) has been in place for about a decade and provides a common platform to compare different automatic segmentation algorithms based on multiparametric magnetic resonance imaging (mpMRI) of gliomas. This year the challenge has taken a big step forward by multiplying the total data by approximately 3. We address the image segmentation challenge by developing a network based on a Bridge-Unet and improved with a concatenation of max and average pooling for downsampling, Squeeze-and-Excitation (SE) block, Atrous Spatial Pyramid Pooling (ASSP), and EvoNorm-S0. Our model was trained using the 1251 training cases from the BraTS 2021 challenge and achieved an average Dice similarity coefficient (DSC) of 0.92457, 0.87811 and 0.84094, as well as a 95% Hausdorff distance (HD) of 4.19442, 7.55256 and 14.13390 mm for the whole tumor, tumor core, and enhanced tumor, respectively on the online validation platform composed of 219 cases. Similarly, our solution achieved a DSC of 0.92548, 0.87628 and 0.87122, as well as HD95 of 4.30711, 17.84987 and 12.23361 mm on the test dataset composed of 530 cases. Overall, our approach yielded well balanced performance for each tumor subregion.
Unleashing the Potential of Sparse DNNs Through Synergistic Hardware-Sparsity Co-Design

Sparsity, a widely recognized path to curbing the computational needs of deep neural networks (DNNs), still suffers a number of roadblocks in practice, despite a decade of intensive research on sparse neural networks. The extant structured sparsity patterns often fail to attain significant model compression, while the hardware challenges posed by unstructured sparsity are yet to be fully overcome. As algorithmic and hardware innovations individually deliver limited benefits, a synergistic approach is necessary to unleash the potential of sparse DNNs. This work proposes a tightly integrated design methodology for the sparsity patterns and associated hardware platforms to reach the highest model compression goals while simultaneously facilitating efficient hardware processing. We demonstrate that novel complementary sparsity patterns can offer utmost expressiveness levels with inherent hardware exploitable regularity. Our novel dynamic training method converts the expressiveness of such sparsity configurations into highly accurate and compact sparse neural networks. Complementary sparsity is represented in a dense format, and when synergistically coupled with minimal yet strategic hardware modifications, can be processed in close concordance with the conventional dataflow of the dense matrix operations. We thus demonstrate that there is ample room for innovation beyond conventional techniques and immense practical potential for sparse neural networks through the synergistic design of sparsity patterns and hardware architectures.
DeepLogic: Joint Learning of Neural Perception and Logical Reasoning

Neural-symbolic learning, aiming to combine the perceiving power of neural perception and the reasoning power of symbolic logic together, has drawn increasing research attention. However, existing works simply cascade the two components together and optimize them isolatedly, failing to utilize the mutual enhancing information between them. To address this problem, we propose DeepLogic, a framework with joint learning of neural perception and logical reasoning, such that these two components are jointly optimized through mutual supervision signals. In particular, the proposed DeepLogic framework contains a deep-logic module that is capable of representing complex first-order-logic formulas in a tree structure with basic logic operators. We then theoretically quantify the mutual supervision signals and propose the deep&logic optimization algorithm for joint optimization. We further prove the convergence of DeepLogic and conduct extensive experiments on model performance, convergence, and generalization, as well as its extension to the continuous domain. The experimental results show that through jointly learning both perceptual ability and logic formulas in a weakly supervised manner, our proposed DeepLogic framework can significantly outperform DNN-based baselines by a great margin and beat other strong baselines without out-of-box tools.
On capabilities of quantum-mechanical computer models

The work includes the analyses of the theoretical capabilities of quantum-mechanical versus classical computer models in terms of their universality, their application domain, their efficacy in solving the problems and their technological feasibility. The concepts of deterministic Turing machine, probabilistic Turing machine and quantum-mechanical Turing machine are exposed. The concept of basic information unit of quantum-mechanical computer modelqubit is introduced and the qubit is represented through the use of Bloch sphere. The use of Dirac's bra-ket notation in description of quantum-mechanical state is explained, and the notation is used to form state equation of a quantummechanical Turing machine cell. Especially, consequences of quantum parallelism, quantum interference and wave function collapse are studied in respect to capabilities of quantum-mechanical computer models. The interrelationship between classes of problems that can be efficiently solved by quantum mechanical and/or classical computer models is displayed.
A Secure and Efficient Data Distribution System in a Multi-Cloud Environment

Cloud computing is a model for providing online services. These services can be accessed on a pay-per-use basis from anywhere at any time. However, data security in the cloud has become a very big problem. A major concern about data security is the cloud service provider (CSP), which has access to data, which in turn increases user concerns and reduces cloud capacity in many areas such as medical record databases and financial data. The problem of data security mainly affects the use of cloud computing by users and organizations. The current research on data protection has been done in multi-cloud computing so that CSPs do not have direct access to the data. In this paper, a secure and efficient cloud data system is proposed in order to ensure data confidentiality and integrity by encrypting the image using DNA encoding combined with the logistical map and spatial map. The encrypted image is then divided into chunks before being stored in the multi-cloud to ensure the confidentiality of the data. The hash value is calculated using the Message Digest (MD5) algorithm to verify the integrity of the data. The proposed system also supports efficient updating of outsourced data. For the security analysis, many experiments were conducted to ensure the confidentiality and privacy of data in a multi-cloud environment, where files of different sizes were used, and the analysis showed that the proposed system is more secure. On the other hand, conducted several tests to analyse performance with dynamic data support and latency, where the analysis showed that the results are at the required level. Finally, through a comprehensive analysis of security and performance, the proposed system is proven to be more secure in a multi-cloud environment and that the system is highly efficient
Multi-scale 2D Representation Learning for weakly-supervised moment retrieval

Video moment retrieval aims to search the moment most relevant to a given language query. However, most existing methods in this community often require temporal boundary annotations which are expensive and time-consuming to label. Hence weakly supervised methods have been put forward recently by only using coarse video-level label. Despite effectiveness, these methods usually process moment candidates independently, while ignoring a critical issue that the natural temporal dependencies between candidates in different temporal scales. To cope with this issue, we propose a Multi-scale 2D Representation Learning method for weakly supervised video moment retrieval. Specifically, we first construct a two-dimensional map for each temporal scale to capture the temporal dependencies between candidates. Two dimensions in this map indicate the start and end time points of these candidates. Then, we select top-K candidates from each scale-varied map with a learnable convolutional neural network. With a newly designed Moments Evaluation Module, we obtain the alignment scores of the selected candidates. At last, the similarity between captions and language query is served as supervision for further training the candidates' selector. Experiments on two benchmark datasets Charades-STA and ActivityNet Captions demonstrate that our approach achieves superior performance to state-of-the-art results.
Construction of Differentially Private Summaries Over Fully Homomorphic Encryption

Cloud computing has garnered attention as a platform of query processing systems. However, data privacy leakage is a critical problem. Chowdhury et al. proposed Cryptε, which executes differential privacy (DP) over encrypted data on two non-colluding semi-honest servers. Further, the DP index proposed by these authors summarizes a dataset to prevent information leakage while improving the performance. However, two problems persist: 1) the original data are decrypted to apply sorting via a garbled circuit, and 2) the added noise becomes large because the sorted data are partitioned with equal width, regardless of the data distribution. To solve these problems, we propose a new method called DP-summary that summarizes a dataset into differentially private data over a homomorphic encryption without decryption, thereby enhancing data security. Furthermore, our scheme adopts Li et al.’s data-aware and workload-aware (DAWA) algorithm for the encrypted data, thereby minimizing the noise caused by DP and reducing the errors of query responses. An experimental evaluation using torus fully homomorphic encryption (TFHE), a bit-wise fully homomorphic encryption library, confirms the applicability of the proposed method, which summarized eight 16-bit data in 12.5 h. We also confirmed that there was no accuracy degradation even after adopting TFHE along with the DAWA algorithm.