Reviewerly/exHarmony huggingface dataset:
Poster: Towards Accurate and Fast Federated Learning in End-Edge-Cloud Orchestrated Networks

This work proposes a novel three-layer federated learning (FL) framework with parameter selection and pre-synchronization (PSPFL) to achieve fast and accurate model training. The basic idea of PSPFL is that clients select partial model parameters for transmission and then base stations aggregate them cooperatively (i.e., pre-synchronization) and send the aggregated results to the server for global model update periodically. However, there is an intrinsic trade-off between parameter transmission overhead and model training loss. To strike a desirable balance between them, we investigate the optimal parameter pre-synchronization round and local training round under PSPFL. Specifically, we propose a Deep Q-Network (DQN)-based method to obtain the local training round and parameter pre-synchronization round. Finally, extensive experiments are conducted to evaluate the performance of the proposed method on commonly used datasets. The results show that the proposed method can reduce the sum of FL completion time and training loss by an average of 8.17%-18.82% compared to benchmarks.
TradeFL: A Trading Mechanism for Cross-Silo Federated Learning

Cross-silo federated learning (CFL) is a distributed learning paradigm that allows organizations (e.g., financial or medical entities) to train a global model on siloed data. Recent studies on mechanisms designed for CFL, however, rarely jointly consider the potential inter-organizational competition and the lack of credibility between organizations, which may discourage organizational participation. In this paper, we investigate the problem of inter-organizational competition and credibility assurance. We propose a distributed trading mechanism, called <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$TradeFL$</tex> , to incentivize organizations to contribute data and computational resources through mutual trading among organizations. Technically, TradeFL characterizes the competition among organizations and compensates for their damage incurred by competition. TradeFL runs on distributed organizations and provides credibility guarantees for compensation through a customized smart contract <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Illustration of the prototype: https://github.com/user10963.. We prove that the interaction among organizations that contribute resources to maximize personal payoffs is a weighted potential game. Then, we propose a centralized algorithm and a distributed algorithm to determine the optimal resource contribution. Simulation results and evaluations based on real-world datasets demonstrate that our scheme achieves higher social welfare, increases the amount of contributed data by up to 64%, and improves the accuracy of the global model by at most 23.2%.
Distributed Online Min-Max Load Balancing with Risk-Averse Assistance

Motivated by a wide range of applications from parallel computing to distributed learning, we study distributed online load balancing among multiple workers. We aim to minimize the pointwise maximum over the workers' local cost functions. We propose a novel algorithm termed Distributed Online Load Balancing with rIsk-averse assistancE (DOLBIE), which jointly considers the worker heterogeneity and system dynamics. The workload is distributed to workers in an online manner, where the underloaded workers learn to provide an appropriate amount of assistance to the most overloaded worker for the next online round without making themselves overwhelmed. In DOLBIE, all workers participate in updating the workload simultaneously, and no computationally intensive gradient or projection calculation is required. DOLBIE can be implemented in both the master-worker and fully-distributed architectures. We analyze the worst-case performance of DOLBIE by deriving an upper bound on its dynamic regret. We further demonstrate the application of DOLBIE to online batch-size tuning in distributed machine learning. Our experimental results show that, in comparison with state-of-the-art alternatives, DOLBIE can substantially speed up the training process and reduce the workers' idle time.
Towards Decentralized Adaptive Control of Cryptocurrency Liquidity via Auction

Sustainable cryptocurrency systems need to address the challenge of continuous inflation. Systematically maintaining stability during market turbulence can be challenging when token minting rates are predetermined and do not account for actual liquidity demand. Mindful of the critical role played by monetary policy, we propose a decentralized, truthful auction approach for tuning the currency system toward self-stabilization in volatile markets. Concretely, when severe inflation or market overheating occurs, the system enforces a contractionary monetary policy to restrain the market liquidity and avoid potential economic disasters. Should severe deflation and market stagnancy occur, more tokens will be minted than burnt to stimulate the market. Through this approach, our studies aim to provide a tool to help sustain the long-term existence of decentralized commodities.
Smart Contract Parallel Execution with Fine-Grained State Accesses

As various optimizations being proposed recently, the performance of blockchains is no longer limited by the consensus protocols, successfully scaling to thousands of transactions per second. To further improve blockchains' throughput, exploiting the parallelism in smart contract executions becomes a clear solution to resolve the new performance bottleneck. The existing techniques perform concurrency control on smart contract transactions based on pre-determined read/write sets, which can hardly be calculated precisely. As a result, many parallelization opportunities are missed in order to maintain the correctness of transaction executions. In this paper, we propose a novel execution scheduling framework, DMVCC, to further increase the parallelism in smart contract executions, via more fine-grained control on state accesses. DMVCC improves over existing techniques with two key features: (1) write versioning, eliminating the write-write conflicts between transactions, and (2) early-write visibility, enabling other transactions to read the writes from a transaction earlier, before it being committed. We integrated DMVCC into the Ethereum Virtual Machine, to evaluate its performance in real-world blockchain environments. The experimental results show that DMVCC doubles the parallel speedup achievable to a 20 x overall speedup, compared with the serial execution baseline, approaching the theoretical optimum.
Demo: Decentralized Autonomous Organization with Centralized Crisis Resolution

Decentralized autonomous organizations (DAOs) offer transparency and integrity in decision-making, but their decentralized structure presents challenges during times of crisis. Our paper implements and demonstrates a solution to this issue by introducing a quasi DAO (qDAO) that allows for centralized decision-making when necessary. By modifying smart contracts and introducing new rules, we strike a balance between decentralization and centralized management, offering potential opportunities for improved crisis management in DAOs. We present a demo of our qDAO implementation on the local Ethereum testnet, along with a web application for user input. Our approach expands the possibilities of DAO administrator control while maintaining the integrity and auditability of DAO operations.
Poster:Multi-Modal-Based Video Stabilization for Mobile Devices in Dynamic Scenes

Handheld devices such as smartphones and cameras often produce shaky videos due to reasons such as hand tremors and movement. Current video stabilization methods mainly estimate camera motion based on feature points, such as corner points, and then smooth the motion to obtain a stabilized video. However, in reality, most videos are shot in dynamic scenes, where some feature points are extracted from moving objects. They seriously affect the estimation of camera motion, and result in unsatisfactory video stabilization. In this paper, we propose a multi-modal-based video stabilization method for dynamic scenes using video frames and gyroscope. The basic idea is to eliminate dynamic feature as their estimated rotation significantly differs from the actual rotation obtained from the gyroscope data. The experimental results show that our method effectively enhances the quality of shaky videos in dynamic scenes.
Interference-Aware VM Placement in Cloud

Cloud computing paradigm uses Virtual Machine (VM)-based Resource provisioning strategy. Allocating VM in an appropriate PM (Physical Machine) is challenging. Efficient VM allocation reduce the number of under-utilized PMs in Cloud. We developed a new algorithm that considers compatibility of VMs in a PM before allocation. We performed basic experiment and presented the results in this article.
Efficient and Secure Spatial Range Query over Large-scale Encrypted Data

Spatial range query enjoys widespread application scenarios due to the ever-growing geo-positioning technology in recent years. Huge amounts of encrypted geo-location data are being outsourced to cloud servers to alleviate local storage and computational overheads without leaking sensitive information. However, most existing Privacy-preserving Spatial Range Query (PSRQ) cannot achieve high efficiency while satisfying strong security over large-scale encrypted spatial data. To strike a best possible balance between security and efficiency, we propose a novel efficient Privacy-preserving Spatial Range Query (eP-SRQ) scheme in dual-cloud architecture over large-scale dataset. Specifically, we propose an efficient PSRQ scheme by designing a novel index structure based on Geohash algorithm, Circular Shift and Coalesce Zero-Sum Garbled Bloom Filter (CSC-ZGBF) and Symmetric Homomorphic Encryption (SHE), which makes the computational complexity of query process independent of dataset size. Formal security analysis proves that our scheme can achieve Indistinguishability against Chosen-Plaintext Attack (IND-CPA), and extensive experiments prove that our scheme is feasible in real-world applications.
FriendSeeker: Inferring Hidden Friendship in Mobile Social Networks with Sparse Check-in Data

Check-in data widely published in mobile social networks (MSNs) pose serious privacy threats to users. Existing inference attack methods show that pairwise social relationship could be estimated by analyzing check-in user records. However, the efficacy of such attacks heavily depends on the density of check-in data, which is often not present in practice. In this work, we propose a new inference attack scheme, which for the first time can effectively reveal hidden social friendship in both the real world and in cyberspace among users with only sparse check-in data. Our attack method enjoys two salient features. First, it requires no prior knowledge about social connections, instead it estimates users' social proximity by exploiting both physical presence and social proximities. Second, our attack scheme can automatically learn representative features based on the significance of various check-in records, rather than relying on heuristic features. We conduct extensive trace-driven simulations, and the results demonstrate that our inference attack method can improve the efficacy of the state-of-the-art learning-based schemes up to 40%. Moreover, our proposed attack method is also robust against common data obfuscation mechanisms.
Poster: Multi-Writer Searchable Encryption with Fast Search and Post-Quantum Security

Searchable encryption enables secure searches over encrypted data in the cloud. Among all paradigms, public key encryption with keyword search (PEKS) is particularly desirable by privacy-preserving distributed computing and IoT applications, since it allows multiple parties (i.e., writers) to independently contribute encrypted data. However, a PEKS search usually requires a linear scan over the entire dataset for keyword search, causing unacceptable latency when facing a large amount of data. All existing efforts to speed up multi-writer searchable encryption are based on conventional hardness assumptions for security, which can be broken provided the advent of quantum computers. In this work, we propose a lattice-based multi-writer searchable encryption scheme, which lets writers build indices for their outsourced data to make keyword searches faster. Meanwhile, the security of the proposed solution relies on the learning with errors assumption, which is known to withstand the potential attack from quantum computers.
Scalable Federated Learning with System Heterogeneity

Federated learning (FL) is a distributed learning framework that inherently provides data privacy and parallel computation capability over a set of participating devices (clients). In real-life applications, these clients can have a great variety in terms of resources (storage, RAM, CPU/GPU speed, network speed, etc.). However, most previous FL studies do not consider this scenario with system heterogeneity and assume that all clients can operate on the same full-size deep neural network (DNN) model. In this work, we demonstrate a scalable FL approach, ScaleFL, which tackles system heterogeneity through hierarchically downscaling the DNN model for clients with limited resources. ScaleFL utilizes early exits to form multi-exit DNN models by injecting early exit networks into the given DNN. During FL, the model is adaptively split along depth (exits) and width (hidden dimensions) based on the resource budget of each participating client. A proof-of-concept demonstration is provided with interactive features, demonstrating the system flow on image classification and NLP benchmark workloads.
On Arbitrary Ignorance of Stragglers with Gradient Coding

Gradient methods, such as gradient descent, are widely deployed to train optimization-based models in machine learning. To train such models on a large dataset, the dataset is commonly split into multiple partitions which are trained on different workers. In order to tolerate stragglers, existing techniques either use gradient coding (GC) to recover the full gradients from a certain number of workers or recover gradients partially from an arbitrary number of workers. In this paper, we propose ignore-straggler gradient coding (IS-GC) that allows GC to tolerate an arbitrary number of stragglers. Compared to approximated gradient descent, IS-GC can recover more gradients when there is the same number of stragglers. We design a graph-based model to decode coded gradients from an arbitrary number of stragglers, and prove that it can maximize the recovery of gradients. We apply IS-GC on fractional repetition (FR) and cyclic repetition (CR), two representative dataset placement schemes of GC. We also propose hybrid repetition (HR) that generalizes over FR and CR and achieves a flexible trade-off between FR and CR. With extensive experiments, we demonstrate that IS-GC can flexibly tolerate an arbitrary number of stragglers and achieve a low completion time of training.
DataVaults: A Secure, Distributed and Privacy Preserving Personal Data Management Platform

With the rapid development of information technology, the ethical use of data and users' privacy has become a big concern. In European Union, the General Data Protection Regulation (GDPR) has been enforced since 2018, aiming to protect a person's privacy. However, the growth of the data economy might be hindered due to the lack of a trusted, secure and privacy-aware tool. Motivated by this observation, this work presents Data Vaults - a secure, distributed and privacy-preserving personal data management platform. The main goal is to mitigate various privacy concerns through allowing an individual to maintain the ownership, handle and share the data based on their willingness. The platform enables a flexible data sharing method with fair compensation schemes. In particular, with the Data Vaults platform, an individual can protect the sharing of personal data and fairly define how value can be captured, created, released and cashed out for the benefit of all the stakeholders involved (companies or not).
Hierarchical Federated Learning with Adaptive Momentum in Multi-Tier Networks

In this paper, we propose and analyze HierAdMo, a three-tier adaptive momentum accelerated client-edge-cloud Federated Learning (FL) algorithm. HierAdMo combines the momentum acceleration on both worker and edge levels. However, simply combining these two levels of momenta may lead to disagreement between them, negatively influencing convergence performance. To this end, we embed an online adaptive method that scales down the momentum when disagreement occurs. We provide mathematical proof for the convergence of HierAdMo for non-i.i.d. data and the tighter convergence upper bound compared with a version of HierAdMo without adaptation (HierAdMo-R). Finally, extensive experiments based on real-world datasets are conducted, verifying that HierAdMo outperforms existing mainstream benchmarks and achieves the optimal or near-optimal convergence performance compared with HierAdMo-R under a wide range of settings.
Embedding Communication for Federated Graph Neural Networks with Privacy Guarantees

Graph Neural Networks (GNNs) have been widely used in many Machine Learning (ML) tasks as they show remarkable performance in modeling graph structure data. While several distributed GNN frameworks have been proposed to tackle the training of huge graphs, uploading the local graph data to the central server for model training is impractical in real-world scenarios due to privacy concerns. Federated Learning (FL) is introduced as an effective technology to address the privacy issue, allowing edge clients to collaboratively train the ML models locally. However, GNNs follow a recursive neighborhood aggregation scheme. Computing the representation vector, also known as embedding, of one node requires aggregating feature vectors of its neighbors. Training the model based on local subgraphs would suffer from information loss and result in accuracy degradation. This paper presents EmbC-FGNN, an efficient Federated _Graph Neural Network framework that enables Node Embedding Communication among training clients in a privacy-preserving way. EmbC-FGNN first proposes an Embedding Server (ES) to maintain and synchronize the shared embeddings among edge workers. It allows training devices to expand the local subgraphs with exchanged embeddings to improve the model accuracy without revealing local node features and graph topology. To minimize the communication costs of the ES, we introduce a periodic embedding synchronization strategy to reduce the communication frequency. Furthermore, we apply asynchronous training to accelerate the convergence speed. Experimental results on several graph neural networks and datasets demonstrate that EmbC-FGNN can improve the overall accuracy (more than 10% for Reddit dataset) and achieve good round-to-accuracy performance.
Demo: Blockchain Application for Hydrogen Production and Distribution Tracing

This paper discusses a blockchain-based system that uses a quantitative tokenized approach for the production and distribution of hydrogen fuel. Currently, the production of hydrogen fuel depends on the supply of water and electricity from external entities. Our decentralized application utilizes a smart contract deployed on a blockchain, enabling the seamless conversion of elements into fuel and the production and distribution process. The paper presents the application's workflow and demonstrates its potential to revolutionize the hydrogen fuel industry. This paper describes the application workflow. The code is available on GitHub.
Decentralized Learning of Bayesian Networks from Private Data with Applications to Global Pandemic

Reasoning under conditions of uncertainty is important in many areas where posterior knowledge depends on prior likelihood estimation from data. Distributed computing can provide a leverage for enhancing Bayesian network (BN) structure learning while keeping data private to users. We propose a decentralized learning framework based on distributed computing of BNs in local sites. Our method yields a higher fitness score (FS) for BNs in local sites, preserves local data privacy and significantly reduces the computation cost by 69.09% in comparison with a centralized approach which yields a low FS score, and does not keep user data private.
Efficient Federated Learning with Adaptive Client-Side Hyper-Parameter Optimization

Federated Learning (FL) trains machine learning (ML) models with privacy protection. However, current FL algorithms use the same hyper-parameters for all clients regardless of their statistical or system heterogeneity, leading to slower convergence. Convergence time and communication rounds may be reduced by using appropriate values for hyper-parameters like learning rate and epochs. We present an adaptive client-side hyper-parameter optimization algorithm, FedAdap, that uses metrics gathered during model training to optimize hyper-parameters on each client and can be used in conjunction with any other FL algorithm. Preliminary results show that FedAdap enhances the performance of existing FL algorithms by decreasing convergence time by up to 34 % and reducing communication rounds by up to 37 % in the case of lID data. Moreover, in non-lID data settings, the convergence time is reduced by up to 82.5 % and the number of communication rounds is reduced by up to 77 %.
A Prototype-Based Knowledge Distillation Framework for Heterogeneous Federated Learning

Federated learning (FL) is an emerging distributed machine learning paradigm, which has shown great potential in collaborative learning with privacy preservation. However, FL clients usually have disparate system resource capabilities (e.g., data, computation, and communication) for model training and aggregation, which can cause a series of system heterogeneity issues with performance degradation. To this end, we propose FedPKD, a Prototype-based Knowledge Distillation framework for FL. FedPKD integrates knowledge distillation and prototype learning with FL, which enables heterogeneous clients and the server to learn collaboratively, with different model architectures and resource capability adaptations. Specifically, FedPKD proposes to transfer dual knowledge of clients including the model output logits and prototypes to the server, and a prototype-based ensemble distillation mechanism is proposed to aggregate the logits and prototypes from clients, which can be used to train the server model with an unlabeled public dataset. The server model knowledge is then transferred back to clients to improve the performance of client models. Moreover, to improve learning performance and reduce communication overhead, we propose a prototype-based data filter mechanism to filter out the samples with low-quality knowledge. Extensive experiments under various settings demonstrate the superiority of FedPKD in learning performance and communication efficiency when compared to state-of-the-art benchmarks.