OpenRaiser/Intern-Atlas huggingface dataset:
Ethereum Proof-of-Stake Consensus Layer: Participation and Decentralization
Venue: arXiv cs.DC 2023
Year: 2023

In September 2022, Ethereum transitioned from Proof-of-Work (PoW) to Proof-of-Stake (PoS) during "the merge" - making it the largest PoS cryptocurrency in terms of market capitalization. With this work, we present a comprehensive measurement study of the current state of the Ethereum PoS consensus layer on the beacon chain. We perform a longitudinal study of the history of the beacon chain. Our work finds that all dips in network participation are caused by network upgrades, issues with major consensus clients, or issues with service operators controlling a large number of validators. Further, our longitudinal staking power decentralization analysis reveals that Ethereum PoS fairs similarly to its PoW counterpart in terms of decentralization and exhibits the immense impact of (liquid) staking services on staking power decentralization. Finally, we highlight the heightened security concerns in Ethereum PoS caused by high degrees of centralization.
SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Faces
Venue: arXiv cs.CV 2023
Year: 2023

Speech-driven 3D face animation technique, extending its applications to various multimedia fields. Previous research has generated promising realistic lip movements and facial expressions from audio signals. However, traditional regression models solely driven by data face several essential problems, such as difficulties in accessing precise labels and domain gaps between different modalities, leading to unsatisfactory results lacking precision and coherence. To enhance the visual accuracy of generated lip movement while reducing the dependence on labeled data, we propose a novel framework SelfTalk, by involving self-supervision in a cross-modals network system to learn 3D talking faces. The framework constructs a network system consisting of three modules: facial animator, speech recognizer, and lip-reading interpreter. The core of SelfTalk is a commutative training diagram that facilitates compatible features exchange among audio, text, and lip shape, enabling our models to learn the intricate connection between these factors. The proposed framework leverages the knowledge learned from the lip-reading interpreter to generate more plausible lip shapes. Extensive experiments and user studies demonstrate that our proposed approach achieves state-of-the-art performance both qualitatively and quantitatively. We recommend watching the supplementary video.
Practical First-Order Bayesian Optimization Algorithms
Venue: arXiv cs.LG 2023
Year: 2023

First Order Bayesian Optimization (FOBO) is a sample efficient sequential approach to find the global maxima of an expensive-to-evaluate black-box objective function by suitably querying for the function and its gradient evaluations. Such methods assume Gaussian process (GP) models for both, the function and its gradient, and use them to construct an acquisition function that identifies the next query point. In this paper, we propose a class of practical FOBO algorithms that efficiently utilizes the information from the gradient GP to identify potential query points with zero gradients. We construct a multi-level acquisition function where in the first step, we optimize a lower level acquisition function with multiple restarts to identify potential query points with zero gradient value. We then use the upper level acquisition function to rank these query points based on their function values to potentially identify the global maxima. As a final step, the potential point of maxima is chosen as the actual query point. We validate the performance of our proposed algorithms on several test functions and show that our algorithms outperform state-of-the-art FOBO algorithms. We also illustrate the application of our algorithms in finding optimal set of hyper-parameters in machine learning and in learning the optimal policy in reinforcement learning tasks.
Pneumatic bellows actuated parallel platform control with adjustable stiffness using a hybrid feed-forward and variable gain I-controller
Venue: arXiv cs.RO 2023
Year: 2023

Redundant cascade manipulators actuated by pneumatic bellows actuators are passively compliant, rugged and dexterous which are qualities making them exceptionally well suited for applications in agriculture. Unfortunately bellows actuators are notoriously difficult to precisely position. This paper presents a novel control algorithm for the control of a parallel platform actuated by pneumatic bellows actuators, which is serving as one module of a cascade manipulator. The algorithm combines a feed-forward controller and a variable gain I-controller. The feed-forward controller was designed using experimental data and two regression steps to create a mathematical representation of the data. The gain of the I-controller depends linearly on the total reference error, which allows the I-controller to work in concert with the feed-forward part of the controller. The presented algorithm was experimentally verified and its performance was compared with two controllers, an ANFIS controller and a constant gain PID controller, to satisfactory results. The controller was also tested under dynamic loading conditions showing promising results.
Blockchain-Enabled Federated Learning: A Reference Architecture Design, Implementation, and Verification
Venue: arXiv cs.LG 2023
Year: 2023

This paper presents a novel reference architecture for blockchain-enabled federated learning (BCFL), a state-of-the-art approach that amalgamates the strengths of federated learning and blockchain technology.We define smart contract functions, stakeholders and their roles, and the use of interplanetary file system (IPFS) as key components of BCFL and conduct a comprehensive analysis. In traditional centralized federated learning, the selection of local nodes and the collection of learning results for each round are merged under the control of a central server. In contrast, in BCFL, all these processes are monitored and managed via smart contracts. Additionally, we propose an extension architecture to support both crossdevice and cross-silo federated learning scenarios. Furthermore, we implement and verify the architecture in a practical real-world Ethereum development environment. Our BCFL reference architecture provides significant flexibility and extensibility, accommodating the integration of various additional elements, as per specific requirements and use cases, thereby rendering it an adaptable solution for a wide range of BCFL applications. As a prominent example of extensibility, decentralized identifiers (DIDs) have been employed as an authentication method to introduce practical utilization within BCFL. This study not only bridges a crucial gap between research and practical deployment but also lays a solid foundation for future explorations in the realm of BCFL. The pivotal contribution of this study is the successful implementation and verification of a realistic BCFL reference architecture. We intend to make the source code publicly accessible shortly, fostering further advancements and adaptations within the community.
Full-Duplex-Enabled Joint Communications and Sensing with Reconfigurable Intelligent Surfaces
Venue: arXiv cs.IT 2023
Year: 2023

The full-duplex (FD) technology has the potential to radically evolve wireless systems, facilitating the integration of both communications and radar functionalities into a single device, thus, enabling joint communication and sensing (JCAS). In this paper, we present a novel approach for JCAS that incorporates a reconfigurable intelligent surface (RIS) in the near-field of an FD multiple-input multiple-output (MIMO) node, which is jointly optimized with the digital beamformers to enable JSAC and efficiently handle self-interference (SI). We propose a novel problem formulation for FD MIMO JCAS systems to jointly minimize the total received power at the FD node's radar receiver while maximizing the sum rate of downlink communications subject to a Cramér-Rao bound (CRB) constraint. In contrast to the typically used CRB in the relevant literature, we derive a novel, more accurate, target estimation bound that fully takes into account the RIS deployment. The considered problem is solved using alternating optimization, which is guaranteed to converge to a local optimum. The simulation results demonstrate that the proposed scheme achieves significant performance improvement both for communications and sensing. It is showcased that, jointly designing the FD MIMO beamformers and the RIS phase configuration to be SI aware can significantly loosen the requirement for additional SI cancellation.
Vision Transformer with Attention Map Hallucination and FFN Compaction
Venue: arXiv cs.CV 2023
Year: 2023

Vision Transformer(ViT) is now dominating many vision tasks. The drawback of quadratic complexity of its token-wise multi-head self-attention (MHSA), is extensively addressed via either token sparsification or dimension reduction (in spatial or channel). However, the therein redundancy of MHSA is usually overlooked and so is the feed-forward network (FFN). To this end, we propose attention map hallucination and FFN compaction to fill in the blank. Specifically, we observe similar attention maps exist in vanilla ViT and propose to hallucinate half of the attention maps from the rest with much cheaper operations, which is called hallucinated-MHSA (hMHSA). As for FFN, we factorize its hidden-to-output projection matrix and leverage the re-parameterization technique to strengthen its capability, making it compact-FFN (cFFN). With our proposed modules, a 10$\%$-20$\%$ reduction of floating point operations (FLOPs) and parameters (Params) is achieved for various ViT-based backbones, including straight (DeiT), hybrid (NextViT) and hierarchical (PVT) structures, meanwhile, the performances are quite competitive.
Vocal Timbre Effects with Differentiable Digital Signal Processing
Venue: arXiv cs.SD 2023
Year: 2023

We explore two approaches to creatively altering vocal timbre using Differentiable Digital Signal Processing (DDSP). The first approach is inspired by classic cross-synthesis techniques. A pretrained DDSP decoder predicts a filter for a noise source and a harmonic distribution, based on pitch and loudness information extracted from the vocal input. Before synthesis, the harmonic distribution is modified by interpolating between the predicted distribution and the harmonics of the input. We provide a real-time implementation of this approach in the form of a Neutone model. In the second approach, autoencoder models are trained on datasets consisting of both vocal and instrument training data. To apply the effect, the trained autoencoder attempts to reconstruct the vocal input. We find that there is a desirable "sweet spot" during training, where the model has learned to reconstruct the phonetic content of the input vocals, but is still affected by the timbre of the instrument mixed into the training data. After further training, that effect disappears. A perceptual evaluation compares the two approaches. We find that the autoencoder in the second approach is able to reconstruct intelligible lyrical content without any explicit phonetic information provided during training.
QoS-Aware Downlink Beamforming for Joint Transmission in Multi-Cell Networks
Venue: arXiv cs.IT 2023
Year: 2023

Multi-cell cooperation is an effective means to improve service quality to cellular users. Existing work primarily focuses on interference cancellation using all the degrees of freedom (DoF). This leads to low service quality for some users with poor channel quality to its serving base station. This work investigates the multi-cell beamforming design for simultaneously enhancing the downlink signal strength and mitigating interference. We first consider the ideal case when perfect channel state information (CSI) is available for determining the beamforming vectors and then extend to the case of imperfect CSI. For both cases, the beamforming optimization problems are non-convex. Assuming perfect CSI, we obtain the optimal joint transmit (JT) beamforming vectors based on the uplink-downlink duality. In the presence of unknown CSI errors, we use the semidefinite relaxation (SDR) with Bernstein-type inequality to derive the robust JT beamforming. Numerical results are presented to evaluate the performance of the proposed schemes.
Knowledge Transfer-Driven Few-Shot Class-Incremental Learning
Venue: arXiv cs.CV 2023
Year: 2023

Few-shot class-incremental learning (FSCIL) aims to continually learn new classes using a few samples while not forgetting the old classes. The key of this task is effective knowledge transfer from the base session to the incremental sessions. Despite the advance of existing FSCIL methods, the proposed knowledge transfer learning schemes are sub-optimal due to the insufficient optimization for the model's plasticity. To address this issue, we propose a Random Episode Sampling and Augmentation (RESA) strategy that relies on diverse pseudo incremental tasks as agents to achieve the knowledge transfer. Concretely, RESA mimics the real incremental setting and constructs pseudo incremental tasks globally and locally, where the global pseudo incremental tasks are designed to coincide with the learning objective of FSCIL and the local pseudo incremental tasks are designed to improve the model's plasticity, respectively. Furthermore, to make convincing incremental predictions, we introduce a complementary model with a squared Euclidean-distance classifier as the auxiliary module, which couples with the widely used cosine classifier to form our whole architecture. By such a way, equipped with model decoupling strategy, we can maintain the model's stability while enhancing the model's plasticity. Extensive quantitative and qualitative experiments on three popular FSCIL benchmark datasets demonstrate that our proposed method, named Knowledge Transfer-driven Relation Complementation Network (KT-RCNet), outperforms almost all prior methods. More precisely, the average accuracy of our proposed KT-RCNet outperforms the second-best method by a margin of 5.26%, 3.49%, and 2.25% on miniImageNet, CIFAR100, and CUB200, respectively. Our code is available at https://github.com/YeZiLaiXi/KT-RCNet.git.
Semi-Supervised Learning for hyperspectral images by non parametrically predicting view assignment
Venue: arXiv cs.CV 2023
Year: 2023

Hyperspectral image (HSI) classification is gaining a lot of momentum in present time because of high inherent spectral information within the images. However, these images suffer from the problem of curse of dimensionality and usually require a large number samples for tasks such as classification, especially in supervised setting. Recently, to effectively train the deep learning models with minimal labelled samples, the unlabeled samples are also being leveraged in self-supervised and semi-supervised setting. In this work, we leverage the idea of semi-supervised learning to assist the discriminative self-supervised pretraining of the models. The proposed method takes different augmented views of the unlabeled samples as input and assigns them the same pseudo-label corresponding to the labelled sample from the downstream task. We train our model on two HSI datasets, namely Houston dataset (from data fusion contest, 2013) and Pavia university dataset, and show that the proposed approach performs better than self-supervised approach and supervised training.
Understanding the Challenges of Deploying Live-Traceability Solutions
Venue: arXiv cs.SE 2023
Year: 2023

Software traceability is the process of establishing and maintaining relationships between artifacts in a software system. This process is crucial to many engineering processes, particularly for safety critical projects; however, it is labor-intensive and error-prone. Automated traceability has been a long awaited tool for project managers of these systems, and due to the semantic similarities between linked artifacts, NLP techniques, such as transformer models, may be leveraged to accomplish this task. SAFA.ai is a startup focusing on fine-tuning project-specific models that deliver automated traceability in a near real-time environment. The following paper describes the challenges that characterize commercializing software traceability and highlights possible future directions.
Spatial Secrecy Spectral Efficiency Optimization Enabled by Reconfigurable Intelligent Surfaces
Venue: arXiv cs.IT 2023
Year: 2023

Reconfigurable Intelligent Surfaces (RISs) constitute a strong candidate physical-layer technology for the $6$-th Generation (6G) of wireless networks, offering new design degrees of freedom for efficiently addressing demanding performance objectives. In this paper, we consider a Multiple-Input Single-Output (MISO) physical-layer security system incorporating a reflective RIS to safeguard wireless communications between a legitimate transmitter and receiver under the presence of an eavesdropper. In contrast to current studies optimizing RISs for given positions of the legitimate and eavesdropping nodes, in this paper, we focus on devising RIS-enabled secrecy for given geographical areas of potential nodes' placement. We propose a novel secrecy metric, capturing the spatially averaged secrecy spectral efficiency, and present a joint design of the transmit digital beamforming and the RIS analog phase profile, which is realized via a combination of alternating optimization and minorization-maximization. The proposed framework bypasses the need for instantaneous knowledge of the eavesdropper's channel or position, and targets providing an RIS-boosted secure area of legitimate communications with a single configuration of the free parameters. Our simulation results showcase significant performance gains with the proposed secrecy scheme, even for cases where the eavesdropper shares similar pathloss attenuation with the legitimate receiver.
UncLe-SLAM: Uncertainty Learning for Dense Neural SLAM
Venue: arXiv cs.CV 2023
Year: 2023

We present an uncertainty learning framework for dense neural simultaneous localization and mapping (SLAM). Estimating pixel-wise uncertainties for the depth input of dense SLAM methods allows re-weighing the tracking and mapping losses towards image regions that contain more suitable information that is more reliable for SLAM. To this end, we propose an online framework for sensor uncertainty estimation that can be trained in a self-supervised manner from only 2D input data. We further discuss the advantages of the uncertainty learning for the case of multi-sensor input. Extensive analysis, experimentation, and ablations show that our proposed modeling paradigm improves both mapping and tracking accuracy and often performs better than alternatives that require ground truth depth or 3D. Our experiments show that we achieve a 38\% and 27\% lower absolute trajectory tracking error (ATE) on the 7-Scenes and TUM-RGBD datasets respectively. On the popular Replica dataset using two types of depth sensors, we report an 11\% F1-score improvement on RGBD SLAM compared to the recent state-of-the-art neural implicit approaches. Source code: https://github.com/kev-in-ta/UncLe-SLAM.
Parallel Data Compression Techniques
Venue: arXiv cs.DC 2023
Year: 2023

With endless amounts of data and very limited bandwidth, fast data compression is one solution for the growing datasharing problem. Compression helps lower transfer times and save memory, but if the compression takes too long, this no longer seems viable. Multi-core processors enable parallel data compression; however, parallelizing the algorithms is anything but straightforward since compression is inherently serial. This paper explores techniques to parallelize three compression schemes: Huffman coding, LZSS, and MP3 coding
Causal Effect Regularization: Automated Detection and Removal of Spurious Attributes
Venue: arXiv cs.LG 2023
Year: 2023

In many classification datasets, the task labels are spuriously correlated with some input attributes. Classifiers trained on such datasets often rely on these attributes for prediction, especially when the spurious correlation is high, and thus fail to generalize whenever there is a shift in the attributes' correlation at deployment. If we assume that the spurious attributes are known a priori, several methods have been proposed to learn a classifier that is invariant to the specified attributes. However, in real-world data, information about spurious attributes is typically unavailable. Therefore, we propose a method to automatically identify spurious attributes by estimating their causal effect on the label and then use a regularization objective to mitigate the classifier's reliance on them. Compared to a recent method for identifying spurious attributes, we find that our method is more accurate in removing the attribute from the learned model, especially when spurious correlation is high. Specifically, across synthetic, semi-synthetic, and real-world datasets, our method shows significant improvement in a metric used to quantify the dependence of a classifier on spurious attributes ($Δ$Prob), while obtaining better or similar accuracy. In addition, our method mitigates the reliance on spurious attributes even under noisy estimation of causal effects. To explain the empirical robustness of our method, we create a simple linear classification task with two sets of attributes: causal and spurious. We prove that our method only requires that the ranking of estimated causal effects is correct across attributes to select the correct classifier.
Fairness-aware Message Passing for Graph Neural Networks
Venue: arXiv cs.LG 2023
Year: 2023

Graph Neural Networks (GNNs) have shown great power in various domains. However, their predictions may inherit societal biases on sensitive attributes, limiting their adoption in real-world applications. Although many efforts have been taken for fair GNNs, most existing works just adopt widely used fairness techniques in machine learning to graph domains and ignore or don't have a thorough understanding of the message passing mechanism with fairness constraints, which is a distinctive feature of GNNs. To fill the gap, we propose a novel fairness-aware message passing framework GMMD, which is derived from an optimization problem that considers both graph smoothness and representation fairness. GMMD can be intuitively interpreted as encouraging a node to aggregate representations of other nodes from different sensitive groups while subtracting representations of other nodes from the same sensitive group, resulting in fair representations. We also provide a theoretical analysis to justify that GMMD can guarantee fairness, which leads to a simpler and theory-guided variant GMMD-S. Extensive experiments on graph benchmarks show that our proposed framework can significantly improve the fairness of various backbone GNN models while maintaining high accuracy.