OpenRaiser/Intern-Atlas huggingface dataset:
Accelerating Parallel Stochastic Gradient Descent via Non-blocking Mini-batches
Venue: arXiv cs.LG 2022
Year: 2022

SOTA decentralized SGD algorithms can overcome the bandwidth bottleneck at the parameter server by using communication collectives like Ring All-Reduce for synchronization. While the parameter updates in distributed SGD may happen asynchronously there is still a synchronization barrier to make sure that the local training epoch at every learner is complete before the learners can advance to the next epoch. The delays in waiting for the slowest learners(stragglers) remain to be a problem in the synchronization steps of these state-of-the-art decentralized frameworks. In this paper, we propose the (de)centralized Non-blocking SGD (Non-blocking SGD) which can address the straggler problem in a heterogeneous environment. The main idea of Non-blocking SGD is to split the original batch into mini-batches, then accumulate the gradients and update the model based on finished mini-batches. The Non-blocking idea can be implemented using decentralized algorithms including Ring All-reduce, D-PSGD, and MATCHA to solve the straggler problem. Moreover, using gradient accumulation to update the model also guarantees convergence and avoids gradient staleness. Run-time analysis with random straggler delays and computational efficiency/throughput of devices is also presented to show the advantage of Non-blocking SGD. Experiments on a suite of datasets and deep learning networks validate the theoretical analyses and demonstrate that Non-blocking SGD speeds up the training and fastens the convergence. Compared with the state-of-the-art decentralized asynchronous algorithms like D-PSGD and MACHA, Non-blocking SGD takes up to 2x fewer time to reach the same training loss in a heterogeneous environment.
New quantum codes from self-dual codes over F_4
Venue: arXiv cs.IT 2022
Year: 2022

We present new constructions of binary quantum codes from quaternary linear Hermitian self-dual codes. Our main ingredients for these constructions are nearly self-orthogonal cyclic or duadic codes over F_4. An infinite family of $0$-dimensional binary quantum codes is provided. We give minimum distance lower bounds for our quantum codes in terms of the minimum distance of their ingredient linear codes. We also present new results on the minimum distance of linear cyclic codes using their fixed subcodes. Finally, we list many new record-breaking quantum codes obtained from our constructions.
Fast-U2++: Fast and Accurate End-to-End Speech Recognition in Joint CTC/Attention Frames
Venue: arXiv cs.SD 2022
Year: 2022

Recently, the unified streaming and non-streaming two-pass (U2/U2++) end-to-end model for speech recognition has shown great performance in terms of streaming capability, accuracy and latency. In this paper, we present fast-U2++, an enhanced version of U2++ to further reduce partial latency. The core idea of fast-U2++ is to output partial results of the bottom layers in its encoder with a small chunk, while using a large chunk in the top layers of its encoder to compensate the performance degradation caused by the small chunk. Moreover, we use knowledge distillation method to reduce the token emission latency. We present extensive experiments on Aishell-1 dataset. Experiments and ablation studies show that compared to U2++, fast-U2++ reduces model latency from 320ms to 80ms, and achieves a character error rate (CER) of 5.06% with a streaming setup.
Low-Cost Traffic Sensing System Based on LoRaWAN for Urban Areas
Venue: arXiv cs.NI 2022
Year: 2022

The advent of Low Power Wide Area Networks (LPWAN) has enabled the feasibility of wireless sensor networks for environmental traffic sensing across urban areas. In this study, we explore the usage of LoRaWAN end nodes as traffic sensing sensors to offer a practical traffic management solution. The monitored Received Signal Strength Indicator (RSSI) factor is reported and used in the gateways to assess the traffic of the environment. Our technique utilizes LoRaWAN as a long-range communication technology to provide a largescale system. In this work, we present a method of using LoRaWAN devices to estimate traffic flows. LoRaWAN end devices then transmit their packets to different gateways. Their RSSI will be affected by the number of cars present on the roadway. We used SVM and clustering methods to classify the approximate number of cars present. This paper details our experiences with the design and real implementation of this system across an area that stretches for miles in urban scenarios. We continuously measured and reported RSSI at different gateways for weeks. Results have shown that if a LoRaWAN end node is placed in an optimal position, up to 96% of correct environment traffic level detection can be obtained. Additionally, we share the l
Holographic-Type Communication for Digital Twin: A Learning-based Auction Approach
Venue: arXiv cs.NI 2022
Year: 2022

Digital Twin (DT) technologies, which aim to build digital replicas of physical entities, are the key to providing efficient, concurrent simulation and analysis of real-world objects. In displaying DTs, Holographic-Type Communication (HTC), which supports the transmission of holographic data such as Light Field (LF), can provide an immersive way for users to interact with Holographic DTs (HDT). However, it is challenging to effectively allocate interactive and resource-intensive HDT services among HDT users and providers. In this paper, we integrate the paradigms of HTC and DT to form a HTC for DT system, design a marketplace for HDT services where HDT users' and providers' prices are evaluated by their valuation functions, and propose an auction-based mechanism to match HDT services using a learning-based Double Dutch Auction (DDA). Specifically, we apply DDA and train an agent acting as the auctioneer to adjust the auction clock dynamically using Deep Reinforcement Learning (DRL), aiming to achieve the best market efficiency. Simulation results demonstrate that the proposed learning-based auctioneer can achieve near-optimal social welfare at halved auction information exchange cost of the baseline method.
Fantasizing with Dual GPs in Bayesian Optimization and Active Learning
Venue: arXiv cs.LG 2022
Year: 2022

Gaussian processes (GPs) are the main surrogate functions used for sequential modelling such as Bayesian Optimization and Active Learning. Their drawbacks are poor scaling with data and the need to run an optimization loop when using a non-Gaussian likelihood. In this paper, we focus on `fantasizing' batch acquisition functions that need the ability to condition on new fantasized data computationally efficiently. By using a sparse Dual GP parameterization, we gain linear scaling with batch size as well as one-step updates for non-Gaussian likelihoods, thus extending sparse models to greedy batch fantasizing acquisition functions.
Gradient Knowledge Distillation for Pre-trained Language Models
Venue: arXiv cs.CL 2022
Year: 2022

Knowledge distillation (KD) is an effective framework to transfer knowledge from a large-scale teacher to a compact yet well-performing student. Previous KD practices for pre-trained language models mainly transfer knowledge by aligning instance-wise outputs between the teacher and student, while neglecting an important knowledge source, i.e., the gradient of the teacher. The gradient characterizes how the teacher responds to changes in inputs, which we assume is beneficial for the student to better approximate the underlying mapping function of the teacher. Therefore, we propose Gradient Knowledge Distillation (GKD) to incorporate the gradient alignment objective into the distillation process. Experimental results show that GKD outperforms previous KD methods regarding student performance. Further analysis shows that incorporating gradient knowledge makes the student behave more consistently with the teacher, improving the interpretability greatly.
Generative Poisoning Using Random Discriminators
Venue: arXiv cs.LG 2022
Year: 2022

We introduce ShortcutGen, a new data poisoning attack that generates sample-dependent, error-minimizing perturbations by learning a generator. The key novelty of ShortcutGen is the use of a randomly-initialized discriminator, which provides spurious shortcuts needed for generating poisons. Different from recent, iterative methods, our ShortcutGen can generate perturbations with only one forward pass in a label-free manner, and compared to the only existing generative method, DeepConfuse, our ShortcutGen is faster and simpler to train while remaining competitive. We also demonstrate that integrating a simple augmentation strategy can further boost the robustness of ShortcutGen against early stopping, and combining augmentation and non-augmentation leads to new state-of-the-art results in terms of final validation accuracy, especially in the challenging, transfer scenario. Lastly, we speculate, through uncovering its working mechanism, that learning a more general representation space could allow ShortcutGen to work for unseen data.
Improving transferability of 3D adversarial attacks with scale and shear transformations
Venue: arXiv cs.CV 2022
Year: 2022

Previous work has shown that 3D point cloud classifiers can be vulnerable to adversarial examples. However, most of the existing methods are aimed at white-box attacks, where the parameters and other information of the classifiers are known in the attack, which is unrealistic for real-world applications. In order to improve the attack performance of the black-box classifiers, the research community generally uses the transfer-based black-box attack. However, the transferability of current 3D attacks is still relatively low. To this end, this paper proposes Scale and Shear (SS) Attack to generate 3D adversarial examples with strong transferability. Specifically, we randomly scale or shear the input point cloud, so that the attack will not overfit the white-box model, thereby improving the transferability of the attack. Extensive experiments show that the SS attack proposed in this paper can be seamlessly combined with the existing state-of-the-art (SOTA) 3D point cloud attack methods to form more powerful attack methods, and the SS attack improves the transferability over 3.6 times compare to the baseline. Moreover, while substantially outperforming the baseline methods, the SS attack achieves SOTA transferability under various defenses. Our code will be available online at https://github.com/cuge1995/SS-attack
The Impostor Among US(B): Off-Path Injection Attacks on USB Communications
Venue: arXiv cs.CR 2022
Year: 2022

USB is the most prevalent peripheral interface in modern computer systems and its inherent insecurities make it an appealing attack vector. A well-known limitation of USB is that traffic is not encrypted. This allows on-path adversaries to trivially perform man-in-the-middle attacks. Off-path attacks that compromise the confidentiality of communications have also been shown to be possible. However, so far no off-path attacks that breach USB communications integrity have been demonstrated. In this work we show that the integrity of USB communications is not guaranteed even against off-path attackers.Specifically, we design and build malicious devices that, even when placed outside of the path between a victim device and the host, can inject data to that path. Using our developed injectors we can falsify the provenance of data input as interpreted by a host computer system. By injecting on behalf of trusted victim devices we can circumvent any software-based authorisation policy defences that computer systems employ against common USB attacks. We demonstrate two concrete attacks. The first injects keystrokes allowing an attacker to execute commands. The second demonstrates file-contents replacement including during system install from a USB disk. We test the attacks on 29 USB 2.0 and USB 3.x hubs and find 14 of them to be vulnerable.
Safety-centric and Smart Outdoor Workplace: A New Research Direction and Its Technical Challenges
Venue: arXiv cs.HC 2022
Year: 2022

Despite the fact that outside is becoming the frontier of indoor workplaces, a large amount of real-world work like road construction has to be done by outdoor human activities in open areas. Given the promise of the smart workplace in various aspects including productivity and safety, we decided to employ smart workplace technologies for a collaborative outdoor project both to improve the work efficiency and to reduce the worker injuries. Nevertheless, our trials on smart workplace implementation have encountered a few problems ranging from the theoretical confusion among different stakeholders, to the technical difficulties in extending underground devices' lifespan. This triggers our rethinking of and discussions about "smart workplace". Eventually, considering the unique characteristics of outdoor work (e.g., more sophisticated workflows and more safety-related situations than office work), we argue that "safety-centric and smart outdoor workplace" deserves dedicated research attentions and efforts under the umbrella discipline of smart environment. In addition, the identified technical challenges can in turn drive different research dimensions of such a distinguishing topic.
Lifted Inference with Linear Order Axiom
Venue: arXiv cs.AI 2022
Year: 2022

We consider the task of weighted first-order model counting (WFOMC) used for probabilistic inference in the area of statistical relational learning. Given a formula $ϕ$, domain size $n$ and a pair of weight functions, what is the weighted sum of all models of $ϕ$ over a domain of size $n$? It was shown that computing WFOMC of any logical sentence with at most two logical variables can be done in time polynomial in $n$. However, it was also shown that the task is $\texttt{#}P_1$-complete once we add the third variable, which inspired the search for extensions of the two-variable fragment that would still permit a running time polynomial in $n$. One of such extension is the two-variable fragment with counting quantifiers. In this paper, we prove that adding a linear order axiom (which forces one of the predicates in $ϕ$ to introduce a linear ordering of the domain elements in each model of $ϕ$) on top of the counting quantifiers still permits a computation time polynomial in the domain size. We present a new dynamic programming-based algorithm which can compute WFOMC with linear order in time polynomial in $n$, thus proving our primary claim.
Multicast Beamformer Design for MIMO Coded Caching Systems
Venue: arXiv cs.IT 2022
Year: 2022

Coded caching (CC) techniques have been shown to be conveniently applicable in multi-input multi-output (MIMO) systems. In a $K$-user network with spatial multiplexing gains of $L$ at the transmitter and $G$ at every receiver, if each user can cache a fraction $γ$ of the file library, a total number of $GKγ+ L$ data streams can be served in parallel. In this paper, we focus on improving the finite-SNR performance of MIMO-CC systems. We first consider a MIMO-CC scheme that relies only on unicasting individual data streams, and then, introduce a decomposition strategy to design a new scheme that delivers the same data streams through multicasting of $G$ parallel codewords. We discuss how optimized beamformers could be designed for each scheme and use numerical simulations to compare their finite-SNR performance. It is shown that while both schemes serve the same number of streams, multicasting provides notable performance improvements. This is because, with multicasting, transmission vectors are built with fewer beamformers, leading to more efficient usage of available power resources.
AmiGo: Computational Design of Amigurumi Crochet Patterns
Venue: arXiv cs.GR 2022
Year: 2022

We propose an approach for generating crochet instructions (patterns) from an input 3D model. We focus on Amigurumi, which are knitted stuffed toys. Given a closed triangle mesh, and a single point specified by the user, we generate crochet instructions, which when knitted and stuffed result in a toy similar to the input geometry. Our approach relies on constructing the geometry and connectivity of a Crochet Graph, which is then translated into a crochet pattern. We segment the shape automatically into chrochetable components, which are connected using the join-as-you-go method, requiring no additional sewing. We demonstrate that our method is applicable to a large variety of shapes and geometries, and yields easily crochetable patterns.
MDS Variable Generation and Secure Summation with User Selection
Venue: arXiv cs.IT 2022
Year: 2022

A collection of $K$ random variables are called $(K,n)$-MDS if any $n$ of the $K$ variables are independent and determine all remaining variables. In the MDS variable generation problem, $K$ users wish to generate variables that are $(K,n)$-MDS using a randomness variable owned by each user. We show that to generate $1$ bit of $(K,n)$-MDS variables for each $n \in \{1,2,\cdots, K\}$, the minimum size of the randomness variable at each user is $1 + 1/2 + \cdots + 1/K$ bits. An intimately related problem is secure summation with user selection, where a server may select an arbitrary subset of $K$ users and securely compute the sum of the inputs of the selected users. We show that to compute $1$ bit of an arbitrarily chosen sum securely, the minimum size of the key held by each user is $1 + 1/2 + \cdots + 1/(K-1)$ bits, whose achievability uses the generation of $(K,n)$-MDS variables for $n \in \{1,2,\cdots,K-1\}$.
Fair Visual Recognition via Intervention with Proxy Features
Venue: arXiv cs.LG 2022
Year: 2022

Deep learning models often learn to make predictions that rely on sensitive social attributes like gender and race, which poses significant fairness risks, especially in societal applications, e.g., hiring, banking, and criminal justice. Existing work tackles this issue by minimizing information about social attributes in models for debiasing. However, the high correlation between target task and social attributes makes bias mitigation incompatible with target task accuracy. Recalling that model bias arises because the learning of features in regard to bias attributes (i.e., bias features) helps target task optimization, we explore the following research question: \emph{Can we leverage proxy features to replace the role of bias feature in target task optimization for debiasing?} To this end, we propose \emph{Proxy Debiasing}, to first transfer the target task's learning of bias information from bias features to artificial proxy features, and then employ causal intervention to eliminate proxy features in inference. The key idea of \emph{Proxy Debiasing} is to design controllable proxy features to on one hand replace bias features in contributing to target task during the training stage, and on the other hand easily to be removed by intervention during the inference stage. This guarantees the elimination of bias features without affecting the target information, thus addressing the fairness-accuracy paradox in previous debiasing solutions. We apply \emph{Proxy Debiasing} to several benchmark datasets, and achieve significant improvements over the state-of-the-art debiasing methods in both of accuracy and fairness.
CircleSnake: Instance Segmentation with Circle Representation
Venue: arXiv cs.CV 2022
Year: 2022

Circle representation has recently been introduced as a medical imaging optimized representation for more effective instance object detection on ball-shaped medical objects. With its superior performance on instance detection, it is appealing to extend the circle representation to instance medical object segmentation. In this work, we propose CircleSnake, a simple end-to-end circle contour deformation-based segmentation method for ball-shaped medical objects. Compared to the prevalent DeepSnake method, our contribution is three-fold: (1) We replace the complicated bounding box to octagon contour transformation with a computation-free and consistent bounding circle to circle contour adaption for segmenting ball-shaped medical objects; (2) Circle representation has fewer degrees of freedom (DoF=2) as compared with the octagon representation (DoF=8), thus yielding a more robust segmentation performance and better rotation consistency; (3) To the best of our knowledge, the proposed CircleSnake method is the first end-to-end circle representation deep segmentation pipeline method with consistent circle detection, circle contour proposal, and circular convolution. The key innovation is to integrate the circular graph convolution with circle detection into an end-to-end instance segmentation framework, enabled by the proposed simple and consistent circle contour representation. Glomeruli are used to evaluate the performance of the benchmarks. From the results, CircleSnake increases the average precision of glomerular detection from 0.559 to 0.614. The Dice score increased from 0.804 to 0.849. The code has been released: https://github.com/hrlblab/CircleSnake
Secure and Efficient Privacy-preserving Authentication Scheme using Cuckoo Filter in Remote Patient Monitoring Network
Venue: arXiv cs.CR 2022
Year: 2022

With the ubiquitous advancement in smart medical devices and systems, the potential of Remote Patient Monitoring (RPM) network is evolving in modern healthcare systems. The medical professionals (doctors, nurses, or medical experts) can access vitals and sensitive physiological information about the patients and provide proper treatment to improve the quality of life through the RPM network. However, the wireless nature of communication in the RPM network makes it challenging to design an efficient mechanism for secure communication. Many authentication schemes have been proposed in recent years to ensure the security of the RPM network. Pseudonym, digital signature, and Authenticated Key Exchange (AKE) protocols are used for the Internet of Medical Things (IoMT) to develop secure authorization and privacy-preserving communication. However, traditional authentication protocols face overhead challenges due to maintaining a large set of key-pairs or pseudonyms results on the hospital cloud server. In this research work, we identify this research gap and propose a novel secure and efficient privacy-preserving authentication scheme using cuckoo filters for the RPM network. The use of cuckoo filters in our proposed scheme provides an efficient way for mutual anonymous authentication and a secret shared key establishment process between medical professionals and patients. Moreover, we identify the misbehaving sensor nodes using a correlation-based anomaly detection model to establish secure communication. The security analysis and formal security validation using SPAN and AVISPA tools show the robustness of our proposed scheme against message modification attacks, replay attacks, and man-in-the-middle attacks.
Boosting word frequencies in authorship attribution
Venue: arXiv cs.CL 2022
Year: 2022

In this paper, I introduce a simple method of computing relative word frequencies for authorship attribution and similar stylometric tasks. Rather than computing relative frequencies as the number of occurrences of a given word divided by the total number of tokens in a text, I argue that a more efficient normalization factor is the total number of relevant tokens only. The notion of relevant words includes synonyms and, usually, a few dozen other words in some ways semantically similar to a word in question. To determine such a semantic background, one of word embedding models can be used. The proposed method outperforms classical most-frequent-word approaches substantially, usually by a few percentage points depending on the input settings.