OpenRaiser/Intern-Atlas huggingface dataset:
Towards Cross-Granularity Few-Shot Learning: Coarse-to-Fine Pseudo-Labeling with Visual-Semantic Meta-Embedding
Venue: arXiv cs.CV 2021
Year: 2021

Few-shot learning aims at rapidly adapting to novel categories with only a handful of samples at test time, which has been predominantly tackled with the idea of meta-learning. However, meta-learning approaches essentially learn across a variety of few-shot tasks and thus still require large-scale training data with fine-grained supervision to derive a generalized model, thereby involving prohibitive annotation cost. In this paper, we advance the few-shot classification paradigm towards a more challenging scenario, i.e., cross-granularity few-shot classification, where the model observes only coarse labels during training while is expected to perform fine-grained classification during testing. This task largely relieves the annotation cost since fine-grained labeling usually requires strong domain-specific expertise. To bridge the cross-granularity gap, we approximate the fine-grained data distribution by greedy clustering of each coarse-class into pseudo-fine-classes according to the similarity of image embeddings. We then propose a meta-embedder that jointly optimizes the visual- and semantic-discrimination, in both instance-wise and coarse class-wise, to obtain a good feature space for this coarse-to-fine pseudo-labeling process. Extensive experiments and ablation studies are conducted to demonstrate the effectiveness and robustness of our approach on three representative datasets.
CellEVAC: An adaptive guidance system for crowd evacuation through behavioral optimization
Venue: arXiv cs.MA 2021
Year: 2021

A critical aspect of crowds' evacuation processes is the dynamism of individual decision making. Here, we investigate how to favor a coordinated group dynamic through optimal exit-choice instructions using behavioral strategy optimization. We propose and evaluate an adaptive guidance system (Cell-based Crowd Evacuation, CellEVAC) that dynamically allocates colors to cells in a cell-based pedestrian positioning infrastructure, to provide efficient exit-choice indications. The operational module of CellEVAC implements an optimized discrete-choice model that integrates the influential factors that would make evacuees adapt their exit choice. To optimize the model, we used a simulation-optimization modeling framework that integrates microscopic pedestrian simulation based on the classical Social Force Model. We paid particular attention to safety by using Pedestrian Fundamental Diagrams that model the dynamics of the exit gates. CellEVAC has been tested in a simulated real scenario (Madrid Arena) under different external pedestrian flow patterns that simulate complex pedestrian interactions. Results showed that CellEVAC outperforms evacuation processes in which the system is not used, with an exponential improvement as interactions become complex. We compared our system with an existing approach based on Cartesian Genetic Programming. Our system exhibited a better overall performance in terms of safety, evacuation time, and the number of revisions of exit-choice decisions. Further analyses also revealed that Cartesian Genetic Programming generates less natural pedestrian reactions and movements than CellEVAC. The fact that the decision logic module is built upon a behavioral model seems to favor a more natural and effective response. We also found that our proposal has a positive influence on evacuations even for a low compliance rate (40%).
Interval Universal Approximation for Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

To verify safety and robustness of neural networks, researchers have successfully applied abstract interpretation, primarily using the interval abstract domain. In this paper, we study the theoretical power and limits of the interval domain for neural-network verification. First, we introduce the interval universal approximation (IUA) theorem. IUA shows that neural networks not only can approximate any continuous function $f$ (universal approximation) as we have known for decades, but we can find a neural network, using any well-behaved activation function, whose interval bounds are an arbitrarily close approximation of the set semantics of $f$ (the result of applying $f$ to a set of inputs). We call this notion of approximation interval approximation. Our theorem generalizes the recent result of Baader et al. (2020) from ReLUs to a rich class of activation functions that we call squashable functions. Additionally, the IUA theorem implies that we can always construct provably robust neural networks under $\ell_\infty$-norm using almost any practical activation function. Second, we study the computational complexity of constructing neural networks that are amenable to precise interval analysis. This is a crucial question, as our constructive proof of IUA is exponential in the size of the approximation domain. We boil this question down to the problem of approximating the range of a neural network with squashable activation functions. We show that the range approximation problem (RA) is a $Δ_2$-intermediate problem, which is strictly harder than $\mathsf{NP}$-complete problems, assuming $\mathsf{coNP}\not\subset \mathsf{NP}$. As a result, IUA is an inherently hard problem: No matter what abstract domain or computational tools we consider to achieve interval approximation, there is no efficient construction of such a universal approximator.
HOBFLOPS CNNs: Hardware Optimized Bitslice-Parallel Floating-Point Operations for Convolutional Neural Networks
Venue: arXiv cs.AR 2021
Year: 2021

Convolutional neural networks (CNNs) are typically trained using 16- or 32-bit floating-point (FP) and researchers show that low-precision floating-point (FP) can be highly effective for inference. Low-precision FP can be implemented in field programmable gate array (FPGA) and application-specific integrated circuit (ASIC) accelerators, but existing processors do not generally support custom precision FP. We propose hardware optimized bitslice-parallel floating-point operators (HOBFLOPS), a method of generating efficient custom-precision emulated bitslice-parallel software FP arithmetic. We generate custom-precision FP routines optimized using a hardware synthesis design flow to create circuits. We provide standard cell libraries matching the bitwise operations on the target microprocessor architecture, and a code-generator to translate the hardware circuits to bitslice software equivalents. We exploit bitslice parallelism to create a very wide (32-512 element) vectorized convolutional neural network (CNN) convolution. Hardware optimized bitslice-parallel floating-point operators (HOBFLOPS) multiply-accumulate (MAC) performance in CNN convolution on Arm and Intel processors are compared to Berkeley's SoftFP16 equivalent MAC. HOBFLOPS16 outperforms SoftFP16 by 8x on Intel AVX512. HOBFLOPS offers arbitrary-precision FP with custom range and precision e.g., HOBFLOPS9 performs at 6x the performance of HOBFLOPS16 on Arm Neon. HOBFLOPS allows researchers to prototype different levels of custom FP precision in the arithmetic of software CNN accelerators. Furthermore, HOBFLOPS fast custom-precision FP CNNs may be valuable in cases where memory bandwidth is limited.
T-Basis: a Compact Representation for Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

We introduce T-Basis, a novel concept for a compact representation of a set of tensors, each of an arbitrary shape, which is often seen in Neural Networks. Each of the tensors in the set is modeled using Tensor Rings, though the concept applies to other Tensor Networks. Owing its name to the T-shape of nodes in diagram notation of Tensor Rings, T-Basis is simply a list of equally shaped three-dimensional tensors, used to represent Tensor Ring nodes. Such representation allows us to parameterize the tensor set with a small number of parameters (coefficients of the T-Basis tensors), scaling logarithmically with each tensor's size in the set and linearly with the dimensionality of T-Basis. We evaluate the proposed approach on the task of neural network compression and demonstrate that it reaches high compression rates at acceptable performance drops. Finally, we analyze memory and operation requirements of the compressed networks and conclude that T-Basis networks are equally well suited for training and inference in resource-constrained environments and usage on the edge devices.
Can Encrypted DNS Be Fast?
Venue: arXiv cs.NI 2021
Year: 2021

In this paper, we study the performance of encrypted DNS protocols and conventional DNS from thousands of home networks in the United States, over one month in 2020. We perform these measurements from the homes of 2,693 participating panelists in the Federal Communications Commission's (FCC) Measuring Broadband America program. We found that clients do not have to trade DNS performance for privacy. For certain resolvers, DoT was able to perform faster than DNS in median response times, even as latency increased. We also found significant variation in DoH performance across recursive resolvers. Based on these results, we recommend that DNS clients (e.g., web browsers) should periodically conduct simple latency and response time measurements to determine which protocol and resolver a client should use. No single DNS protocol nor resolver performed the best for all clients.
Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglement
Venue: arXiv cs.CV 2021
Year: 2021

Learning a good 3D human pose representation is important for human pose related tasks, e.g. human 3D pose estimation and action recognition. Within all these problems, preserving the intrinsic pose information and adapting to view variations are two critical issues. In this work, we propose a novel Siamese denoising autoencoder to learn a 3D pose representation by disentangling the pose-dependent and view-dependent feature from the human skeleton data, in a fully unsupervised manner. These two disentangled features are utilized together as the representation of the 3D pose. To consider both the kinematic and geometric dependencies, a sequential bidirectional recursive network (SeBiReNet) is further proposed to model the human skeleton data. Extensive experiments demonstrate that the learned representation 1) preserves the intrinsic information of human pose, 2) shows good transferability across datasets and tasks. Notably, our approach achieves state-of-the-art performance on two inherently different tasks: pose denoising and unsupervised action recognition. Code and models are available at: \url{https://github.com/NIEQiang001/unsupervised-human-pose.git}
Learning to Sample with Local and Global Contexts in Experience Replay Buffer
Venue: arXiv cs.LG 2021
Year: 2021

Experience replay, which enables the agents to remember and reuse experience from the past, has played a significant role in the success of off-policy reinforcement learning (RL). To utilize the experience replay efficiently, the existing sampling methods allow selecting out more meaningful experiences by imposing priorities on them based on certain metrics (e.g. TD-error). However, they may result in sampling highly biased, redundant transitions since they compute the sampling rate for each transition independently, without consideration of its importance in relation to other transitions. In this paper, we aim to address the issue by proposing a new learning-based sampling method that can compute the relative importance of transition. To this end, we design a novel permutation-equivariant neural architecture that takes contexts from not only features of each transition (local) but also those of others (global) as inputs. We validate our framework, which we refer to as Neural Experience Replay Sampler (NERS), on multiple benchmark tasks for both continuous and discrete control tasks and show that it can significantly improve the performance of various off-policy RL methods. Further analysis confirms that the improvements of the sample efficiency indeed are due to sampling diverse and meaningful transitions by NERS that considers both local and global contexts.
An Adaptive Self-Scheduling Loop Scheduler
Venue: arXiv cs.DC 2021
Year: 2021

Many shared-memory parallel irregular applications, such as sparse linear algebra and graph algorithms, depend on efficient loop scheduling (LS) in a fork-join manner despite that the work per loop iteration can greatly vary depending on the application and the input. Because of its importance, many different methods, e.g., workload-aware self-scheduling, and parameters, e.g., chunk size, have been explored to achieve reasonable performance that requires expert prior knowledge about the application and input. This work proposes a new LS method that requires little to no expert knowledge to achieve speedups close to those of tuned LS methods by self-managing chunk size based on a heuristic of workload variance and using work-stealing. This method, named \ichunk, is implemented into libgomp for testing. It is evaluated against OpenMP's guided, dynamic, and taskloop methods and is evaluated against BinLPT and generic work-stealing on an array of applications that includes: a synthetic benchmark, breadth-first search, K-Means, the molecular dynamics code LavaMD, and sparse matrix-vector multiplication. On 28 thread Intel system, \ichunk is the only method to always be one of the top three LS methods. On average across all applications, \ichunk is within 5.4% of the best method and is even able to outperform other LS methods for breadth-first search and K-Means.
Fair Ride Allocation on a Line
Venue: arXiv cs.GT 2021
Year: 2021

The airport game is a classical and well-known model of fair cost-sharing for a single facility among multiple agents. This paper extends it to the so-called assignment setting, that is, for multiple facilities and agents, each agent chooses a facility to use and shares the cost with the other agents. Such a situation can be often seen in sharing economy, such as sharing fees for office desks among workers, taxis among customers of possibly different destinations on a line, and so on. Our model is regarded as a coalition formation game based on the fair cost-sharing of the airport game; we call our model \emph{a fair ride allocation on a line}. As criteria of solution concepts, we incorporate Nash stability and envy-freeness into our setting. We show that a Nash-stable feasible allocation that minimizes the social cost of agents can be computed efficiently if a feasible allocation exists. For envy-freeness, we provide several structural properties of envy-free allocations. Based on these, we design efficient algorithms for finding an envy-free allocation when at least one of (1) the number of facilities, (2) the capacity of facilities, and (3) the number of agent types, is small. Moreover, we show that a consecutive envy-free allocation can be computed in polynomial time. On the negative front, we show the NP-hardness of determining the existence of an allocation under two relaxed envy-free concepts.
Active Learning under Label Shift
Venue: arXiv cs.LG 2021
Year: 2021

We address the problem of active learning under label shift: when the class proportions of source and target domains differ. We introduce a "medial distribution" to incorporate a tradeoff between importance weighting and class-balanced sampling and propose their combined usage in active learning. Our method is known as Mediated Active Learning under Label Shift (MALLS). It balances the bias from class-balanced sampling and the variance from importance weighting. We prove sample complexity and generalization guarantees for MALLS which show active learning reduces asymptotic sample complexity even under arbitrary label shift. We empirically demonstrate MALLS scales to high-dimensional datasets and can reduce the sample complexity of active learning by 60% in deep active learning tasks.
A Biologically Plausible Audio-Visual Integration Model for Continual Learning
Venue: arXiv cs.NE 2021
Year: 2021

The problem of catastrophic forgetting has a history of more than 30 years and has not been completely solved yet. Since the human brain has natural ability to perform continual lifelong learning, learning from the brain may provide solutions to this problem. In this paper, we propose a novel biologically plausible audio-visual integration model (AVIM) based on the assumption that the integration of audio and visual perceptual information in the medial temporal lobe during learning is crucial to form concepts and make continual learning possible. Specifically, we use multi-compartment Hodgkin-Huxley neurons to build the model and adopt the calcium-based synaptic tagging and capture as the model's learning rule. Furthermore, we define a new continual learning paradigm to simulate the possible continual learning process in the human brain. We then test our model under this new paradigm. Our experimental results show that the proposed AVIM can achieve state-of-the-art continual learning performance compared with other advanced methods such as OWM, iCaRL and GEM. Moreover, it can generate stable representations of objects during learning. These results support our assumption that concept formation is essential for continuous lifelong learning and suggest the proposed AVIM is a possible concept formation mechanism.
Indivisible Mixed Manna: On the Computability of MMS + PO Allocations
Venue: arXiv cs.GT 2021
Year: 2021

In this paper we initiate the study of finding fair and efficient allocations of an indivisible mixed manna: Divide m indivisible items among n agents under the fairness notion of maximin share (MMS) and the efficiency notion of Pareto optimality (PO). A mixed manna allows an item to be a good for some agents and a chore for others. The problem of finding $α$-MMS allocation for the (near) best $α\in(0,1]$ for which it exists, remains unresolved even for a goods manna with constantly many agents, while the problem of finding $α$-MMS+PO allocation is unexplored for any $α\in(0,1]$. We make significant progress on the above questions for a mixed manna. First, we show that for any $α>0$, an $α$-MMS allocation may not always exist, thus ruling out solving the problem for a fixed $α$. Second, towards computing $α$-MMS+PO allocation for the best possible $α$, we obtain a dichotomous result: We derive two conditions and show that the problem is tractable under these two conditions, while dropping either renders the problem intractable. The two conditions are: (i) number of agents is a constant, and (ii) for every agent, her absolute value for all the items is at least a constant factor of her total (absolute) value for all the goods or all the chores. In particular, first, for instances satisfying (i) and (ii) we design a PTAS - an efficient algorithm to find an $(α-ε)$-MMS and $γ$-PO allocation when given $ε,γ>0$, for the highest possible $α\in(0,1]$. Second, we show that if either condition is not satisfied then finding an $α$-MMS allocation for any $α\in(0,1]$ is NP-hard, even when a solution exists for $α=1$. To the best of our knowledge, ours is the first algorithm that ensures both approximate MMS and PO guarantees.
Analysis of Slotted ALOHA with an Age Threshold
Venue: arXiv cs.IT 2021
Year: 2021

We present a comprehensive steady-state analysis of threshold-ALOHA, a distributed age-aware modification of slotted ALOHA proposed in recent literature. In threshold-ALOHA, each terminal suspends its transmissions until the Age of Information (AoI) of the status update flow it is sending reaches a certain threshold $Γ$. Once the age exceeds $Γ$, the terminal attempts transmission with constant probability $τ$ in each slot, as in standard slotted ALOHA. We analyze the time-average expected AoI attained by this policy, and explore its scaling with network size, $n$. We derive the probability distribution of the number of active users at steady state, and show that as network size increases the policy converges to one that runs slotted ALOHA with fewer sources: on average about one fifth of the users is active at any time. We obtain an expression for steady-state expected AoI and use this to optimize the parameters $Γ$ and $τ$, resolving the conjectures in \cite{doga} by confirming that the optimal age threshold and transmission probability are $2.2n$ and $4.69/n$, respectively. We find that the optimal AoI scales with the network size as $1.4169n$, which is almost half the minimum AoI achievable with slotted ALOHA, while the loss from the maximum throughput of $e^{-1}$ remains below $1\%$. We compare the performance of this rudimentary algorithm to that of the SAT policy that dynamically adapts its transmission probabilities.
Towards Augmented Reality-driven Human-City Interaction: Current Research on Mobile Headsets and Future Challenges
Venue: arXiv cs.HC 2021
Year: 2021

Interaction design for Augmented Reality (AR) is gaining increasing attention from both academia and industry. This survey discusses 260 articles (68.8% of articles published between 2015 - 2019) to review the field of human interaction in connected cities with emphasis on augmented reality-driven interaction. We provide an overview of Human-City Interaction and related technological approaches, followed by a review of the latest trends of information visualization, constrained interfaces, and embodied interaction for AR headsets. We highlight under-explored issues in interface design and input techniques that warrant further research, and conjecture that AR with complementary Conversational User Interfaces (CUIs) is a key enabler for ubiquitous interaction with immersive systems in smart cities. Our work helps researchers understand the current potential and future needs of AR in Human-City Interaction.
Multi-Task Federated Learning for Personalised Deep Neural Networks in Edge Computing
Venue: arXiv cs.LG 2021
Year: 2021

Federated Learning (FL) is an emerging approach for collaboratively training Deep Neural Networks (DNNs) on mobile devices, without private user data leaving the devices. Previous works have shown that non-Independent and Identically Distributed (non-IID) user data harms the convergence speed of the FL algorithms. Furthermore, most existing work on FL measures global-model accuracy, but in many cases, such as user content-recommendation, improving individual User model Accuracy (UA) is the real objective. To address these issues, we propose a Multi-Task FL (MTFL) algorithm that introduces non-federated Batch-Normalization (BN) layers into the federated DNN. MTFL benefits UA and convergence speed by allowing users to train models personalised to their own data. MTFL is compatible with popular iterative FL optimisation algorithms such as Federated Averaging (FedAvg), and we show empirically that a distributed form of Adam optimisation (FedAvg-Adam) benefits convergence speed even further when used as the optimisation strategy within MTFL. Experiments using MNIST and CIFAR10 demonstrate that MTFL is able to significantly reduce the number of rounds required to reach a target UA, by up to $5\times$ when using existing FL optimisation strategies, and with a further $3\times$ improvement when using FedAvg-Adam. We compare MTFL to competing personalised FL algorithms, showing that it is able to achieve the best UA for MNIST and CIFAR10 in all considered scenarios. Finally, we evaluate MTFL with FedAvg-Adam on an edge-computing testbed, showing that its convergence and UA benefits outweigh its overhead.