OpenRaiser/Intern-Atlas huggingface dataset:
Physically Inspired Dense Fusion Networks for Relighting
Venue: arXiv cs.CV 2021
Year: 2021

Image relighting has emerged as a problem of significant research interest inspired by augmented reality applications. Physics-based traditional methods, as well as black box deep learning models, have been developed. The existing deep networks have exploited training to achieve a new state of the art; however, they may perform poorly when training is limited or does not represent problem phenomenology, such as the addition or removal of dense shadows. We propose a model which enriches neural networks with physical insight. More precisely, our method generates the relighted image with new illumination settings via two different strategies and subsequently fuses them using a weight map (w). In the first strategy, our model predicts the material reflectance parameters (albedo) and illumination/geometry parameters of the scene (shading) for the relit image (we refer to this strategy as intrinsic image decomposition (IID)). The second strategy is solely based on the black box approach, where the model optimizes its weights based on the ground-truth images and the loss terms in the training stage and generates the relit output directly (we refer to this strategy as direct). While our proposed method applies to both one-to-one and any-to-any relighting problems, for each case we introduce problem-specific components that enrich the model performance: 1) For one-to-one relighting we incorporate normal vectors of the surfaces in the scene to adjust gloss and shadows accordingly in the image. 2) For any-to-any relighting, we propose an additional multiscale block to the architecture to enhance feature extraction. Experimental results on the VIDIT 2020 and the VIDIT 2021 dataset (used in the NTIRE 2021 relighting challenge) reveals that our proposal can outperform many state-of-the-art methods in terms of well-known fidelity metrics and perceptual loss.
ADAM: A Sandbox for Implementing Language Learning
Venue: arXiv cs.CL 2021
Year: 2021

We present ADAM, a software system for designing and running child language learning experiments in Python. The system uses a virtual world to simulate a grounded language acquisition process in which the language learner utilizes cognitively plausible learning algorithms to form perceptual and linguistic representations of the observed world. The modular nature of ADAM makes it easy to design and test different language learning curricula as well as learning algorithms. In this report, we describe the architecture of the ADAM system in detail, and illustrate its components with examples. We provide our code.
Byzantine-Robust and Privacy-Preserving Framework for FedML
Venue: arXiv cs.CR 2021
Year: 2021

Federated learning has emerged as a popular paradigm for collaboratively training a model from data distributed among a set of clients. This learning setting presents, among others, two unique challenges: how to protect privacy of the clients' data during training, and how to ensure integrity of the trained model. We propose a two-pronged solution that aims to address both challenges under a single framework. First, we propose to create secure enclaves using a trusted execution environment (TEE) within the server. Each client can then encrypt their gradients and send them to verifiable enclaves. The gradients are decrypted within the enclave without the fear of privacy breaches. However, robustness check computations in a TEE are computationally prohibitive. Hence, in the second step, we perform a novel gradient encoding that enables TEEs to encode the gradients and then offloading Byzantine check computations to accelerators such as GPUs. Our proposed approach provides theoretical bounds on information leakage and offers a significant speed-up over the baseline in empirical evaluation.
Optimal Codes Correcting Localized Deletions
Venue: arXiv cs.IT 2021
Year: 2021

We consider the problem of constructing codes that can correct deletions that are localized within a certain part of the codeword that is unknown a priori. Namely, the model that we study is when at most $k$ deletions occur in a window of size $k$, where the positions of the deletions within this window are not necessarily consecutive. Localized deletions are thus a generalization of burst deletions that occur in consecutive positions. We present novel explicit codes that are efficiently encodable and decodable and can correct up to $k$ localized deletions. Furthermore, these codes have $\log n+\mathcal{O}(k \log^2 (k\log n))$ redundancy, where $n$ is the length of the information message, which is asymptotically optimal in $n$ for $k=o(\log n/(\log \log n)^2)$.
A Step Toward More Inclusive People Annotations for Fairness
Venue: arXiv cs.CV 2021
Year: 2021

The Open Images Dataset contains approximately 9 million images and is a widely accepted dataset for computer vision research. As is common practice for large datasets, the annotations are not exhaustive, with bounding boxes and attribute labels for only a subset of the classes in each image. In this paper, we present a new set of annotations on a subset of the Open Images dataset called the MIAP (More Inclusive Annotations for People) subset, containing bounding boxes and attributes for all of the people visible in those images. The attributes and labeling methodology for the MIAP subset were designed to enable research into model fairness. In addition, we analyze the original annotation methodology for the person class and its subclasses, discussing the resulting patterns in order to inform future annotation efforts. By considering both the original and exhaustive annotation sets, researchers can also now study how systematic patterns in training annotations affect modeling.
Estimating the Impact of BGP Prefix Hijacking
Venue: arXiv cs.NI 2021
Year: 2021

BGP prefix hijacking is a critical threat to the resilience and security of communications in the Internet. While several mechanisms have been proposed to prevent, detect or mitigate hijacking events, it has not been studied how to accurately quantify the impact of an ongoing hijack. When detecting a hijack, existing methods do not estimate how many networks in the Internet are affected (before and/or after its mitigation). In this paper, we study fundamental and practical aspects of the problem of estimating the impact of an ongoing hijack through network measurements. We derive analytical results for the involved trade-offs and limits, and investigate the performance of different measurement approaches (control/data-plane measurements) and use of public measurement infrastructure. Our findings provide useful insights for the design of accurate hijack impact estimation methodologies. Based on these insights, we design (i) a lightweight and practical estimation methodology that employs ping measurements, and (ii) an estimator that employs public infrastructure measurements and eliminates correlations between them to improve the accuracy. We validate the proposed methodologies and findings against results from hijacking experiments we conduct in the real Internet.
Survey on Multi-Agent Q-Learning frameworks for resource management in wireless sensor network
Venue: arXiv cs.MA 2021
Year: 2021

This report aims to survey multi-agent Q-Learning algorithms, analyze different game theory frameworks used, address each framework's applications, and report challenges and future directions. The target application for this study is resource management in the wireless sensor network. In the first section, the author provided an introduction regarding the applications of wireless sensor networks. After that, the author presented a summary of the Q-Learning algorithm, a well-known classic solution for model-free reinforcement learning problems. In the third section, the author extended the Q-Learning algorithm for multi-agent scenarios and discussed its challenges. In the fourth section, the author surveyed sets of game-theoretic frameworks that researchers used to address this problem for resource allocation and task scheduling in the wireless sensor networks. Lastly, the author mentioned some interesting open challenges in this domain.
From Bounded Checking to Verification of Equivalence via Symbolic Up-to Techniques
Venue: arXiv cs.PL 2021
Year: 2021

We present a bounded equivalence verification technique for higher-order programs with local state. This technique combines fully abstract symbolic environmental bisimulations similar to symbolic game semantics, novel up-to techniques, and lightweight state invariant annotations. This yields an equivalence verification technique with no false positives or negatives. The technique is bounded-complete, in that all inequivalences are automatically detected given large enough bounds. Moreover, several hard equivalences are proved automatically or after being annotated with state invariants. We realise the technique in a tool prototype called Hobbit and benchmark it with an extensive set of new and existing examples. Hobbit can prove many classical equivalences including all Meyer and Sieber examples.
Challenges and Obstacles Towards Deploying Deep Learning Models on Mobile Devices
Venue: arXiv cs.LG 2021
Year: 2021

From computer vision and speech recognition to forecasting trajectories in autonomous vehicles, deep learning approaches are at the forefront of so many domains. Deep learning models are developed using plethora of high-level, generic frameworks and libraries. Running those models on the mobile devices require hardware-aware optimizations and in most cases converting the models to other formats or using a third-party framework. In reality, most of the developed models need to undergo a process of conversion, adaptation, and, in some cases, full retraining to match the requirements and features of the framework that is deploying the model on the target platform. Variety of hardware platforms with heterogeneous computing elements, from wearable devices to high-performance GPU clusters are used to run deep learning models. In this paper, we present the existing challenges, obstacles, and practical solutions towards deploying deep learning models on mobile devices.
Analytical Differential Calculus with Integration
Venue: arXiv cs.PL 2021
Year: 2021

Differential lambda-calculus was first introduced by Thomas Ehrhard and Laurent Regnier in 2003. Despite more than 15 years of history, little work has been done on a differential calculus with integration. In this paper, we shall propose a differential calculus with integration from programming point of view. We show its good correspondence with mathematics, which is manifested by how we construct these reduction rules and how we preserve important mathematical theorems in our calculus. Moreover, we highlight applications of the calculus in incremental computation, automatic differentiation, and computation approximation.
A Framework of Explanation Generation toward Reliable Autonomous Robots
Venue: arXiv cs.AI 2021
Year: 2021

To realize autonomous collaborative robots, it is important to increase the trust that users have in them. Toward this goal, this paper proposes an algorithm which endows an autonomous agent with the ability to explain the transition from the current state to the target state in a Markov decision process (MDP). According to cognitive science, to generate an explanation that is acceptable to humans, it is important to present the minimum information necessary to sufficiently understand an event. To meet this requirement, this study proposes a framework for identifying important elements in the decision-making process using a prediction model for the world and generating explanations based on these elements. To verify the ability of the proposed method to generate explanations, we conducted an experiment using a grid environment. It was inferred from the result of a simulation experiment that the explanation generated using the proposed method was composed of the minimum elements important for understanding the transition from the current state to the target state. Furthermore, subject experiments showed that the generated explanation was a good summary of the process of state transition, and that a high evaluation was obtained for the explanation of the reason for an action.
Not all Strangers are the Same: The Impact of Tolerance in Schelling Games
Venue: arXiv cs.GT 2021
Year: 2021

Schelling's famous model of segregation assumes agents of different types who would like to be located in neighborhoods having at least a certain fraction of agents of the same type. We consider natural generalizations that allow for the possibility of agents being tolerant towards other agents, even if they are not of the same type. In particular, we consider an ordering of the types, and make the realistic assumption that the agents are in principle more tolerant towards agents of types that are closer to their own according to the ordering. Based on this, we study the strategic games induced when the agents aim to maximize their utility, for a variety of tolerance levels. We provide a collection of results about the existence of equilibria, and their quality in terms of social welfare.
Mobile Robot Localization Using Fuzzy Neural Network Based Extended Kalman Filter
Venue: arXiv cs.RO 2021
Year: 2021

This paper proposes a novel approach to improve the performance of the extended Kalman filter (EKF) for the problem of mobile robot localization. A fuzzy logic system is employed to continuous-ly adjust the noise covariance matrices of the filter. A neural network is implemented to regulate the membership functions of the antecedent and consequent parts of the fuzzy rules. The aim is to gain the accuracy and avoid the divergence of the EKF when the noise covariance matrices are fixed or wrongly determined. Simulations and experiments have been conducted. The results show that the proposed filter is better than the EKF in localizing the mobile robot.
SparseConvMIL: Sparse Convolutional Context-Aware Multiple Instance Learning for Whole Slide Image Classification
Venue: arXiv cs.CV 2021
Year: 2021

Multiple instance learning (MIL) is the preferred approach for whole slide image classification. However, most MIL approaches do not exploit the interdependencies of tiles extracted from a whole slide image, which could provide valuable cues for classification. This paper presents a novel MIL approach that exploits the spatial relationship of tiles for classifying whole slide images. To do so, a sparse map is built from tiles embeddings, and is then classified by a sparse-input CNN. It obtained state-of-the-art performance over popular MIL approaches on the classification of cancer subtype involving 10000 whole slide images. Our results suggest that the proposed approach might (i) improve the representation learning of instances and (ii) exploit the context of instance embeddings to enhance the classification performance. The code of this work is open-source at {github censored for review}.
A Guide for New Program Committee Members at Theoretical Computer Science Conferences
Venue: arXiv cs.GL 2021
Year: 2021

In theoretical computer science, conferences play an important role in the scientific process. The decisions whether to accept or reject articles is taken by the program committee (PC) members. Serving on a PC for the first time can be a daunting experience. This guide will help new program-committee members to understand how the system works, and provide useful tips and guidelines. It discusses every phase of the paper-selection process, and the tasks associated to it.
Total Novel and Complexity. Literature and Complexity Science
Venue: arXiv cs.OH 2021
Year: 2021

A strong link between complexity theory and literature is possible, i.e. feasible, under one proviso, namely that total novels be considered. However, neither in literature at large nor in complexity science has been literature seriously taken into consideration. This paper argues that a total novel is most conspicuous example of a complex system. The argument is supported by a clear characterization of what a total novel is and entails. Science and literature can be thus complemented and developed, hand in hand.
Holistic Privacy and Usability of a Cryptocurrency Wallet
Venue: arXiv cs.CR 2021
Year: 2021

In this study, we overview the problems associated with the usability of cryptocurrency wallets, such as those used by ZCash, for end-users. The concept of "holistic privacy," where information leaks in one part of a system can violate the privacy expectations of different parts of the system, is introduced as a requirement. To test this requirement with real-world software, we did a 60 person task-based evaluation of the usability of a ZCash cryptocurrency wallet by having users install and try to both send and receive anonymized ZCash transactions, as well as install a VPN and Tor. While the initial wallet installation was difficult, we found even a larger amount of difficulty integrating the ZCash wallet into network-level protection like VPNs or Tor, so only a quarter of users could complete a real-world purchase using the wallet.
Izhikevich-Inspired Optoelectronic Neurons with Excitatory and Inhibitory Inputs for Energy-Efficient Photonic Spiking Neural Networks
Venue: arXiv cs.NE 2021
Year: 2021

We designed, prototyped, and experimentally demonstrated, for the first time to our knowledge, an optoelectronic spiking neuron inspired by the Izhikevich model incorporating both excitatory and inhibitory optical spiking inputs and producing optical spiking outputs accordingly. The optoelectronic neurons consist of three transistors acting as electrical spiking circuits, a vertical-cavity surface-emitting laser (VCSEL) for optical spiking outputs, and two photodetectors for excitatory and inhibitory optical spiking inputs. Additional inclusion of capacitors and resistors complete the Izhikevich-inspired optoelectronic neurons, which receive excitatory and inhibitory optical spikes as inputs from other optoelectronic neurons. We developed a detailed optoelectronic neuron model in Verilog-A and simulated the circuit-level operation of various cases with excitatory input and inhibitory input signals. The experimental results closely resemble the simulated results and demonstrate how the excitatory inputs trigger the optical spiking outputs while the inhibitory inputs suppress the outputs. Utilizing the simulated neuron model, we conducted simulations using fully connected (FC) and convolutional neural networks (CNN). The simulation results using MNIST handwritten digits recognition show 90% accuracy on unsupervised learning and 97% accuracy on a supervised modified FC neural network. We further designed a nanoscale optoelectronic neuron utilizing quantum impedance conversion where a 200 aJ/spike input can trigger the output from on-chip nanolasers with 10 fJ/spike. The nanoscale neuron can support a fanout of ~80 or overcome 19 dB excess optical loss while running at 10 GSpikes/second in the neural network, which corresponds to 100x throughput and 1000x energy-efficiency improvement compared to state-of-art electrical neuromorphic hardware such as Loihi and NeuroGrid.
Online Preconditioning of Experimental Inkjet Hardware by Bayesian Optimization in Loop
Venue: arXiv cs.CV 2021
Year: 2021

High-performance semiconductor optoelectronics such as perovskites have high-dimensional and vast composition spaces that govern the performance properties of the material. To cost-effectively search these composition spaces, we utilize a high-throughput experimentation method of rapidly printing discrete droplets via inkjet deposition, in which each droplet is comprised of a unique permutation of semiconductor materials. However, inkjet printer systems are not optimized to run high-throughput experimentation on semiconductor materials. Thus, in this work, we develop a computer vision-driven Bayesian optimization framework for optimizing the deposited droplet structures from an inkjet printer such that it is tuned to perform high-throughput experimentation on semiconductor materials. The goal of this framework is to tune to the hardware conditions of the inkjet printer in the shortest amount of time using the fewest number of droplet samples such that we minimize the time and resources spent on setting the system up for material discovery applications. We demonstrate convergence on optimum inkjet hardware conditions in 10 minutes using Bayesian optimization of computer vision-scored droplet structures. We compare our Bayesian optimization results with stochastic gradient descent.
Age of Gossip in Networks with Community Structure
Venue: arXiv cs.IT 2021
Year: 2021

We consider a network consisting of a single source and $n$ receiver nodes that are grouped into $m$ equal size communities, i.e., clusters, where each cluster includes $k$ nodes and is served by a dedicated cluster head. The source node keeps versions of an observed process and updates each cluster through the associated cluster head. Nodes within each cluster are connected to each other according to a given network topology. Based on this topology, each node relays its current update to its neighboring nodes by $local$ $gossiping$. We use the $version$ $age$ metric to quantify information timeliness at the receiver nodes. We consider disconnected, ring, and fully connected network topologies for each cluster. For each of these network topologies, we characterize the average version age at each node and find the version age scaling as a function of the network size $n$. Our results indicate that per node version age scalings of $O(\sqrt{n})$, $O(n^{\frac{1}{3}})$, and $O(\log n)$ are achievable in disconnected, ring, and fully connected cluster models, respectively. Finally, through numerical evaluations, we determine the version age-optimum $(m,k)$ pairs as a function of the source, cluster head, and node update rates.
Contextual Bandits with Sparse Data in Web setting
Venue: arXiv cs.LG 2021
Year: 2021

This paper is a scoping study to identify current methods used in handling sparse data with contextual bandits in web settings. The area is highly current and state of the art methods are identified. The years 2017-2020 are investigated, and 19 method articles are identified, and two review articles. Five categories of methods are described, making it easy to choose how to address sparse data using contextual bandits with a method available for modification in the specific setting of concern. In addition, each method has multiple techniques to choose from for future evaluation. The problem areas are also mentioned that each article covers. An overall updated understanding of sparse data problems using contextual bandits in web settings is given. The identified methods are policy evaluation (off-line and on-line) , hybrid-method, model representation (clusters and deep neural networks), dimensionality reduction, and simulation.