OpenRaiser/Intern-Atlas huggingface dataset:
Generative Probabilistic Image Colorization
Venue: arXiv cs.CV 2021
Year: 2021

We propose Generative Probabilistic Image Colorization, a diffusion-based generative process that trains a sequence of probabilistic models to reverse each step of noise corruption. Given a line-drawing image as input, our method suggests multiple candidate colorized images. Therefore, our method accounts for the ill-posed nature of the colorization problem. We conducted comprehensive experiments investigating the colorization of line-drawing images, report the influence of a score-based MCMC approach that corrects the marginal distribution of estimated samples, and further compare different combinations of models and the similarity of their generated images. Despite using only a relatively small training dataset, we experimentally develop a method to generate multiple diverse colorization candidates which avoids mode collapse and does not require any additional constraints, losses, or re-training with alternative training conditions. Our proposed approach performed well not only on color-conditional image generation tasks using biased initial values, but also on some practical image completion and inpainting tasks.
Federated Learning over Next-Generation Ethernet Passive Optical Networks
Venue: arXiv cs.NI 2021
Year: 2021

Federated Learning (FL) is a distributed machine learning (ML) type of processing that preserves the privacy of user data, sharing only the parameters of ML models with a common server. The processing of FL requires specific latency and bandwidth demands that need to be fulfilled by the operation of the communication network. This paper introduces a Dynamic Wavelength and Bandwidth Allocation algorithm for Quality of Service (QoS) provisioning for FL traffic over 50 Gb/s Ethernet Passive Optical Networks. The proposed algorithm prioritizes FL traffic and reduces the delay of FL and delay-critical applications supported on the same infrastructure.
Fast B4B: Fast BFT for Blockchains
Venue: arXiv cs.DC 2021
Year: 2021

Low latency is one of the desired properties for partially synchronous Byzantine consensus protocols. Previous protocols have achieved consensus with just two communication steps either by reducing the bound on the number of faults the protocol can tolerate ($f \leq \frac{n-1}{5}$) or use of trusted hardware like Trusted Execution Environment or TEEs. In this paper, we propose a protocol called Fast B4B, in which the protocol achieves consensus in just two communication steps. Fast B4B can tolerate maximum number of faults a partial BFT consensus can tolerate ($f \leq \frac{n-1}{3}$). Furthermore, Fast B4B does not require the use of any trusted hardware. The trade-off for this achievement is that at most $f$ times some nodes may revert their blocks. We show that this reversion of a block will not compromise the safety of the protocol at all, yet it may incur a small amount of additional latency during view change.
Visualization Design Sprints for Online and On-Campus Courses
Venue: arXiv cs.HC 2021
Year: 2021

We present how to integrate Design Sprints and project-based learning into introductory visualization courses. A design sprint is a unique process based on rapid prototyping and user testing to define goals and validate ideas before starting costly development. The well-defined, interactive, and time-constrained design cycle makes design sprints a promising option for teaching project-based and active-learning-centered courses to increase student engagement and hands-on experience. Over the past five years, we have adjusted the design sprint methodology for teaching a range of visualization courses. We present a detailed guide on incorporating design sprints into large undergraduate and small professional development courses in both online and on-campus settings. Design sprint results, including quantitative and qualitative student feedback, show that design sprints engage students and help practice and apply visualization and design skills. We provide design sprint teaching materials, show examples of student-created work, and discuss limitations and lessons learned.
Privacy Policy Question Answering Assistant: A Query-Guided Extractive Summarization Approach
Venue: arXiv cs.CL 2021
Year: 2021

Existing work on making privacy policies accessible has explored new presentation forms such as color-coding based on the risk factors or summarization to assist users with conscious agreement. To facilitate a more personalized interaction with the policies, in this work, we propose an automated privacy policy question answering assistant that extracts a summary in response to the input user query. This is a challenging task because users articulate their privacy-related questions in a very different language than the legal language of the policy, making it difficult for the system to understand their inquiry. Moreover, existing annotated data in this domain are limited. We address these problems by paraphrasing to bring the style and language of the user's question closer to the language of privacy policies. Our content scoring module uses the existing in-domain data to find relevant information in the policy and incorporates it in a summary. Our pipeline is able to find an answer for 89% of the user queries in the privacyQA dataset.
Understanding Egocentric Hand-Object Interactions from Hand Pose Estimation
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we address the problem of estimating the hand pose from the egocentric view when the hand is interacting with objects. Specifically, we propose a method to label a dataset Ego-Siam which contains the egocentric images pair-wisely. We also use the collected pairwise data to train our encoder-decoder style network which has been proven efficient in. This could bring extra training efficiency and testing accuracy. Our network is lightweight and can be performed with over 30 FPS with an outdated GPU. We demonstrate that our method outperforms Mueller et al. which is the state of the art work dealing with egocentric hand-object interaction problems on the GANerated dataset. To show the ability to preserve the semantic information of our method, we also report the performance of grasp type classification on GUN-71 dataset and outperforms the benchmark by only using the predicted 3-d hand pose.
Batched Bandits with Crowd Externalities
Venue: arXiv cs.LG 2021
Year: 2021

In Batched Multi-Armed Bandits (BMAB), the policy is not allowed to be updated at each time step. Usually, the setting asserts a maximum number of allowed policy updates and the algorithm schedules them so that to minimize the expected regret. In this paper, we describe a novel setting for BMAB, with the following twist: the timing of the policy update is not controlled by the BMAB algorithm, but instead the amount of data received during each batch, called \textit{crowd}, is influenced by the past selection of arms. We first design a near-optimal policy with approximate knowledge of the parameters that we prove to have a regret in $\mathcal{O}(\sqrt{\frac{\ln x}{x}}+ε)$ where $x$ is the size of the crowd and $ε$ is the parameter error. Next, we implement a UCB-inspired algorithm that guarantees an additional regret in $\mathcal{O}\left(\max(K\ln T,\sqrt{T\ln T})\right)$, where $K$ is the number of arms and $T$ is the horizon.
Unlocking the potential of deep learning for marine ecology: overview, applications, and outlook
Venue: arXiv cs.LG 2021
Year: 2021

The deep learning revolution is touching all scientific disciplines and corners of our lives as a means of harnessing the power of big data. Marine ecology is no exception. These new methods provide analysis of data from sensors, cameras, and acoustic recorders, even in real time, in ways that are reproducible and rapid. Off-the-shelf algorithms can find, count, and classify species from digital images or video and detect cryptic patterns in noisy data. Using these opportunities requires collaboration across ecological and data science disciplines, which can be challenging to initiate. To facilitate these collaborations and promote the use of deep learning towards ecosystem-based management of the sea, this paper aims to bridge the gap between marine ecologists and computer scientists. We provide insight into popular deep learning approaches for ecological data analysis in plain language, focusing on the techniques of supervised learning with deep neural networks, and illustrate challenges and opportunities through established and emerging applications of deep learning to marine ecology. We use established and future-looking case studies on plankton, fishes, marine mammals, pollution, and nutrient cycling that involve object detection, classification, tracking, and segmentation of visualized data. We conclude with a broad outlook of the field's opportunities and challenges, including potential technological advances and issues with managing complex data sets.
Here To Stay: Measuring Hologram Stability in Markerless Smartphone Augmented Reality
Venue: arXiv cs.HC 2021
Year: 2021

Markerless augmented reality (AR) has the potential to provide engaging experiences and improve outcomes across a wide variety of industries; the overlaying of virtual content, or holograms, onto a view of the real world without the need for predefined markers provides great convenience and flexibility. However, unwanted hologram movement frequently occurs in markerless smartphone AR due to challenging visual conditions or device movement, and resulting error in device pose tracking. We develop a method for measuring hologram positional errors on commercial smartphone markerless AR platforms, implement it as an open-source AR app, HoloMeasure, and use the app to conduct systematic quantitative characterizations of hologram stability across 6 different user actions, 3 different smartphone models, and over 200 different environments. Our study demonstrates significant levels of spatial instability in holograms in all but the simplest settings, and underscores the need for further enhancements to pose tracking algorithms for smartphone-based markerless AR.
TopoShot: Uncovering Ethereum's Network Topology Leveraging Replacement Transactions
Venue: arXiv cs.NI 2021
Year: 2021

Ethereum relies on a peer-to-peer overlay network to propagate information. The knowledge of Ethereum network topology holds the key to understanding Ethereum's security, availability, and user anonymity. From a measurement perspective, an Ethereum network's topology is routing-table information hidden inside individual Ethereum nodes, measuring which poses challenges and remains an open research problem in the existing literature. This paper presents TopoShot, a new method uniquely repurposing Ethereum's transaction replacement/eviction policies for topology measurement. TopoShot can be configured to support Geth, Parity, and other major Ethereum clients. As validated on local nodes, TopoShot achieves 100% measurement precision and high recall 88% - 97%. To efficiently measure the large Ethereum networks in the wild, we propose a non-trivial schedule to run pair-wise measurements in parallel. To enable ethical measurement on Ethereum mainnet, we propose workload-adaptive configurations of TopoShot to minimize the service interruption to target nodes/network. We systematically measure a variety of Ethereum networks and obtain new knowledge including the full-network topology in major testnets (Ropsten, Rinkeby and Goerli) and critical sub-network topology in the mainnet. The results on testnets show interesting graph-theoretic properties, such as all testnets exhibit graph modularity significantly lower than random graphs, implying resilience to network partitions. The mainnet results show biased neighbor selection strategies adopted by critical Ethereum services such as mining pools and transaction relays, implying a degree of centralization in real Ethereum networks.
Secure Machine Learning over Relational Data
Venue: arXiv cs.CR 2021
Year: 2021

A closer integration of machine learning and relational databases has gained steam in recent years due to the fact that the training data to many ML tasks is the results of a relational query (most often, a join-select query). In a federated setting, this poses an additional challenge, that the tables are held by different parties as their private data, and the parties would like to train the model without having to use a trusted third party. Existing work has only considered the case where the training data is stored in a flat table that has been vertically partitioned, which corresponds to a simple PK-PK join. In this paper, we describe secure protocols to compute the join results of multiple tables conforming to a general foreign-key acyclic schema, and how to feed the results in secret-shared form to a secure ML toolbox. Furthermore, existing secure ML systems reveal the PKs in the join results. We strengthen the privacy protection to higher levels and achieve zero information leakage beyond the trained model. If the model itself is considered sensitive, we show how differential privacy can be incorporated into our framework to also prevent the model from breaching individuals' privacy.
Constant-Cost Spatio-Angular Prefiltering of Glinty Appearance Using Tensor Decomposition
Venue: arXiv cs.GR 2021
Year: 2021

The detailed glinty appearance from complex surface microstructures enhances the level of realism, but is both space- and time-consuming to render, especially when viewed from far away (large spatial coverage) and/or illuminated by area lights (large angular coverage). In this paper, we formulate the glinty appearance rendering process as a spatio-angular range query problem of the Normal Distribution Functions (NDFs), and introduce an efficient spatio-angular prefiltering solution to it. We start by exhaustively precomputing all possible NDFs with differently sized positional coverages. Then we compress the precomputed data using tensor rank decomposition, which enables accurate and fast angular range queries. With our spatio-angular prefiltering scheme, we are able to solve both the storage and performance issues at the same time, leading to efficient rendering of glinty appearance with both constant storage and constant performance, regardless of the range of spatio-angular queries. Finally, we demonstrate that our method easily applies to practical rendering applications that were traditionally considered difficult. For example, efficient bidirectional reflection distribution function (BRDF) evaluation accurate NDF importance sampling, fast global illumination between glinty objects, high-frequency preserving rendering with environment lighting, and tile-based synthesis of glinty appearance.
COVID-19 Fake News Detection Using Bidirectional Encoder Representations from Transformers Based Models
Venue: arXiv cs.CL 2021
Year: 2021

Nowadays, the development of social media allows people to access the latest news easily. During the COVID-19 pandemic, it is important for people to access the news so that they can take corresponding protective measures. However, the fake news is flooding and is a serious issue especially under the global pandemic. The misleading fake news can cause significant loss in terms of the individuals and the society. COVID-19 fake news detection has become a novel and important task in the NLP field. However, fake news always contain the correct portion and the incorrect portion. This fact increases the difficulty of the classification task. In this paper, we fine tune the pre-trained Bidirectional Encoder Representations from Transformers (BERT) model as our base model. We add BiLSTM layers and CNN layers on the top of the finetuned BERT model with frozen parameters or not frozen parameters methods respectively. The model performance evaluation results showcase that our best model (BERT finetuned model with frozen parameters plus BiLSTM layers) achieves state-of-the-art results towards COVID-19 fake news detection task. We also explore keywords evaluation methods using our best model and evaluate the model performance after removing keywords.
A system on chip for melanoma detection using FPGA-based SVM classifier
Venue: arXiv cs.LG 2021
Year: 2021

Support Vector Machine (SVM) is a robust machine learning model that shows high accuracy with different classification problems, and is widely used for various embedded applications. However , implementation of embedded SVM classifiers is challenging, due to the inherent complicated computations required. This motivates implementing the SVM on hardware platforms for achieving high performance computing at low cost and power consumption. Melanoma is the most aggressive form of skin cancer that increases the mortality rate. We aim to develop an optimized embedded SVM classifier dedicated for a low-cost handheld device for early detection of melanoma at the primary healthcare. In this paper, we propose a hardware/software co-design for implementing the SVM classifier onto FPGA to realize melanoma detection on a chip. The implemented SVM on a recent hybrid FPGA (Zynq) platform utilizing the modern UltraFast High-Level Synthesis design methodology achieves efficient melanoma classification on chip. The hardware implementation results demonstrate classification accuracy of 97.9%, and a significant hardware acceleration rate of 21 with only 3% resources utilization and 1.69W for power consumption. These results show that the implemented system on chip meets crucial embedded system constraints of high performance and low resources utilization, power consumption, and cost, while achieving efficient classification with high classification accuracy.
Breaking the hegemony of the triangle method in clique detection
Venue: arXiv cs.DS 2021
Year: 2021

We consider the fundamental problem of detecting/counting copies of a fixed pattern graph in a host graph. The recent progress on this problem has not included complete pattern graphs, i.e., cliques (and their complements, i.e., edge-free pattern graphs, in the induced setting). The fastest algorithms for the aforementioned patterns are based on a straightforward reduction to triangle detection/counting. We provide an alternative method of detection/counting copies of fixed size cliques based on a multi-dimensional matrix product. It is at least as time efficient as the triangle method in cases of $K_4$ and $K_5.$ The complexity of the multi-dimensional matrix product is of interest in its own rights. We provide also another alternative method for detection/counting $K_r$ copies, again time efficient for $r\in \{ 4,\ 5 \}$.
Crowdsourcing through Cognitive Opportunistic Networks
Venue: arXiv cs.NI 2021
Year: 2021

Untile recently crowdsourcing has been primarily conceived as an online activity to harness resources for problem solving. However the emergence of opportunistic networking (ON) has opened up crowdsourcing to the spatial domain. In this paper we bring the ON model for potential crowdsourcing in the smart city environment. We introduce cognitive features to the ON that allow users' mobile devices to become aware of the surrounding physical environment. Specifically, we exploit cognitive psychology studies on dynamic memory structures and cognitive heuristics, i.e. mental models that describe how the human brain handle decision-making amongst complex and real-time stimuli. Combined with ON, these cognitive features allow devices to act as proxies in the cyber-world of their users and exchange knowledge to deliver awareness of places in an urban environment. This is done through tags associated with locations. They represent features that are perceived by humans about a place. We consider the extent to which this knowledge becomes available to participants, using interactions with locations and other nodes. This is assessed taking into account a wide range of cognitive parameters. Outcomes are important because this functionality could support a new type of recommendation system that is independent of the traditional forms of networking.
Augmented reality navigation system for visual prosthesis
Venue: arXiv cs.CV 2021
Year: 2021

The visual functions of visual prostheses such as field of view, resolution and dynamic range, seriously restrict the person's ability to navigate in unknown environments. Implanted patients still require constant assistance for navigating from one location to another. Hence, there is a need for a system that is able to assist them safely during their journey. In this work, we propose an augmented reality navigation system for visual prosthesis that incorporates a software of reactive navigation and path planning which guides the subject through convenient, obstacle-free route. It consists on four steps: locating the subject on a map, planning the subject trajectory, showing it to the subject and re-planning without obstacles. We have also designed a simulated prosthetic vision environment which allows us to systematically study navigation performance. Twelve subjects participated in the experiment. Subjects were guided by the augmented reality navigation system and their instruction was to navigate through different environments until they reached two goals, cross the door and find an object (bin), as fast and accurately as possible. Results show how our augmented navigation system help navigation performance by reducing the time and distance to reach the goals, even significantly reducing the number of obstacles collisions, compared to other baseline methods.
Unified Data Collection for Visual-Inertial Calibration via Deep Reinforcement Learning
Venue: arXiv cs.RO 2021
Year: 2021

Visual-inertial sensors have a wide range of applications in robotics. However, good performance often requires different sophisticated motion routines to accurately calibrate camera intrinsics and inter-sensor extrinsics. This work presents a novel formulation to learn a motion policy to be executed on a robot arm for automatic data collection for calibrating intrinsics and extrinsics jointly. Our approach models the calibration process compactly using model-free deep reinforcement learning to derive a policy that guides the motions of a robotic arm holding the sensor to efficiently collect measurements that can be used for both camera intrinsic calibration and camera-IMU extrinsic calibration. Given the current pose and collected measurements, the learned policy generates the subsequent transformation that optimizes sensor calibration accuracy. The evaluations in simulation and on a real robotic system show that our learned policy generates favorable motion trajectories and collects enough measurements efficiently that yield the desired intrinsics and extrinsics with short path lengths. In simulation we are able to perform calibrations 10 times faster than hand-crafted policies, which transfers to a real-world speed up of 3 times over a human expert.
A Priori Calibration of Transient Kinetics Data via Machine Learning
Venue: arXiv cs.LG 2021
Year: 2021

The temporal analysis of products reactor provides a vast amount of transient kinetic information that may be used to describe a variety of chemical features including the residence time distribution, kinetic coefficients, number of active sites, and the reaction mechanism. However, as with any measurement device, the TAP reactor signal is convoluted with noise. To reduce the uncertainty of the kinetic measurement and any derived parameters or mechanisms, proper preprocessing must be performed prior to any advanced analysis. This preprocessing consists of baseline correction, i.e., a shift in the voltage response, and calibration, i.e., a scaling of the flux response based on prior experiments. The current methodology of preprocessing requires significant user discretion and reliance on previous experiments that may drift over time. Herein we use machine learning techniques combined with physical constraints to convert the raw instrument signal to chemical information. As such, the proposed methodology demonstrates clear benefits over the traditional preprocessing in the calibration of the inert and feed mixture products without need of prior calibration experiments or heuristic input from the user.