OpenRaiser/Intern-Atlas huggingface dataset:
High-quality Panorama Stitching based on Asymmetric Bidirectional Optical Flow
Venue: arXiv cs.CV 2020
Year: 2020

In this paper, we propose a panorama stitching algorithm based on asymmetric bidirectional optical flow. This algorithm expects multiple photos captured by fisheye lens cameras as input, and then, through the proposed algorithm, these photos can be merged into a high-quality 360-degree spherical panoramic image. For photos taken from a distant perspective, the parallax among them is relatively small, and the obtained panoramic image can be nearly seamless and undistorted. For photos taken from a close perspective or with a relatively large parallax, a seamless though partially distorted panoramic image can also be obtained. Besides, with the help of Graphics Processing Unit (GPU), this algorithm can complete the whole stitching process at a very fast speed: typically, it only takes less than 30s to obtain a panoramic image of 9000-by-4000 pixels, which means our panorama stitching algorithm is of high value in many real-time applications. Our code is available at https://github.com/MungoMeng/Panorama-OpticalFlow.
Word-Emoji Embeddings from large scale Messaging Data reflect real-world Semantic Associations of Expressive Icons
Venue: arXiv cs.CL 2020
Year: 2020

We train word-emoji embeddings on large scale messaging data obtained from the Jodel online social network. Our data set contains more than 40 million sentences, of which 11 million sentences are annotated with a subset of the Unicode 13.0 standard Emoji list. We explore semantic emoji associations contained in this embedding by analyzing associations between emojis, between emojis and text, and between text and emojis. Our investigations demonstrate anecdotally that word-emoji embeddings trained on large scale messaging data can reflect real-world semantic associations. To enable further research we release the Jodel Emoji Embedding Dataset (JEED1488) containing 1488 emojis and their embeddings along 300 dimensions.
Walking through Doors is Hard, even without Staircases: Proving PSPACE-hardness via Planar Assemblies of Door Gadgets
Venue: arXiv cs.CC 2020
Year: 2020

A door gadget has two states and three tunnels that can be traversed by an agent (player, robot, etc.): the "open" and "close" tunnel sets the gadget's state to open and closed, respectively, while the "traverse" tunnel can be traversed if and only if the door is in the open state. We prove that it is PSPACE-complete to decide whether an agent can move from one location to another through a planar assembly of such door gadgets, removing the traditional need for crossover gadgets and thereby simplifying past PSPACE-hardness proofs of Lemmings and Nintendo games Super Mario Bros., Legend of Zelda, and Donkey Kong Country. Our result holds in all but one of the possible local planar embedding of the open, close, and traverse tunnels within a door gadget; in the one remaining case, we prove NP-hardness. We also introduce and analyze a simpler type of door gadget, called the self-closing door. This gadget has two states and only two tunnels, similar to the "open" and "traverse" tunnels of doors, except that traversing the traverse tunnel also closes the door. In a variant called the symmetric self-closing door, the "open" tunnel can be traversed if and only if the door is closed. We prove that it is PSPACE-complete to decide whether an agent can move from one location to another through a planar assembly of either type of self-closing door. Then we apply this framework to prove new PSPACE-hardness results for eight different 3D Mario games and Sokobond.
Scalable Top-k Query on Information Networks with Hierarchical Inheritance Relations
Venue: arXiv cs.DB 2020
Year: 2020

Graph query, pattern mining and knowledge discovery become challenging on large-scale heterogeneous information networks (HINs). State-of-the-art techniques involving path propagation mainly focus on the inference on nodes labels and neighborhood structures. However, entity links in the real world also contain rich hierarchical inheritance relations. For example, the vulnerability of a product version is likely to be inherited from its older versions. Taking advantage of the hierarchical inheritances can potentially improve the quality of query results. Motivated by this, we explore hierarchical inheritance relations between entities and formulate the problem of graph query on HINs with hierarchical inheritance relations. We propose a graph query search algorithm by decomposing the original query graph into multiple star queries and apply a star query algorithm to each star query. Further candidates from each star query result are then constructed for final top-k query answers to the original query. To efficiently obtain the graph query result from a large-scale HIN, we design a bound-based pruning technique by using uniform cost search to prune search spaces. We implement our algorithm in GraphX to test the effectiveness and efficiency on synthetic and real-world datasets. Compared with two common graph query algorithms, our algorithm can effectively obtain more accurate results and competitive performances.
Independent Component Analysis for Trustworthy Cyberspace during High Impact Events: An Application to Covid-19
Venue: arXiv cs.LG 2020
Year: 2020

Social media has become an important communication channel during high impact events, such as the COVID-19 pandemic. As misinformation in social media can rapidly spread, creating social unrest, curtailing the spread of misinformation during such events is a significant data challenge. While recent solutions that are based on machine learning have shown promise for the detection of misinformation, most widely used methods include approaches that rely on either handcrafted features that cannot be optimal for all scenarios, or those that are based on deep learning where the interpretation of the prediction results is not directly accessible. In this work, we propose a data-driven solution that is based on the ICA model, such that knowledge discovery and detection of misinformation are achieved jointly. To demonstrate the effectiveness of our method and compare its performance with deep learning methods, we developed a labeled COVID-19 Twitter dataset based on socio-linguistic criteria.
Fusion of Real Time Thermal Image and 1D/2D/3D Depth Laser Readings for Remote Thermal Sensing in Industrial Plants by Means of UAVs and/or Robots
Venue: arXiv cs.RO 2020
Year: 2020

This paper presents fast procedures for thermal infrared remote sensing in dark, GPS-denied environments, such as those found in industrial plants such as in High-Voltage Direct Current (HVDC) converter stations. These procedures are based on the combination of the depth estimation obtained from either a 1-Dimensional LIDAR laser or a 2-Dimensional Hokuyo laser or a 3D MultiSense SLB laser sensor and the visible and thermal cameras from a FLIR Duo R dual-sensor thermal camera. The combination of these sensors/cameras is suitable to be mounted on Unmanned Aerial Vehicles (UAVs) and/or robots in order to provide reliable information about the potential malfunctions, which can be found within the hazardous environment. For example, the capabilities of the developed software and hardware system corresponding to the combination of the 1-D LIDAR sensor and the FLIR Duo R dual-sensor thermal camera is assessed from the point of the accuracy of results and the required computational times: the obtained computational times are under 10 ms, with a maximum localization error of 8 mm and an average standard deviation for the measured temperatures of 1.11 degree Celsius, which results are obtained for a number of test cases. The paper is structured as follows: the description of the system used for identification and localization of hotspots in industrial plants is presented in section II. In section III, the method for faults identification and localization in plants by using a 1-Dimensional LIDAR laser sensor and thermal images is described together with results. In section IV the real time thermal image processing is presented. Fusion of the 2-Dimensional depth laser Hokuyo and the thermal images is described in section V. In section VI the combination of the 3D MultiSense SLB laser and thermal images is described. In section VII a discussion and several conclusions are drawn.
A GAN-Based Image Transformation Scheme for Privacy-Preserving Deep Neural Networks
Venue: arXiv cs.CR 2020
Year: 2020

We propose a novel image transformation scheme using generative adversarial networks (GANs) for privacy-preserving deep neural networks (DNNs). The proposed scheme enables us not only to apply images without visual information to DNNs, but also to enhance robustness against ciphertext-only attacks (COAs) including DNN-based attacks. In this paper, the proposed transformation scheme is demonstrated to be able to protect visual information on plain images, and the visually-protected images are directly applied to DNNs for privacy-preserving image classification. Since the proposed scheme utilizes GANs, there is no need to manage encryption keys. In an image classification experiment, we evaluate the effectiveness of the proposed scheme in terms of classification accuracy and robustness against COAs.
Activity River: Visualizing Planned and Logged Personal Activities for Reflection
Venue: arXiv cs.HC 2020
Year: 2020

We present Activity River, a personal visualization tool which enables individuals to plan, log, and reflect on their self-defined activities. We are interested in supporting this type of reflective practice as prior work has shown that reflection can help people plan and manage their time effectively. Hence, we designed Activity River based on five design goals (visualize historical and contextual data, facilitate comparison of goals and achievements, engage viewers with delightful visuals, support authorship, and enable flexible planning and logging) which we distilled from the Information Visualization and Human-Computer Interaction literature. To explore our approach's strengths and limitations, we conducted a qualitative study of Activity River using a role-playing method. Through this qualitative exploration, we illustrate how our participants envisioned using our visualization to perform dynamic and continuous reflection on their activities. We observed that they were able to assess their progress towards their plans and adapt to unforeseen circumstances using our tool.
Distribution Aligned Multimodal and Multi-Domain Image Stylization
Venue: arXiv cs.CV 2020
Year: 2020

Multimodal and multi-domain stylization are two important problems in the field of image style transfer. Currently, there are few methods that can perform both multimodal and multi-domain stylization simultaneously. In this paper, we propose a unified framework for multimodal and multi-domain style transfer with the support of both exemplar-based reference and randomly sampled guidance. The key component of our method is a novel style distribution alignment module that eliminates the explicit distribution gaps between various style domains and reduces the risk of mode collapse. The multimodal diversity is ensured by either guidance from multiple images or random style code, while the multi-domain controllability is directly achieved by using a domain label. We validate our proposed framework on painting style transfer with a variety of different artistic styles and genres. Qualitative and quantitative comparisons with state-of-the-art methods demonstrate that our method can generate high-quality results of multi-domain styles and multimodal instances with reference style guidance or random sampled style.
BERT Based Multilingual Machine Comprehension in English and Hindi
Venue: arXiv cs.CL 2020
Year: 2020

Multilingual Machine Comprehension (MMC) is a Question-Answering (QA) sub-task that involves quoting the answer for a question from a given snippet, where the question and the snippet can be in different languages. Recently released multilingual variant of BERT (m-BERT), pre-trained with 104 languages, has performed well in both zero-shot and fine-tuned settings for multilingual tasks; however, it has not been used for English-Hindi MMC yet. We, therefore, present in this article, our experiments with m-BERT for MMC in zero-shot, mono-lingual (e.g. Hindi Question-Hindi Snippet) and cross-lingual (e.g. English QuestionHindi Snippet) fine-tune setups. These model variants are evaluated on all possible multilingual settings and results are compared against the current state-of-the-art sequential QA system for these languages. Experiments show that m-BERT, with fine-tuning, improves performance on all evaluation settings across both the datasets used by the prior model, therefore establishing m-BERT based MMC as the new state-of-the-art for English and Hindi. We also publish our results on an extended version of the recently released XQuAD dataset, which we propose to use as the evaluation benchmark for future research.
How Twitter Data Sampling Biases U.S. Voter Behavior Characterizations
Venue: arXiv cs.CY 2020
Year: 2020

Online social media are key platforms for the public to discuss political issues. As a result, researchers have used data from these platforms to analyze public opinions and forecast election results. Recent studies reveal the existence of inauthentic actors such as malicious social bots and trolls, suggesting that not every message is a genuine expression from a legitimate user. However, the prevalence of inauthentic activities in social data streams is still unclear, making it difficult to gauge biases of analyses based on such data. In this paper, we aim to close this gap using Twitter data from the 2018 U.S. midterm elections. Hyperactive accounts are over-represented in volume samples. We compare their characteristics with those of randomly sampled accounts and self-identified voters using a fast and low-cost heuristic. We show that hyperactive accounts are more likely to exhibit various suspicious behaviors and share low-credibility information compared to likely voters. Random accounts are more similar to likely voters, although they have slightly higher chances to display suspicious behaviors. Our work provides insights into biased voter characterizations when using online observations, underlining the importance of accounting for inauthentic actors in studies of political issues based on social media data.
Light-in-the-loop: using a photonics co-processor for scalable training of neural networks
Venue: arXiv cs.LG 2020
Year: 2020

As neural networks grow larger and more complex and data-hungry, training costs are skyrocketing. Especially when lifelong learning is necessary, such as in recommender systems or self-driving cars, this might soon become unsustainable. In this study, we present the first optical co-processor able to accelerate the training phase of digitally-implemented neural networks. We rely on direct feedback alignment as an alternative to backpropagation, and perform the error projection step optically. Leveraging the optical random projections delivered by our co-processor, we demonstrate its use to train a neural network for handwritten digits recognition.
Efficient Receive Beamformers for Secure Spatial Modulation against a Malicious Full-duplex Attacker with Eavesdropping Ability
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we consider a new secure spatial modulation scenario with a full-duplex (FD) malicious attacker Mallory owning eavesdropping capacity, where Mallory works on FD model and transmits a malicious jamming such as artificial noise (AN) to interfere with Bob. To suppress the malicious jamming on Bob from Mallory, a conventional maximum receive power (Max-RP) at Bob is presented firstly. Subsequently, to exploit the colored property of noise plus interference at Bob, a whitening-filter-based Max-RP (Max-WFRP) is proposed with an obvious performance enhancement over Max-RP. To completely remove the malicious jamming from Mallory, a Max-RP with a constraint of forcing the malicious jamming from Mallory to zero at Bob is proposed. To further improve secrecy rate (SR) by removing the ZF contraint (ZFC), the maximum signal-to-jamming-plus-noise ratio (Max-SJNR) is proposed. Our proposed methods have closed-form expressions. From simulation results, the four receive beamforming methods have an increasing order in performance: Max-RP, Max-RP with ZFC and Max-SJNR$\approx$Max-WFRP. Additionally, the latter two harvest a substantial performance gains over Max-RP and Max-RP with ZFC in the low and medium signal-to-noise ratio regions.
The Fine-Grained and Parallel Complexity of Andersen's Pointer Analysis
Venue: arXiv cs.PL 2020
Year: 2020

Pointer analysis is one of the fundamental problems in static program analysis. Given a set of pointers, the task is to produce a useful over-approximation of the memory locations that each pointer may point-to at runtime. The most common formulation is Andersen's Pointer Analysis (APA), defined as an inclusion-based set of $m$ pointer constraints over a set of $n$ pointers. Existing algorithms solve APA in $O(n^2\cdot m)$ time, while it has been conjectured that the problem has no truly sub-cubic algorithm, with a proof so far having remained elusive. In this work we draw a rich fine-grained and parallel complexity landscape of APA, and present upper and lower bounds. First, we establish an $O(n^3)$ upper-bound for general APA, improving over $O(n^2\cdot m)$ as $n=O(m)$. Second, we show that even on-demand APA ("may a specific pointer $a$ point to a specific location $b$?") has an $Ω(n^3)$ (combinatorial) lower bound under standard complexity-theoretic hypotheses. This formally establishes the long-conjectured "cubic bottleneck" of APA, and shows that our $O(n^3)$-time algorithm is optimal. Third, we show that under mild restrictions, APA is solvable in $\tilde{O}(n^ω)$ time, where $ω<2.373$ is the matrix-multiplication exponent. It is believed that $ω=2+o(1)$, in which case this bound becomes quadratic. Fourth, we show that even under such restrictions, even the on-demand problem has an $Ω(n^2)$ lower bound under standard complexity-theoretic hypotheses, and hence our algorithm is optimal when $ω=2+o(1)$. Fifth, we study the parallelizability of APA and establish lower and upper bounds: (i) in general, the problem is P-complete and hence unlikely parallelizable, whereas (ii) under mild restrictions, the problem is parallelizable. Our theoretical treatment formalizes several insights that can lead to practical improvements in the future.
SAT Heritage: a community-driven effort for archiving, building and running more than thousand SAT solvers
Venue: arXiv cs.AI 2020
Year: 2020

SAT research has a long history of source code and binary releases, thanks to competitions organized every year. However, since every cycle of competitions has its own set of rules and an adhoc way of publishing source code and binaries, compiling or even running any solver may be harder than what it seems. Moreover, there has been more than a thousand solvers published so far, some of them released in the early 90's. If the SAT community wants to archive and be able to keep track of all the solvers that made its history, it urgently needs to deploy an important effort. We propose to initiate a community-driven effort to archive and to allow easy compilation and running of all SAT solvers that have been released so far. We rely on the best tools for archiving and building binaries (thanks to Docker, GitHub and Zenodo) and provide a consistent and easy way for this. Thanks to our tool, building (or running) a solver from its source (or from its binary) can be done in one line.
Psychiatric Home Treatment for Inpatient Care -- Design, Implementation and Participation
Venue: arXiv cs.CY 2020
Year: 2020

The use of information and communication technologies (ICT) to support long-term care is gaining attention, also in the light of population ageing. Known in Scandinavian countries under the term of welfare technology, it aims to increase the quality of life and independence of people with physical, psychological or social impairments. In Germany, a new form of psychiatric home treatment, inpatient equivalent treatment (IET), is offered since 2018. It should allow service users with severe mental health issues to stay in their familiar environment during crisis, while being treated in the same complexity and flexibility like in an inpatient unit. However, this change in delivering healthcare services leads to sociotechnical challenges, such as coordination of work, integration into existing healthcare workflows and ensuring continuity of care. Hence, the objective of this exploratory study is to examine how information and communication technologies (ICT) interact in the new setting and how this process can be improved. Further, we also ask how service users can participate in designing home treatment services. Methodologically, this study follows a qualitative research approach. Different methods including participant observation, interviews and focus groups were conducted to answer the research questions. Data was collected during a field visit at the psychiatric department of a German clinic in summer 2019. Field notes and interviews were analyzed using the R package for qualitative data analysis RQDA. A list of socio-technical challenges and opportunities related to IET were identified. New forms of communication, gaps in documentation practices and continuity of care are seen to be highly relevant for designing and implementing home treatment services in psychiatric care. We also discuss how service users and health professionals can take pro-active part in designing these services.
WikiBERT models: deep transfer learning for many languages
Venue: arXiv cs.CL 2020
Year: 2020

Deep neural language models such as BERT have enabled substantial recent advances in many natural language processing tasks. Due to the effort and computational cost involved in their pre-training, language-specific models are typically introduced only for a small number of high-resource languages such as English. While multilingual models covering large numbers of languages are available, recent work suggests monolingual training can produce better models, and our understanding of the tradeoffs between mono- and multilingual training is incomplete. In this paper, we introduce a simple, fully automated pipeline for creating language-specific BERT models from Wikipedia data and introduce 42 new such models, most for languages up to now lacking dedicated deep neural language models. We assess the merits of these models using the state-of-the-art UDify parser on Universal Dependencies data, contrasting performance with results using the multilingual BERT model. We find that UDify using WikiBERT models outperforms the parser using mBERT on average, with the language-specific models showing substantially improved performance for some languages, yet limited improvement or a decrease in performance for others. We also present preliminary results as first steps toward an understanding of the conditions under which language-specific models are most beneficial. All of the methods and models introduced in this work are available under open licenses from https://github.com/turkunlp/wikibert.