OpenRaiser/Intern-Atlas huggingface dataset:
WHOSe Heritage: Classification of UNESCO World Heritage "Outstanding Universal Value" Documents with Soft Labels
Venue: arXiv cs.CL 2021
Year: 2021

The UNESCO World Heritage List (WHL) includes the exceptionally valuable cultural and natural heritage to be preserved for mankind. Evaluating and justifying the Outstanding Universal Value (OUV) is essential for each site inscribed in the WHL, and yet a complex task, even for experts, since the selection criteria of OUV are not mutually exclusive. Furthermore, manual annotation of heritage values and attributes from multi-source textual data, which is currently dominant in heritage studies, is knowledge-demanding and time-consuming, impeding systematic analysis of such authoritative documents in terms of their implications on heritage management. This study applies state-of-the-art NLP models to build a classifier on a new dataset containing Statements of OUV, seeking an explainable and scalable automation tool to facilitate the nomination, evaluation, research, and monitoring processes of World Heritage sites. Label smoothing is innovatively adapted to improve the model performance by adding prior inter-class relationship knowledge to generate soft labels. The study shows that the best models fine-tuned from BERT and ULMFiT can reach 94.3% top-3 accuracy. A human study with expert evaluation on the model prediction shows that the models are sufficiently generalizable. The study is promising to be further developed and applied in heritage research and practice.
Discrete-Time Analysis of Wireless Blockchain Networks
Venue: arXiv cs.NI 2021
Year: 2021

Blockchain (BC) technology can revolutionize future networks by providing a distributed, secure, and unalterable way to boost collaboration among operators, users, and other stakeholders. Its implementations have traditionally been supported by wired communications, with performance indicators like the high latency introduced by the BC being one of the key technology drawbacks. However, when applied to wireless communications, the performance of BC remains unknown, especially if running over contention-based networks. In this paper, we evaluate the latency performance of BC technology when the supporting communication platform is wireless, specifically we focus on IEEE 802.11ax, for the use case of users' radio resource provisioning. For that purpose, we propose a discrete-time Markov model to capture the expected delay incurred by the BC. Unlike other models in the literature, we consider the effect that timers and forks have on the end-to-end latency.
Understanding Overparameterization in Generative Adversarial Networks
Venue: arXiv cs.LG 2021
Year: 2021

A broad class of unsupervised deep learning methods such as Generative Adversarial Networks (GANs) involve training of overparameterized models where the number of parameters of the model exceeds a certain threshold. A large body of work in supervised learning have shown the importance of model overparameterization in the convergence of the gradient descent (GD) to globally optimal solutions. In contrast, the unsupervised setting and GANs in particular involve non-convex concave mini-max optimization problems that are often trained using Gradient Descent/Ascent (GDA). The role and benefits of model overparameterization in the convergence of GDA to a global saddle point in non-convex concave problems is far less understood. In this work, we present a comprehensive analysis of the importance of model overparameterization in GANs both theoretically and empirically. We theoretically show that in an overparameterized GAN model with a $1$-layer neural network generator and a linear discriminator, GDA converges to a global saddle point of the underlying non-convex concave min-max problem. To the best of our knowledge, this is the first result for global convergence of GDA in such settings. Our theory is based on a more general result that holds for a broader class of nonlinear generators and discriminators that obey certain assumptions (including deeper generators and random feature discriminators). We also empirically study the role of model overparameterization in GANs using several large-scale experiments on CIFAR-10 and Celeb-A datasets. Our experiments show that overparameterization improves the quality of generated samples across various model architectures and datasets. Remarkably, we observe that overparameterization leads to faster and more stable convergence behavior of GDA across the board.
Exploring Polarization of Users Behavior on Twitter During the 2019 South American Protests
Venue: arXiv cs.SI 2021
Year: 2021

Research across different disciplines has documented the expanding polarization in social media. However, much of it focused on the US political system or its culturally controversial topics. In this work, we explore polarization on Twitter in a different context, namely the protest that paralyzed several countries in the South American region in 2019. By leveraging users' endorsement of politicians' tweets and hashtag campaigns with defined stances towards the protest (for or against), we construct a weakly labeled stance dataset with millions of users. We explore polarization in two related dimensions: language and news consumption patterns. In terms of linguistic polarization, we apply recent insights that leveraged machine translation methods, showing that the two communities speak consistently "different" languages, mainly along ideological lines (e.g., fascist translates to communist). Our results indicate that this recently-proposed methodology is also informative in different languages and contexts than originally applied. In terms of news consumption patterns, we cluster news agencies based on homogeneity of their user bases and quantify the observed polarization in its consumption. We find empirical evidence of the "filter bubble" phenomenon during the event, as we not only show that the user bases are homogeneous in terms of stance, but the probability that a user transitions from media of different clusters is low.
What Kind of Person Wins the Turing Award?
Venue: arXiv cs.GL 2021
Year: 2021

Computer science has grown rapidly since its inception in the 1950s and the pioneers in the field are celebrated annually by the A.M. Turing Award. In this paper, we attempt to shed light on the path to influential computer scientists by examining the characteristics of the 72 Turing Award laureates. To achieve this goal, we build a comprehensive dataset of the Turing Award laureates and analyze their characteristics, including their personal information, family background, academic background, and industry experience. The FP-Growth algorithm is used for frequent feature mining. Logistic regression plot, pie chart, word cloud and map are generated accordingly for each of the interesting features to uncover insights regarding personal factors that drive influential work in the field of computer science. In particular, we show that the Turing Award laureates are most commonly white, male, married, United States citizen, and received a PhD degree. Our results also show that the age at which the laureate won the award increases over the years; most of the Turing Award laureates did not major in computer science; birth order is strongly related to the winners' success; and the number of citations is not as important as one would expect.
End-to-End Mandarin Tone Classification with Short Term Context Information
Venue: arXiv cs.SD 2021
Year: 2021

In this paper, we propose an end-to-end Mandarin tone classification method from continuous speech utterances utilizing both the spectrogram and the short-term context information as the input. Both spectrograms and context segment features are used to train the tone classifier. We first divide the spectrogram frames into syllable segments using force alignment results produced by an ASR model. Then we extract the short-term segment features to capture the context information across multiple syllables. Feeding both the spectrogram and the short-term context segment features into an end-to-end model could significantly improve the performance. Experiments are performed on a large-scale open-source Mandarin speech dataset to evaluate the proposed method. Results show that this method improves the classification accuracy from 79.5% to 92.6% on the AISHELL3 database.
Stochastic Binning and Coded Demixing for Unsourced Random Access
Venue: arXiv cs.IT 2021
Year: 2021

Unsourced random access is a novel communication paradigm designed for handling a large number of uncoordinated users that sporadically transmit very short messages. Under this model, coded compressed sensing (CCS) has emerged as a low-complexity scheme that exhibits good error performance. Yet, one of the challenges faced by CCS pertains to disentangling a large number of codewords present on a single factor graph. To mitigate this issue, this article introduces a modified CCS scheme whereby active devices stochastically partition themselves into groups that utilize separate sampling matrices with low cross-coherence for message transmission. At the receiver, ideas from the field of compressed demixing are employed for support recovery, and separate factor graphs are created for message disambiguation in each cluster. This reduces the number of active users on a factor graph, which improves performance significantly in typical scenarios. Indeed, coded demixing reduces the probability of error as the number of groups increases, up to a point. Findings are supported with numerical simulations.
Holistic Image Manipulation Detection using Pixel Co-occurrence Matrices
Venue: arXiv cs.CV 2021
Year: 2021

Digital image forensics aims to detect images that have been digitally manipulated. Realistic image forgeries involve a combination of splicing, resampling, region removal, smoothing and other manipulation methods. While most detection methods in literature focus on detecting a particular type of manipulation, it is challenging to identify doctored images that involve a host of manipulations. In this paper, we propose a novel approach to holistically detect tampered images using a combination of pixel co-occurrence matrices and deep learning. We extract horizontal and vertical co-occurrence matrices on three color channels in the pixel domain and train a model using a deep convolutional neural network (CNN) framework. Our method is agnostic to the type of manipulation and classifies an image as tampered or untampered. We train and validate our model on a dataset of more than 86,000 images. Experimental results show that our approach is promising and achieves more than 0.99 area under the curve (AUC) evaluation metric on the training and validation subsets. Further, our approach also generalizes well and achieves around 0.81 AUC on an unseen test dataset comprising more than 19,740 images released as part of the Media Forensics Challenge (MFC) 2020. Our score was highest among all other teams that participated in the challenge, at the time of announcement of the challenge results.
Multi-View Image-to-Image Translation Supervised by 3D Pose
Venue: arXiv cs.CV 2021
Year: 2021

We address the task of multi-view image-to-image translation for person image generation. The goal is to synthesize photo-realistic multi-view images with pose-consistency across all views. Our proposed end-to-end framework is based on a joint learning of multiple unpaired image-to-image translation models, one per camera viewpoint. The joint learning is imposed by constraints on the shared 3D human pose in order to encourage the 2D pose projections in all views to be consistent. Experimental results on the CMU-Panoptic dataset demonstrate the effectiveness of the suggested framework in generating photo-realistic images of persons with new poses that are more consistent across all views in comparison to a standard Image-to-Image baseline. The code is available at: https://github.com/sony-si/MultiView-Img2Img
Efficient Optimal Transport Algorithm by Accelerated Gradient descent
Venue: arXiv cs.LG 2021
Year: 2021

Optimal transport (OT) plays an essential role in various areas like machine learning and deep learning. However, computing discrete optimal transport plan for large scale problems with adequate accuracy and efficiency is still highly challenging. Recently, methods based on the Sinkhorn algorithm add an entropy regularizer to the prime problem and get a trade off between efficiency and accuracy. In this paper, we propose a novel algorithm to further improve the efficiency and accuracy based on Nesterov's smoothing technique. Basically, the non-smooth c-transform of the Kantorovich potential is approximated by the smooth Log-Sum-Exp function, which finally smooths the original non-smooth Kantorovich dual functional (energy). The smooth Kantorovich functional can be optimized by the fast proximal gradient algorithm (FISTA) efficiently. Theoretically, the computational complexity of the proposed method is given by $O(n^{\frac{5}{2}} \sqrt{\log n} /ε)$, which is lower than that of the Sinkhorn algorithm. Empirically, compared with the Sinkhorn algorithm, our experimental results demonstrate that the proposed method achieves faster convergence and better accuracy with the same parameter.
What's Your Value of Travel Time? Collecting Traveler-Centered Mobility Data via Crowdsourcing
Venue: arXiv cs.CY 2021
Year: 2021

Mobility and transport, by their nature, involve crowds and require the coordination of multiple stakeholders - such as policy-makers, planners, transport operators, and the travelers themselves. However, traditional approaches have been focused on time savings, proposing to users solutions that include the shortest or fastest paths. We argue that this approach towards travel time value is not centered on a traveler's perspective. To date, very few works have mined data from crowds of travelers to test the efficacy and efficiency of novel mobility paradigms. In this paper, we build upon a different paradigm of worthwhile time in which travelers can use their travel time for other activities; we present a new dataset, which contains data about travelers and their journeys, collected from a dedicated mobile application. Each trip contains multi-faceted information: from the transport mode, through its evaluation, to the positive and negative experience factors. To showcase this new dataset's potential, we also present a use case, which compares corresponding trip legs with different transport modes, studying experience factors that negatively impact users using cycling and public transport as alternatives to cars. We conclude by discussing other application domains and research opportunities enabled by the dataset.
Enhancing User' s Income Estimation with Super-App Alternative Data
Venue: arXiv cs.LG 2021
Year: 2021

This paper presents the advantages of alternative data from Super-Apps to enhance user' s income estimation models. It compares the performance of these alternative data sources with the performance of industry-accepted bureau income estimators that takes into account only financial system information; successfully showing that the alternative data manage to capture information that bureau income estimators do not. By implementing the TreeSHAP method for Stochastic Gradient Boosting Interpretation, this paper highlights which of the customer' s behavioral and transactional patterns within a Super-App have a stronger predictive power when estimating user' s income. Ultimately, this paper shows the incentive for financial institutions to seek to incorporate alternative data into constructing their risk profiles.
Generative Design of NU's Husky Carbon, A Morpho-Functional, Legged Robot
Venue: arXiv cs.RO 2021
Year: 2021

We report the design of a morpho-functional robot called Husky Carbon. Our goal is to integrate two forms of mobility, aerial and quadrupedal-legged locomotion, within a single platform. There are prohibitive design restrictions such as tight power budget and payload, which can particularly become important in aerial flights. To address these challenges, we pose a problem called the Mobility Value of Added Mass (MVAM) problem. In the MVAM problem, we attempt to allocate mass in our designs such that the energetic performance is affected the least. To solve the MVAM problem, we adopted a generative design approach using Grasshopper's evolutionary solver to synthesize a parametric design space for Husky. Then, this space was searched for the morphologies that could yield a minimized Total Cost Of Transport (TCOT) and payload. This approach revealed that a front-heavy quadrupedal robot can achieve a lower TCOT while retaining larger margins on allowable added mass to its design. Based on this framework Husky was built and tested as a front-heavy robot.
Remote Recording of Emotional and Activity Data: A Methodological Study
Venue: arXiv cs.HC 2021
Year: 2021

The impact of physical exercise on emotional well-being is one of the most important factors that drive sustained physical activity engagement. It is also one of the least studied topics on account of the rather elaborated setups required to quantify emotional expression during exercise. The wide adoption of at-home, physical exercise solutions has compounded this problem due to the secluded nature of these activities. We propose here a new methodology that would allow for mass emotional expression and physical activity data gathering using publicly available sources such as at-home exercise videos. We have shown that the methodology is robust enough to extract high resolution, reliable data from home videos posted on popular video share sites. The source-code and instructions for practical use are published online such that researchers can access data pertinent to emotional response during exertion in real world situations with applicability for prospective and retrospective studies.
Balboa: Bobbing and Weaving around Network Censorship
Venue: arXiv cs.CR 2021
Year: 2021

We introduce Balboa, a link obfuscation framework for censorship circumvention. Balboa provides a general framework for tunneling data through existing applications. Balboa sits between an application and the operating system, intercepting outgoing network traffic and rewriting it to embed data. To avoid introducing any distinguishable divergence from the expected application behavior, Balboa only rewrites traffic that matches an externally specified \emph{traffic model} pre-shared between the communicating parties. The traffic model captures some subset of the network traffic (e.g., some subset of music an audio streaming server streams). The sender uses this model to replace outgoing data with a pointer to the associated location in the model and embed data in the freed up space. The receiver then extracts the data, replacing the pointer with the original data from the model before passing the data on to the application. When using TLS, this approach means that application behavior with Balboa is \emph{equivalent}, modulo small (protocol-dependent) timing differences, to if the application was running without Balboa. Balboa differs from prior approaches in that it (1) provides a framework for tunneling data through arbitrary (TLS-protected) protocols/applications, and (2) runs the unaltered application binaries on standard inputs, as opposed to most prior tunneling approaches which run the application on non-standard -- and thus potentially distinguishable -- inputs. We present two instantiations of Balboa -- one for audio streaming and one for web browsing -- and demonstrate the difficulty of identifying Balboa by a machine learning classifier.
Gradient Kernel Regression
Venue: arXiv cs.AI 2021
Year: 2021

In this article a surprising result is demonstrated using the neural tangent kernel. This kernel is defined as the inner product of the vector of the gradient of an underlying model evaluated at training points. This kernel is used to perform kernel regression. The surprising thing is that the accuracy of that regression is independent of the accuracy of the underlying network.
On the validity of kernel approximations for orthogonally-initialized neural networks
Venue: arXiv cs.LG 2021
Year: 2021

In this note we extend kernel function approximation results for neural networks with Gaussian-distributed weights to single-layer networks initialized using Haar-distributed random orthogonal matrices (with possible rescaling). This is accomplished using recent results from random matrix theory.
NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Media
Venue: arXiv cs.CV 2021
Year: 2021

Online misinformation is a prevalent societal issue, with adversaries relying on tools ranging from cheap fakes to sophisticated deep fakes. We are motivated by the threat scenario where an image is used out of context to support a certain narrative. While some prior datasets for detecting image-text inconsistency generate samples via text manipulation, we propose a dataset where both image and text are unmanipulated but mismatched. We introduce several strategies for automatically retrieving convincing images for a given caption, capturing cases with inconsistent entities or semantic context. Our large-scale automatically generated NewsCLIPpings Dataset: (1) demonstrates that machine-driven image repurposing is now a realistic threat, and (2) provides samples that represent challenging instances of mismatch between text and image in news that are able to mislead humans. We benchmark several state-of-the-art multimodal models on our dataset and analyze their performance across different pretraining domains and visual backbones.
Inertial Collaborative Localisation for Autonomous Vehicles using a Minimum Energy Filter
Venue: arXiv cs.RO 2021
Year: 2021

Collaborative Localisation has been studied extensively in recent years as a way to improve pose estimation of unmanned aerial vehicles in challenging environments. However little attention has been paid toward advancing the underlying filter design beyond standard Extended Kalman Filter-based approaches. In this paper, we detail a discrete-time collaborative localisation filter using the deterministic minimum-energy framework. The filter incorporates measurements from an inertial measurement unit and models the effects of sensor bias and gravitational acceleration. We present a simulation based on real-world vehicle trajectories and IMU data that demonstrates how collaborative localisation can improve performance over single-vehicle methods.
MESD: Exploring Optical Flow Assessment on Edge of Motion Objects with Motion Edge Structure Difference
Venue: arXiv cs.CV 2021
Year: 2021

The optical flow estimation has been assessed in various applications. In this paper, we propose a novel method named motion edge structure difference(MESD) to assess estimation errors of optical flow fields on edge of motion objects. We implement comparison experiments for MESD by evaluating five representative optical flow algorithms on four popular benchmarks: MPI Sintel, Middlebury, KITTI 2012 and KITTI 2015. Our experimental results demonstrate that MESD can reasonably and discriminatively assess estimation errors of optical flow fields on motion edge. The results indicate that MESD could be a supplementary metric to existing general assessment metrics for evaluating optical flow algorithms in related computer vision applications.