OpenRaiser/Intern-Atlas huggingface dataset:
Screen2Vec: Semantic Embedding of GUI Screens and GUI Components
Venue: arXiv cs.HC 2021
Year: 2021

Representing the semantics of GUI screens and components is crucial to data-driven computational methods for modeling user-GUI interactions and mining GUI designs. Existing GUI semantic representations are limited to encoding either the textual content, the visual design and layout patterns, or the app contexts. Many representation techniques also require significant manual data annotation efforts. This paper presents Screen2Vec, a new self-supervised technique for generating representations in embedding vectors of GUI screens and components that encode all of the above GUI features without requiring manual annotation using the context of user interaction traces. Screen2Vec is inspired by the word embedding method Word2Vec, but uses a new two-layer pipeline informed by the structure of GUIs and interaction traces and incorporates screen- and app-specific metadata. Through several sample downstream tasks, we demonstrate Screen2Vec's key useful properties: representing between-screen similarity through nearest neighbors, composability, and capability to represent user tasks.
In-IDE Code Generation from Natural Language: Promise and Challenges
Venue: arXiv cs.SE 2021
Year: 2021

A great part of software development involves conceptualizing or communicating the underlying procedures and logic that needs to be expressed in programs. One major difficulty of programming is turning concept into code, especially when dealing with the APIs of unfamiliar libraries. Recently, there has been a proliferation of machine learning methods for code generation and retrieval from natural language queries, but these have primarily been evaluated purely based on retrieval accuracy or overlap of generated code with developer-written code, and the actual effect of these methods on the developer workflow is surprisingly unattested. We perform the first comprehensive investigation of the promise and challenges of using such technology inside the IDE, asking "at the current state of technology does it improve developer productivity or accuracy, how does it affect the developer experience, and what are the remaining gaps and challenges?" We first develop a plugin for the IDE that implements a hybrid of code generation and code retrieval functionality, and orchestrate virtual environments to enable collection of many user events. We ask developers with various backgrounds to complete 14 Python programming tasks ranging from basic file manipulation to machine learning or data visualization, with or without the help of the plugin. While qualitative surveys of developer experience are largely positive, quantitative results with regards to increased productivity, code quality, or program correctness are inconclusive. Analysis identifies several pain points that could improve the effectiveness of future machine learning based code generation/retrieval developer assistants, and demonstrates when developers prefer code generation over code retrieval and vice versa. We release all data and software to pave the road for future empirical studies and development of better models.
Optimal Oracles for Point-to-Set Principles
Venue: arXiv cs.CC 2021
Year: 2021

The point-to-set principle \cite{LutLut17} characterizes the Hausdorff dimension of a subset $E\subseteq\R^n$ by the \textit{effective} (or algorithmic) dimension of its individual points. This characterization has been used to prove several results in classical, i.e., without any computability requirements, analysis. Recent work has shown that algorithmic techniques can be fruitfully applied to Marstrand's projection theorem, a fundamental result in fractal geometry. In this paper, we introduce an extension of point-to-set principle - the notion of \textit{optimal oracles} for subsets $E\subseteq\R^n$. One of the primary motivations of this definition is that, if $E$ has optimal oracles, then the conclusion of Marstrand's projection theorem holds for $E$. We show that every analytic set has optimal oracles. We also prove that if the Hausdorff and packing dimensions of $E$ agree, then $E$ has optimal oracles. Moreover, we show that the existence of sufficiently nice outer measures on $E$ implies the existence of optimal Hausdorff oracles. In particular, the existence of exact gauge functions for a set $E$ is sufficient for the existence of optimal Hausdorff oracles, and is therefore sufficient for Marstrand's theorem. Thus, the existence of optimal oracles extends the currently known sufficient conditions for Marstrand's theorem to hold. Under certain assumptions, every set has optimal oracles. However, assuming the axiom of choice and the continuum hypothesis, we construct sets which do not have optimal oracles. This construction naturally leads to a generalization of Davies theorem on projections.
On Small-World Networks: Survey and Properties Analysis
Venue: arXiv cs.CY 2021
Year: 2021

Complex networks has been a hot topic of research over the past several years over crossing many disciplines, starting from mathematics and computer science and ending by the social and biological sciences. Random graphs were studied to observe the qualitative features they have in common in planetary scale data sets which helps us to project the insights proven to real world networks. In this paper, We survey the particular case of small-world phenomena and decentralized search algorithms. We start by explaining the first empirical study for the six degrees of separation phenomenon in social networks; then we review some of the probabilistic network models based on this work, elaborating how these models tried to explain the phenomenon properties, and lastly, we review few of the recent empirical studies empowered by these models. Finally, some future works are proposed in this area of research.
Automated Crop Field Surveillance using Computer Vision
Venue: arXiv cs.CV 2021
Year: 2021

Artificial Intelligence is everywhere today. But unfortunately, Agriculture has not been able to get that much attention from Artificial Intelligence (AI). A lack of automation persists in the agriculture industry. For over many years, farmers and crop field owners have been facing a problem of trespassing of wild animals for which no feasible solution has been provided. Installing a fence or barrier like structure is neither feasible nor efficient due to the large areas covered by the fields. Also, if the landowner can afford to build a wall or barrier, government policies for building walls are often very irksome. The paper intends to give a simple intelligible solution to the problem with Automated Crop Field Surveillance using Computer Vision. The solution will significantly reduce the cost of crops destroyed annually and completely automate the security of the field.
Im2Mesh GAN: Accurate 3D Hand Mesh Recovery from a Single RGB Image
Venue: arXiv cs.CV 2021
Year: 2021

This work addresses hand mesh recovery from a single RGB image. In contrast to most of the existing approaches where the parametric hand models are employed as the prior, we show that the hand mesh can be learned directly from the input image. We propose a new type of GAN called Im2Mesh GAN to learn the mesh through end-to-end adversarial training. By interpreting the mesh as a graph, our model is able to capture the topological relationship among the mesh vertices. We also introduce a 3D surface descriptor into the GAN architecture to further capture the 3D features associated. We experiment two approaches where one can reap the benefits of coupled groundtruth data availability of images and the corresponding meshes, while the other combats the more challenging problem of mesh estimations without the corresponding groundtruth. Through extensive evaluations we demonstrate that the proposed method outperforms the state-of-the-art.
Modeling opinion leader's role in the diffusion of innovation
Venue: arXiv cs.SI 2021
Year: 2021

The diffusion of innovations is an important topic for the consumer markets. Early research focused on how innovations spread on the level of the whole society. To get closer to the real world scenarios agent based models (ABM) started focusing on individual-level agents. In our work we will translate an existing ABM that investigates the role of opinion leaders in the process of diffusion of innovations to a new, more expressive platform designed for agent based modeling, GAMA. We will do it to show that taking advantage of new features of the chosen platform should be encouraged when making models in the field of social sciences in the future, because it can be beneficial for the explanatory power of simulation results.
FedH2L: Federated Learning with Model and Statistical Heterogeneity
Venue: arXiv cs.LG 2021
Year: 2021

Federated learning (FL) enables distributed participants to collectively learn a strong global model without sacrificing their individual data privacy. Mainstream FL approaches require each participant to share a common network architecture and further assume that data are are sampled IID across participants. However, in real-world deployments participants may require heterogeneous network architectures; and the data distribution is almost certainly non-uniform across participants. To address these issues we introduce FedH2L, which is agnostic to both the model architecture and robust to different data distributions across participants. In contrast to approaches sharing parameters or gradients, FedH2L relies on mutual distillation, exchanging only posteriors on a shared seed set between participants in a decentralized manner. This makes it extremely bandwidth efficient, model agnostic, and crucially produces models capable of performing well on the whole data distribution when learning from heterogeneous silos.
Multilingual and cross-lingual document classification: A meta-learning approach
Venue: arXiv cs.CL 2021
Year: 2021

The great majority of languages in the world are considered under-resourced for the successful application of deep learning methods. In this work, we propose a meta-learning approach to document classification in limited-resource setting and demonstrate its effectiveness in two different settings: few-shot, cross-lingual adaptation to previously unseen languages; and multilingual joint training when limited target-language data is available during training. We conduct a systematic comparison of several meta-learning methods, investigate multiple settings in terms of data availability and show that meta-learning thrives in settings with a heterogeneous task distribution. We propose a simple, yet effective adjustment to existing meta-learning methods which allows for better and more stable learning, and set a new state of the art on several languages while performing on-par on others, using only a small amount of labeled data.
Developing for personalised learning: the long road from educational objectives to development and feedback
Venue: arXiv cs.HC 2021
Year: 2021

This paper describes the development needed to support the functional and teaching requirements of iRead, a 4-year EU-funded project which produced an award-winning serious game utilising lexical and syntactical game content. The main functional requirement was that the game should retain different profiles for each student, encapsulating both the respective language model (which language features should be taught/used in the game first, before moving on to more advanced ones) and the user model (mastery level for each feature, as reported by the student's performance in the game). In addition to this, researchers and stakeholders stated additional requirements related to learning objectives and strategies to make the game more interesting and successful; these were implemented as a set of selection rules which take into account not only the mastery level for each feature, but also respect the priorities set by teachers, helping avoid repetition of content and features, and maintaining a balance between new content and revision of already mastered features to give students the sense of progress, while also reinforcing learning.
Detecting discriminatory risk through data annotation based on Bayesian inferences
Venue: arXiv cs.LG 2021
Year: 2021

Thanks to the increasing growth of computational power and data availability, the research in machine learning has advanced with tremendous rapidity. Nowadays, the majority of automatic decision making systems are based on data. However, it is well known that machine learning systems can present problematic results if they are built on partial or incomplete data. In fact, in recent years several studies have found a convergence of issues related to the ethics and transparency of these systems in the process of data collection and how they are recorded. Although the process of rigorous data collection and analysis is fundamental in the model design, this step is still largely overlooked by the machine learning community. For this reason, we propose a method of data annotation based on Bayesian statistical inference that aims to warn about the risk of discriminatory results of a given data set. In particular, our method aims to deepen knowledge and promote awareness about the sampling practices employed to create the training set, highlighting that the probability of success or failure conditioned to a minority membership is given by the structure of the data available. We empirically test our system on three datasets commonly accessed by the machine learning community and we investigate the risk of racial discrimination.
Learning Abstract Representations through Lossy Compression of Multi-Modal Signals
Venue: arXiv cs.LG 2021
Year: 2021

A key competence for open-ended learning is the formation of increasingly abstract representations useful for driving complex behavior. Abstract representations ignore specific details and facilitate generalization. Here we consider the learning of abstract representations in a multi-modal setting with two or more input modalities. We treat the problem as a lossy compression problem and show that generic lossy compression of multimodal sensory input naturally extracts abstract representations that tend to strip away modalitiy specific details and preferentially retain information that is shared across the different modalities. Furthermore, we propose an architecture to learn abstract representations by identifying and retaining only the information that is shared across multiple modalities while discarding any modality specific information.
Variable Petri Nets for Mobility
Venue: arXiv cs.SE 2021
Year: 2021

Mobile computing systems, service-based systems and some other systems with mobile interacting components have recently received much attention. However, because of their characteristics such as mobility and disconnection, it is difficult to model and analyze them by using a structure-fixed model. This work proposes a new Petri net model called Variable Petri Net (VPN) for modeling and analyzing these systems. The definition, firing rule, and related analysis technology of VPN are introduced in detail. In a VPN, the possible interaction interfaces are abstracted as a new kind of places called virtual places, and the occurrences of (dis)connections are described by new functions, which makes it appropriate to describe the component collaboration in systems and realize the scalability and pluggability of systems. Moreover, to overcome the shortcoming that markings cannot reflect link capability of a system, VPNs add a constraint function along with a marking to represent a complete system configuration. Several examples are used to demonstrate the newly proposed model and method.
Why polls fail to predict elections
Venue: arXiv cs.SI 2021
Year: 2021

In the past decade we have witnessed the failure of traditional polls in predicting presidential election outcomes across the world. To understand the reasons behind these failures we analyze the raw data of a trusted pollster which failed to predict, along with the rest of the pollsters, the surprising 2019 presidential election in Argentina which has led to a major market collapse in that country. Analysis of the raw and re-weighted data from longitudinal surveys performed before and after the elections reveals clear biases (beyond well-known low-response rates) related to mis-representation of the population and, most importantly, to social-desirability biases, i.e., the tendency of respondents to hide their intention to vote for controversial candidates. We then propose a longitudinal opinion tracking method based on big-data analytics from social media, machine learning, and network theory that overcomes the limits of traditional polls. The model achieves accurate results in the 2019 Argentina elections predicting the overwhelming victory of the candidate Alberto Fernández over the president Mauricio Macri; a result that none of the traditional pollsters in the country was able to predict. Beyond predicting political elections, the framework we propose is more general and can be used to discover trends in society; for instance, what people think about economics, education or climate change.
geoGAT: Graph Model Based on Attention Mechanism for Geographic Text Classification
Venue: arXiv cs.CL 2021
Year: 2021

In the area of geographic information processing. There are few researches on geographic text classification. However, the application of this task in Chinese is relatively rare. In our work, we intend to implement a method to extract text containing geographical entities from a large number of network text. The geographic information in these texts is of great practical significance to transportation, urban and rural planning, disaster relief and other fields. We use the method of graph convolutional neural network with attention mechanism to achieve this function. Graph attention networks is an improvement of graph convolutional neural networks. Compared with GCN, the advantage of GAT is that the attention mechanism is proposed to weight the sum of the characteristics of adjacent nodes. In addition, We construct a Chinese dataset containing geographical classification from multiple datasets of Chinese text classification. The Macro-F Score of the geoGAT we used reached 95\% on the new Chinese dataset.
Pseudo-Adaptive Penalization to Handle Constraints in Particle Swarm Optimizers
Venue: arXiv cs.NE 2021
Year: 2021

The penalization method is a popular technique to provide particle swarm optimizers with the ability to handle constraints. The downside is the need of penalization coefficients whose settings are problem-specific. While adaptive coefficients can be found in the literature, a different adaptive scheme is proposed in this paper, where coefficients are kept constant. A pseudo-adaptive relaxation of the tolerances for constraint violations while penalizing only violations beyond such tolerances results in a pseudo-adaptive penalization. A particle swarm optimizer is tested on a suite of benchmark problems for three types of tolerance relaxation: no relaxation; self-tuned initial relaxation with deterministic decrease; and self-tuned initial relaxation with pseudo-adaptive decrease. Other authors' results are offered as frames of reference.
An Ultra-Specific Image Dataset for Automated Insect Identification
Venue: arXiv cs.MM 2021
Year: 2021

Automated identification of insects is a tough task where many challenges like data limitation, imbalanced data count, and background noise needs to be overcome for better performance. This paper describes such an image dataset which consists of a limited, imbalanced number of images regarding six genera of subfamily Cicindelinae (tiger beetles) of order Coleoptera. The diversity of image collection is at a high level as the images were taken from different sources, angles and on different scales. Thus, the salient regions of the images have a large variation. Therefore, one of the main intentions in this process was to get an idea about the image dataset while comparing different unique patterns and features in images. The dataset was evaluated on different classification algorithms including deep learning models based on different approaches to provide a benchmark. The dynamic nature of the dataset poses a challenge to the image classification algorithms. However transfer learning models using softmax classifier performed well on current dataset. The tiger beetle classification can be challenging even to a trained human eye, therefore, this dataset opens a new avenue for the classification algorithms to develop, to identify features which human eyes have not identified.
Medical Segment Coloring of Clinical Notes
Venue: arXiv cs.CL 2021
Year: 2021

This paper proposes a deep learning-based method to identify the segments of a clinical note corresponding to ICD-9 broad categories which are further color-coded with respect to 17 ICD-9 categories. The proposed Medical Segment Colorer (MSC) architecture is a pipeline framework that works in three stages: (1) word categorization, (2) phrase allocation, and (3) document classification. MSC uses gated recurrent unit neural networks (GRUs) to map from an input document to word multi-labels to phrase allocations, and uses statistical median to map phrase allocation to document multi-label. We compute variable length segment coloring from overlapping phrase allocation probabilities. These cross-level bidirectional contextual links identify adaptive context and then produce segment coloring. We train and evaluate MSC using the document labeled MIMIC-III clinical notes. Training is conducted solely using document multi-labels without any information on phrases, segments, or words. In addition to coloring a clinical note, MSC generates as byproducts document multi-labeling and word tagging -- creation of ICD9 category keyword lists based on segment coloring. Performance comparison of MSC byproduct document multi-labels versus methods whose purpose is to produce justifiable document multi-labels is 64% vs 52.4% micro-average F1-score against the CAML (CNN attention multi label) method. For evaluation of MSC segment coloring results, medical practitioners independently assigned the colors to broad ICD9 categories given a sample of 40 colored notes and a sample of 50 words related to each category based on the word tags. Binary scoring of this evaluation has a median value of 83.3% and mean of 63.7%.
Deriving the Traveler Behavior Information from Social Media: A Case Study in Manhattan with Twitter
Venue: arXiv cs.SI 2021
Year: 2021

Social media platforms, such as Twitter, provide a totally new perspective in dealing with the traffic problems and is anticipated to complement the traditional methods. The geo-tagged tweets can provide the Twitter users' location information and is being applied in traveler behavior analysis. This paper explores the full potentials of Twitter in deriving travel behavior information and conducts a case study in Manhattan Area. A systematic method is proposed to extract displacement information from Twitter locations. Our study shows that Twitter has a unique demographics which combine not only local residents but also the tourists or passengers. For individual user, Twitter can uncover his/her travel behavior features including the time-of-day and location distributions on both weekdays and weekends. For all Twitter users, the aggregated travel behavior results also show that the time-of-day travel patterns in Manhattan Island resemble that of the traffic flow; the identification of OD pattern is also promising by comparing with the results of travel survey.