OpenRaiser/Intern-Atlas huggingface dataset:
Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrieval
Venue: arXiv cs.CV 2020
Year: 2020

Sketch as an image search query is an ideal alternative to text in capturing the fine-grained visual details. Prior successes on fine-grained sketch-based image retrieval (FG-SBIR) have demonstrated the importance of tackling the unique traits of sketches as opposed to photos, e.g., temporal vs. static, strokes vs. pixels, and abstract vs. pixel-perfect. In this paper, we study a further trait of sketches that has been overlooked to date, that is, they are hierarchical in terms of the levels of detail -- a person typically sketches up to various extents of detail to depict an object. This hierarchical structure is often visually distinct. In this paper, we design a novel network that is capable of cultivating sketch-specific hierarchies and exploiting them to match sketch with photo at corresponding hierarchical levels. In particular, features from a sketch and a photo are enriched using cross-modal co-attention, coupled with hierarchical node fusion at every level to form a better embedding space to conduct retrieval. Experiments on common benchmarks show our method to outperform state-of-the-arts by a significant margin.
Social Influences in Recommendation Systems
Venue: arXiv cs.IR 2020
Year: 2020

Social networking sites such as Flickr and Facebook allow users to share content with family, friends, and interest groups. Also, tags can often assign to resources. In the previous research using few association rules FAR, we have seen that high-quality and efficient association-based tag recommendation is possible, but the set-up that we considered was very generic and did not take social information into account. The proposed method in the previous paper, FAR, in particular, exhibited a favorable trade-off between recommendation quality and runtime. Unfortunately, recommendation quality is unlikely to be optimal because the algorithms are not aware of any social information that may be available. Two proposed approaches take a more social view on tag recommendation regarding the issue: social contact variants and social groups of interest. The user data is varied and used as a source of associations. The adoption of social contact variants has two approaches. The first social variant is User-centered Knowledge, to contrast Collective Knowledge. It improves tag recommendation by grouping historic tag data according to friend relationships and interests. The second variant is dubbed 'social batched personomy' and attempts to address both quality and scalability issues by processing queries in batches instead of individually, such as done in a conventional personomy approach. For the social group of interest, 'community batched personomy' is proposed to provide better accuracy groups of recommendation systems in contrast also to Collective Knowledge. By taking social information into account can enhance the performance of recommendation systems.
The Return of Lexical Dependencies: Neural Lexicalized PCFGs
Venue: arXiv cs.CL 2020
Year: 2020

In this paper we demonstrate that $\textit{context free grammar (CFG) based methods for grammar induction benefit from modeling lexical dependencies}$. This contrasts to the most popular current methods for grammar induction, which focus on discovering $\textit{either}$ constituents $\textit{or}$ dependencies. Previous approaches to marry these two disparate syntactic formalisms (e.g. lexicalized PCFGs) have been plagued by sparsity, making them unsuitable for unsupervised grammar induction. However, in this work, we present novel neural models of lexicalized PCFGs which allow us to overcome sparsity problems and effectively induce both constituents and dependencies within a single model. Experiments demonstrate that this unified framework results in stronger results on both representations than achieved when modeling either formalism alone. Code is available at https://github.com/neulab/neural-lpcfg.
Prediction of hierarchical time series using structured regularization and its application to artificial neural networks
Venue: arXiv cs.LG 2020
Year: 2020

This paper discusses the prediction of hierarchical time series, where each upper-level time series is calculated by summing appropriate lower-level time series. Forecasts for such hierarchical time series should be coherent, meaning that the forecast for an upper-level time series equals the sum of forecasts for corresponding lower-level time series. Previous methods for making coherent forecasts consist of two phases: first computing base (incoherent) forecasts and then reconciling those forecasts based on their inherent hierarchical structure. With the aim of improving time series predictions, we propose a structured regularization method for completing both phases simultaneously. The proposed method is based on a prediction model for bottom-level time series and uses a structured regularization term to incorporate upper-level forecasts into the prediction model. We also develop a backpropagation algorithm specialized for application of our method to artificial neural networks for time series prediction. Experimental results using synthetic and real-world datasets demonstrate the superiority of our method in terms of prediction accuracy and computational efficiency.
Communication-Efficient Federated Learning via Optimal Client Sampling
Venue: arXiv cs.LG 2020
Year: 2020

Federated learning (FL) ameliorates privacy concerns in settings where a central server coordinates learning from data distributed across many clients. The clients train locally and communicate the models they learn to the server; aggregation of local models requires frequent communication of large amounts of information between the clients and the central server. We propose a novel, simple and efficient way of updating the central model in communication-constrained settings based on collecting models from clients with informative updates and estimating local updates that were not communicated. In particular, modeling the progression of model's weights by an Ornstein-Uhlenbeck process allows us to derive an optimal sampling strategy for selecting a subset of clients with significant weight updates. The central server collects updated local models from only the selected clients and combines them with estimated model updates of the clients that were not selected for communication. We test this policy on a synthetic dataset for logistic regression and two FL benchmarks, namely, a classification task on EMNIST and a realistic language modeling task using the Shakespeare dataset. The results demonstrate that the proposed framework provides significant reduction in communication while maintaining competitive or achieving superior performance compared to a baseline. Our method represents a new line of strategies for communication-efficient FL that is orthogonal to the existing user-local methods such as quantization or sparsification, thus complementing rather than aiming to replace those existing methods.
Detecting Suspicious Behavior: How to Deal with Visual Similarity through Neural Networks
Venue: arXiv cs.CV 2020
Year: 2020

Suspicious behavior is likely to threaten security, assets, life, or freedom. This behavior has no particular pattern, which complicates the tasks to detect it and define it. Even for human observers, it is complex to spot suspicious behavior in surveillance videos. Some proposals to tackle abnormal and suspicious behavior-related problems are available in the literature. However, they usually suffer from high false-positive rates due to different classes with high visual similarity. The Pre-Crime Behavior method removes information related to a crime commission to focus on suspicious behavior before the crime happens. The resulting samples from different types of crime have a high-visual similarity with normal-behavior samples. To address this problem, we implemented 3D Convolutional Neural Networks and trained them under different approaches. Also, we tested different values in the number-of-filter parameter to optimize computational resources. Finally, the comparison between the performance using different training approaches shows the best option to improve the suspicious behavior detection on surveillance videos.
Local Conflict Coloring Revisited: Linial for Lists
Venue: arXiv cs.DS 2020
Year: 2020

Linial's famous color reduction algorithm reduces a given $m$-coloring of a graph with maximum degree $Δ$ to a $O(Δ^2\log m)$-coloring, in a single round in the LOCAL model. We show a similar result when nodes are restricted to choose their color from a list of allowed colors: given an $m$-coloring in a directed graph of maximum outdegree $β$, if every node has a list of size $Ω(β^2 (\log β+\log\log m + \log \log |\mathcal{C}|))$ from a color space $\mathcal{C}$ then they can select a color in two rounds in the LOCAL model. Moreover, the communication of a node essentially consists of sending its list to the neighbors. This is obtained as part of a framework that also contains Linial's color reduction (with an alternative proof) as a special case. Our result also leads to a defective list coloring algorithm. As a corollary, we improve the state-of-the-art truly local $(deg+1)$-list coloring algorithm from Barenboim et al. [PODC'18] by slightly reducing the runtime to $O(\sqrt{Δ\logΔ})+\log^* n$ and significantly reducing the message size (from huge to roughly $Δ$). Our techniques are inspired by the local conflict coloring framework of Fraigniaud et al. [FOCS'16].
Black-box Adversarial Sample Generation Based on Differential Evolution
Venue: arXiv cs.LG 2020
Year: 2020

Deep Neural Networks (DNNs) are being used in various daily tasks such as object detection, speech processing, and machine translation. However, it is known that DNNs suffer from robustness problems -- perturbed inputs called adversarial samples leading to misbehaviors of DNNs. In this paper, we propose a black-box technique called Black-box Momentum Iterative Fast Gradient Sign Method (BMI-FGSM) to test the robustness of DNN models. The technique does not require any knowledge of the structure or weights of the target DNN. Compared to existing white-box testing techniques that require accessing model internal information such as gradients, our technique approximates gradients through Differential Evolution and uses approximated gradients to construct adversarial samples. Experimental results show that our technique can achieve 100% success in generating adversarial samples to trigger misclassification, and over 95% success in generating samples to trigger misclassification to a specific target output label. It also demonstrates better perturbation distance and better transferability. Compared to the state-of-the-art black-box technique, our technique is more efficient. Furthermore, we conduct testing on the commercial Aliyun API and successfully trigger its misbehavior within a limited number of queries, demonstrating the feasibility of real-world black-box attack.
Determination of 2-Adic Complexity of Generalized Binary Sequences of Order 2
Venue: arXiv cs.IT 2020
Year: 2020

The generalized binary sequences of order 2 have been used to construct good binary cyclic codes [4]. The linear complexity of these sequences has been computed in [2]. The autocorrelation values of such sequences have been determined in [1] and [3]. Some lower bounds of 2-adic complexity for such sequences have been presented in [5] and [7]. In this paper we determine the exact value of 2-adic complexity for such sequences. Particularly, we improve the lower bounds presented in [5] and [7] and the condition for the 2-adic complexity reaching the maximum value.
GIS-AHP Multi-Decision-Criteria-Analysis for the Optimal Location of Solar Energy Plants at Indonesia
Venue: arXiv cs.CY 2020
Year: 2020

A reliable tool for site-suitability assessment of solar power plants capable to account for the sustainable development and protection of cultural and biodiversity conservation areas is proposed. We present a novel Analytic Hierarchy Process (AHP) based approach for the Multi-Decision Criteria Analysis (MDCA) of SSI satellite retrieved data and local information sources, it within a GIS platform tailored to fit the needs of energy stakeholders at Indonesia, simultaneously ensuring the conservation of legally protected areas. This imposes significant challenges due to the wide diversification of cultural, natural, and ecological protected areas that need to be considered, in landmarks that demand for high resolution imaging of surface solar irradiance (SSI) within $\pm 4^{\circ}$ of the equator. To overcome these challenges, a GIS spatial weighted overlay analysis for criteria layers such as climatology, topography, electrical grid, and road infrastructure has been performed, it based on the technical and economic feasibility for solar plants deployment within three approximation schemes focused on their proximity to the existing (i) power network, (ii) road infrastructure, and (iii) community settlements. Here, we focused on the West Kalimantan Province of Borneo Island (WKP), it mainly due to its possibility of onshore inter-connectivity and energy share with Malaysia and Brunei, and the high national and international importance that brings forward the protection of the biodiversity of Borneo. It has been found that the optimal location of PV plants can be reduced to just $0.03\%$ ($46.60~km^{2}$) and $0.07\%$ ($108.58~{km}^{2}$) of WKP, in what we report as the best-suitable conditions out of the $33.05\%$ exploitable area found after the exclusion of conservation areas. This corresponds to ...
Bayesian Optimization for Developmental Robotics with Meta-Learning by Parameters Bounds Reduction
Venue: arXiv cs.RO 2020
Year: 2020

In robotics, methods and softwares usually require optimizations of hyperparameters in order to be efficient for specific tasks, for instance industrial bin-picking from homogeneous heaps of different objects. We present a developmental framework based on long-term memory and reasoning modules (Bayesian Optimisation, visual similarity and parameters bounds reduction) allowing a robot to use meta-learning mechanism increasing the efficiency of such continuous and constrained parameters optimizations. The new optimization, viewed as a learning for the robot, can take advantage of past experiences (stored in the episodic and procedural memories) to shrink the search space by using reduced parameters bounds computed from the best optimizations realized by the robot with similar tasks of the new one (e.g. bin-picking from an homogenous heap of a similar object, based on visual similarity of objects stored in the semantic memory). As example, we have confronted the system to the constrained optimizations of 9 continuous hyperparameters for a professional software (Kamido) in industrial robotic arm bin-picking tasks, a step that is needed each time to handle correctly new object. We used a simulator to create bin-picking tasks for 8 different objects (7 in simulation and one with real setup, without and with meta-learning with experiences coming from other similar objects) achieving goods results despite a very small optimization budget, with a better performance reached when meta-learning is used (84.3% vs 78.9% of success overall, with a small budget of 30 iterations for each optimization) for every object tested (p-value=0.036).
Label or Message: A Large-Scale Experimental Survey of Texts and Objects Co-Occurrence
Venue: arXiv cs.CV 2020
Year: 2020

Our daily life is surrounded by textual information. Nowadays, the automatic collection of textual information becomes possible owing to the drastic improvement of scene text detectors and recognizer. The purpose of this paper is to conduct a large-scale survey of co-occurrence between visual objects (such as book and car) and scene texts with a large image dataset and a state-of-the-art scene text detector and recognizer. Especially, we focus on the function of "label" texts, which are attached to objects for detailing the objects. By analyzing co-occurrence between objects and scene texts, it is possible to observe the statistics about the label texts and understand how the scene texts will be useful for recognizing the objects and vice versa.
Regional Rainfall Prediction Using Support Vector Machine Classification of Large-Scale Precipitation Maps
Venue: arXiv cs.LG 2020
Year: 2020

Rainfall prediction helps planners anticipate potential social and economic impacts produced by too much or too little rain. This research investigates a class-based approach to rainfall prediction from 1-30 days in advance. The study made regional predictions based on sequences of daily rainfall maps of the continental US, with rainfall quantized at 3 levels: light or no rain; moderate; and heavy rain. Three regions were selected, corresponding to three squares from a $5\times5$ grid covering the map area. Rainfall predictions up to 30 days ahead for these three regions were based on a support vector machine (SVM) applied to consecutive sequences of prior daily rainfall map images. The results show that predictions for corner squares in the grid were less accurate than predictions obtained by a simple untrained classifier. However, SVM predictions for a central region outperformed the other two regions, as well as the untrained classifier. We conclude that there is some evidence that SVMs applied to large-scale precipitation maps can under some conditions give useful information for predicting regional rainfall, but care must be taken to avoid pitfall
Evolving Context-Aware Recommender Systems With Users in Mind
Venue: arXiv cs.LG 2020
Year: 2020

A context-aware recommender system (CARS) applies sensing and analysis of user context to provide personalized services. The contextual information can be driven from sensors in order to improve the accuracy of the recommendations. Yet, generating accurate recommendations is not enough to constitute a useful system from the users' perspective, since certain contextual information may cause different issues, such as draining the user's battery, privacy issues, and more. Adding high-dimensional contextual information may increase both the dimensionality and sparsity of the model. Previous studies suggest reducing the amount of contextual information by selecting the most suitable contextual information using a domain knowledge. Another solution is compressing it into a denser latent space, thus disrupting the ability to explain the recommendation item to the user, and damaging users' trust. In this paper we present an approach for selecting low-dimensional subsets of the contextual information and incorporating them explicitly within CARS. Specifically, we present a novel feature-selection algorithm, based on genetic algorithms (GA), that outperforms SOTA dimensional-reduction CARS algorithms, improves the accuracy and the explainability of the recommendations, and allows for controlling user aspects, such as privacy and battery consumption. Furthermore, we exploit the top subsets that are generated along the evolutionary process, by learning multiple deep context-aware models and applying a stacking technique on them, thus improving the accuracy while remaining at the explicit space. We evaluated our approach on two high-dimensional context-aware datasets driven from smartphones. An empirical analysis of our results validates that our proposed approach outperforms SOTA CARS models while improving transparency and explainability to the user.
Clustering and Power Allocation for UAV-assisted NOMA-VLC Systems: A Swarm Intelligence Approach
Venue: arXiv cs.NI 2020
Year: 2020

Integrating unmanned aerial vehicles (UAV) to non-orthogonal multiple access (NOMA) visible light communications (VLC) exposes many potentials over VLC and NOMA-VLC systems. In this circumstance, user grouping is of importance to reduce the NOMA decoding complexity when the number of users is large; however, this issue has not been considered in the existing study. In this paper, we aim to maximize the weighted sum-rate of all the users by jointly optimizing UAV placement, user grouping, and power allocation in downlink NOMA-VLC systems. We first consider an efficient user clustering strategy, then apply a swarm intelligence approach, namely Harris Hawk Optimization (HHO), to solve the joint UAV placement and power allocation problem. Simulation results show outperformance of the proposed algorithm in comparison with four alternatives: OMA, NOMA without pairing, NOMA-VLC with fixed UAV placement, and random user clustering.
In-Depth Behavior Understanding and Use: The Behavior Informatics Approach
Venue: arXiv cs.SI 2020
Year: 2020

The in-depth analysis of human behavior has been increasingly recognized as a crucial means for disclosing interior driving forces, causes and impact on businesses in handling many challenging issues. The modeling and analysis of behaviors in virtual organizations is an open area. Traditional behavior modeling mainly relies on qualitative methods from behavioral science and social science perspectives. The so-called behavior analysis is actually based on human demographic and business usage data, where behavior-oriented elements are hidden in routinely collected transactional data. As a result, it is ineffective or even impossible to deeply scrutinize native behavior intention, lifecycle and impact on complex problems and business issues. We propose the approach of Behavior Informatics (BI), in order to support explicit and quantitative behavior involvement through a conversion from source data to behavioral data, and further conduct genuine analysis of behavior patterns and impacts. BI consists of key components including behavior representation, behavioral data construction, behavior impact analysis, behavior pattern analysis, behavior simulation, and behavior presentation and behavior use. We discuss the concepts of behavior and an abstract behavioral model, as well as the research tasks, process and theoretical underpinnings of BI. Substantial experiments have shown that BI has the potential to greatly complement the existing empirical and specific means by finding deeper and more informative patterns leading to greater in-depth behavior understanding. BI creates new directions and means to enhance the quantitative, formal and systematic modeling and analysis of behaviors in both physical and virtual organizations.
PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to Rewards
Venue: arXiv cs.LG 2020
Year: 2020

Reinforcement learning (RL), particularly in sparse reward settings, often requires prohibitively large numbers of interactions with the environment, thereby limiting its applicability to complex problems. To address this, several prior approaches have used natural language to guide the agent's exploration. However, these approaches typically operate on structured representations of the environment, and/or assume some structure in the natural language commands. In this work, we propose a model that directly maps pixels to rewards, given a free-form natural language description of the task, which can then be used for policy learning. Our experiments on the Meta-World robot manipulation domain show that language-based rewards significantly improves the sample efficiency of policy learning, both in sparse and dense reward settings.