OpenRaiser/Intern-Atlas huggingface dataset:
PAGANI: A Parallel Adaptive GPU Algorithm for Numerical
Venue: arXiv cs.DC 2021
Year: 2021

We present a new adaptive parallel algorithm for the challenging problem of multi-dimensional numerical integration on massively parallel architectures. Adaptive algorithms have demonstrated the best performance, but efficient many-core utilization is difficult to achieve because the adaptive work-load can vary greatly across the integration space and is impossible to predict a priori. Existing parallel algorithms utilize sequential computations on independent processors, which results in bottlenecks due to the need for data redistribution and processor synchronization. Our algorithm employs a high-throughput approach in which all existing sub-regions are processed and sub-divided in parallel. Repeated sub-region classification and filtering improves upon a brute-force approach and allows the algorithm to make efficient use of computation and memory resources. A CUDA implementation shows orders of magnitude speedup over the fastest open-source CPU method and extends the achievable accuracy for difficult integrands. Our algorithm typically outperforms other existing deterministic parallel methods.
Automatic Assembly Planning based on Digital Product Descriptions
Venue: arXiv cs.RO 2021
Year: 2021

This paper proposes a new concept in which a digital twin derived from a digital product description will automatically perform assembly planning and orchestrate the production resources in a manufacturing cell. Thus the manufacturing cell has generic services with minimal assumptions about what kind of product will be assembled, while the digital product description is designed collaboratively between the designer at an OEM and automated services at potential manufacturers. This has several advantages. Firstly, the resulting versatile manufacturing facility can handle a broad variety of products with minimal or no reconfiguration effort, so it can cost-effectively offer its services to a large number of OEMs. Secondly, a solution is presented to the problem of performing concurrent product design and assembly planning over the organizational boundary. Thirdly, the product design at the OEM is not constrained to the capabilities of specific manufacturing facilities. The concept is presented in general terms in UML and an implementation is provided in a 3D simulation environment using Automation Markup Language for digital product descriptions. Finally, two case studies are presented and applications in a real industrial context are discussed.
Lets Make A Story Measuring MR Child Engagement
Venue: arXiv cs.HC 2021
Year: 2021

We present the result of a pilot study measuring child engagement with the Lets Make A Story system, a novel mixed reality, MR, collaborative storytelling system designed for grandparents and grandchildren. We compare our MR experience against an equivalent paper story experience. The goal of our pilot was to test the system with actual child users and assess the goodness of using metrics of time, user generated story content and facial expression analysis as metrics of child engagement. We find that multiple confounding variables make these metrics problematic including attribution of engagement time, spontaneous non-story related conversation and having the childs full forward face continuously in view during the story. We present our platform and experiences and our finding that the strongest metric was user comments in the post-experiential interview.
Large-Scale Contextualised Language Modelling for Norwegian
Venue: arXiv cs.CL 2021
Year: 2021

We present the ongoing NorLM initiative to support the creation and use of very large contextualised language models for Norwegian (and in principle other Nordic languages), including a ready-to-use software environment, as well as an experience report for data preparation and training. This paper introduces the first large-scale monolingual language models for Norwegian, based on both the ELMo and BERT frameworks. In addition to detailing the training process, we present contrastive benchmark results on a suite of NLP tasks for Norwegian. For additional background and access to the data, models, and software, please see http://norlm.nlpl.eu
Developing a Conversational Recommendation System for Navigating Limited Options
Venue: arXiv cs.CL 2021
Year: 2021

We have developed a conversational recommendation system designed to help users navigate through a set of limited options to find the best choice. Unlike many internet scale systems that use a singular set of search terms and return a ranked list of options from amongst thousands, our system uses multi-turn user dialog to deeply understand the users preferences. The system responds in context to the users specific and immediate feedback to make sequential recommendations. We envision our system would be highly useful in situations with intrinsic constraints, such as finding the right restaurant within walking distance or the right retail item within a limited inventory. Our research prototype instantiates the former use case, leveraging real data from Google Places, Yelp, and Zomato. We evaluated our system against a similar system that did not incorporate user feedback in a 16 person remote study, generating 64 scenario-based search journeys. When our recommendation system was successfully triggered, we saw both an increase in efficiency and a higher confidence rating with respect to final user choice. We also found that users preferred our system (75%) compared with the baseline.
Impact of risk factors on work zone crashes using logistic models and Random Forest
Venue: arXiv cs.CY 2021
Year: 2021

Work zone safety is influenced by many risk factors. Consequently, a comprehensive knowledge of the risk factors identified from crash data analysis becomes critical in reducing risk levels and preventing severe crashes in work zones. This study focuses on the 2016 severe crashes that occurred in the State of Michigan (USA) in work zones along highway I-94. The study identified the risk factors from a wide range of crash variables characterizing environmental, driver, crash and road-related variables. The impact of these risk factors on crash severity was investigated using frequency analyses, logistic regression statistics, and a machine learning Random Forest (RF) algorithm. It is anticipated that the findings of this study will help traffic engineers and departments of transportation in developing work zone countermeasures to improve safety and reduce the crash risk. It was found that some of these factors could be overlooked when designing and devising work zone traffic control plans. Results indicate, for example, the need for appropriate traffic control mechanisms such as harmonizing the speed of vehicles before approaching work zones, the need to provide illumination at specific locations of the work zone, and the need to establish frequent public education programs, flyers, and ads targeting high-risk driver groups. Moreover, the Random Forest algorithm was found to be efficient, promising, and recommended in crash data analysis, specifically, when the data sample size is small.
ABEM: An Adaptive Agent-based Evolutionary Approach for Mining Influencers in Online Social Networks
Venue: arXiv cs.SI 2021
Year: 2021

A key step in influence maximization in online social networks is the identification of a small number of users, known as influencers, who are able to spread influence quickly and widely to other users. The evolving nature of the topological structure of these networks makes it difficult to locate and identify these influencers. In this paper, we propose an adaptive agent-based evolutionary approach to address this problem in the context of both static and dynamic networks. This approach is shown to be able to adapt the solution as the network evolves. It is also applicable to large-scale networks due to its distributed framework. Evaluation of our approach is performed by using both synthetic networks and real-world datasets. Experimental results demonstrate that the proposed approach outperforms state-of-the-art seeding algorithms in terms of maximizing influence.
Preventing Manipulation Attack in Local Differential Privacy using Verifiable Randomization Mechanism
Venue: arXiv cs.CR 2021
Year: 2021

Several randomization mechanisms for local differential privacy (LDP) (e.g., randomized response) are well-studied to improve the utility. However, recent studies show that LDP is generally vulnerable to malicious data providers in nature. Because a data collector has to estimate background data distribution only from already randomized data, malicious data providers can manipulate their output before sending, i.e., randomization would provide them plausible deniability. Attackers can skew the estimations effectively since they are calculated by normalizing with randomization probability defined in the LDP protocol, and can even control the estimations. In this paper, we show how we prevent malicious attackers from compromising LDP protocol. Our approach is to utilize a verifiable randomization mechanism. The data collector can verify the completeness of executing an agreed randomization mechanism for every data provider. Our proposed method completely protects the LDP protocol from output-manipulations, and significantly mitigates the expected damage from attacks. We do not assume any specific attacks, and it works effectively against general output-manipulation, and thus is more powerful than previously proposed countermeasures. We describe the secure version of three state-of-the-art LDP protocols and empirically show they cause acceptable overheads according to several parameters.
Learning How to Ask: Querying LMs with Mixtures of Soft Prompts
Venue: arXiv cs.CL 2021
Year: 2021

Natural-language prompts have recently been used to coax pretrained language models into performing other AI tasks, using a fill-in-the-blank paradigm (Petroni et al., 2019) or a few-shot extrapolation paradigm (Brown et al., 2020). For example, language models retain factual knowledge from their training corpora that can be extracted by asking them to "fill in the blank" in a sentential prompt. However, where does this prompt come from? We explore the idea of learning prompts by gradient descent -- either fine-tuning prompts taken from previous work, or starting from random initialization. Our prompts consist of "soft words," i.e., continuous vectors that are not necessarily word type embeddings from the language model. Furthermore, for each task, we optimize a mixture of prompts, learning which prompts are most effective and how to ensemble them. Across multiple English LMs and tasks, our approach hugely outperforms previous methods, showing that the implicit factual knowledge in language models was previously underestimated. Moreover, this knowledge is cheap to elicit: random initialization is nearly as good as informed initialization.
Computing the Union Join and Subset Graph of Acyclic Hypergraphs in Subquadratic Time
Venue: arXiv cs.DS 2021
Year: 2021

We investigate the two problems of computing the union join graph as well as computing the subset graph for acyclic hypergraphs and their subclasses. In the union join graph $G$ of an acyclic hypergraph $H$, each vertex of $G$ represents a hyperedge of $H$ and two vertices of $G$ are adjacent if there exits a join tree $T$ for $H$ such that the corresponding hyperedges are adjacent in $T$. The subset graph of a hypergraph $H$ is a directed graph where each vertex represents a hyperedge of $H$ and there is a directed edge from a vertex $u$ to a vertex $v$ if the hyperedge corresponding to $u$ is a subset of the hyperedge corresponding to $v$. For a given hypergraph $H = (V, \mathcal{E})$, let $n = |V|$, $m = |\mathcal{E}|$, and $N = \sum_{E \in \mathcal{E}} |E|$. We show that, if the Strong Exponential Time Hypothesis is true, both problems cannot be solved in $\mathcal{O} \bigl( N^{2 - \varepsilon} \bigr)$ time for $α$-acyclic hypergraphs and any constant $\varepsilon > 0$, even if the created graph is sparse. Additionally, we present algorithms that solve both problems in $\mathcal{O} \bigl( N^2 / \log N + |G| \bigr)$ time for $α$-acyclic hypergraphs, in $\mathcal{O} \bigl( N \log (n + m) + |G| \bigr)$ time for $β$-acyclic hypergaphs, and in $\mathcal{O} \bigl( N + |G| \bigr)$ time for $γ$-acyclic hypergraphs as well as for interval hypergraphs, where $|G|$ is the size of the computed graph.
A Novel Malware Detection Mechanism based on Features Extracted from Converted Malware Binary Images
Venue: arXiv cs.CR 2021
Year: 2021

Our computer systems for decades have been threatened by various types of hardware and software attacks of which Malwares have been one of them. This malware has the ability to steal, destroy, contaminate, gain unintended access, or even disrupt the entire system. There have been techniques to detect malware by performing static and dynamic analysis of malware files, but, stealthy malware has circumvented the static analysis method and for dynamic analysis, there have been previous works that propose different methods to detect malware but, in this work we propose a novel technique to detect malware. We use malware binary images and then extract different features from the same and then employ different ML-classifiers on the dataset thus obtained. We show that this technique is successful in differentiating classes of malware based on the features extracted.
Towards Time-Optimal Any-Angle Path Planning With Dynamic Obstacles
Venue: arXiv cs.AI 2021
Year: 2021

Path finding is a well-studied problem in AI, which is often framed as graph search. Any-angle path finding is a technique that augments the initial graph with additional edges to build shorter paths to the goal. Indeed, optimal algorithms for any-angle path finding in static environments exist. However, when dynamic obstacles are present and time is the objective to be minimized, these algorithms can no longer guarantee optimality. In this work, we elaborate on why this is the case and what techniques can be used to solve the problem optimally. We present two algorithms, grounded in the same idea, that can obtain provably optimal solutions to the considered problem. One of them is a naive algorithm and the other one is much more involved. We conduct a thorough empirical evaluation showing that, in certain setups, the latter algorithm might be as fast as the previously-known greedy non-optimal solver while providing solutions of better quality. In some (rare) cases, the difference in cost is up to 76%, while on average it is lower than one percent (the same cost difference is typically observed between optimal and greedy any-angle solvers in static environments).
Reward function shape exploration in adversarial imitation learning: an empirical study
Venue: arXiv cs.LG 2021
Year: 2021

For adversarial imitation learning algorithms (AILs), no true rewards are obtained from the environment for learning the strategy. However, the pseudo rewards based on the output of the discriminator are still required. Given the implicit reward bias problem in AILs, we design several representative reward function shapes and compare their performances by large-scale experiments. To ensure our results' reliability, we conduct the experiments on a series of Mujoco and Box2D continuous control tasks based on four different AILs. Besides, we also compare the performance of various reward function shapes using varying numbers of expert trajectories. The empirical results reveal that the positive logarithmic reward function works well in typical continuous control tasks. In contrast, the so-called unbiased reward function is limited to specific kinds of tasks. Furthermore, several designed reward functions perform excellently in these environments as well.
Sentence Embeddings by Ensemble Distillation
Venue: arXiv cs.CL 2021
Year: 2021

This paper contributes a new State Of The Art (SOTA) for Semantic Textual Similarity (STS). We compare and combine a number of recently proposed sentence embedding methods for STS, and propose a novel and simple ensemble knowledge distillation scheme that improves on previous approaches. Our experiments demonstrate that a model trained to learn the average embedding space from multiple ensemble students outperforms all the other individual models with high robustness. Utilizing our distillation method in combination with previous methods, we significantly improve on the SOTA unsupervised STS, and by proper hyperparameter tuning of previous methods we improve the supervised SOTA scores.
Temporally-Aware Feature Pooling for Action Spotting in Soccer Broadcasts
Venue: arXiv cs.CV 2021
Year: 2021

Toward the goal of automatic production for sports broadcasts, a paramount task consists in understanding the high-level semantic information of the game in play. For instance, recognizing and localizing the main actions of the game would allow producers to adapt and automatize the broadcast production, focusing on the important details of the game and maximizing the spectator engagement. In this paper, we focus our analysis on action spotting in soccer broadcast, which consists in temporally localizing the main actions in a soccer game. To that end, we propose a novel feature pooling method based on NetVLAD, dubbed NetVLAD++, that embeds temporally-aware knowledge. Different from previous pooling methods that consider the temporal context as a single set to pool from, we split the context before and after an action occurs. We argue that considering the contextual information around the action spot as a single entity leads to a sub-optimal learning for the pooling module. With NetVLAD++, we disentangle the context from the past and future frames and learn specific vocabularies of semantics for each subsets, avoiding to blend and blur such vocabulary in time. Injecting such prior knowledge creates more informative pooling modules and more discriminative pooled features, leading into a better understanding of the actions. We train and evaluate our methodology on the recent large-scale dataset SoccerNet-v2, reaching 53.4% Average-mAP for action spotting, a +12.7% improvement w.r.t the current state-of-the-art.
Short-term bus travel time prediction for transfer synchronization with intelligent uncertainty handling
Venue: arXiv cs.LG 2021
Year: 2021

This paper presents two novel approaches for uncertainty estimation adapted and extended for the multi-link bus travel time problem. The uncertainty is modeled directly as part of recurrent artificial neural networks, but using two fundamentally different approaches: one based on Deep Quantile Regression (DQR) and the other on Bayesian Recurrent Neural Networks (BRNN). Both models predict multiple time steps into the future, but handle the time-dependent uncertainty estimation differently. We present a sampling technique in order to aggregate quantile estimates for link level travel time to yield the multi-link travel time distribution needed for a vehicle to travel from its current position to a specific downstream stop point or transfer site. To motivate the relevance of uncertainty-aware models in the domain, we focus on the connection assurance application as a case study: An expert system to determine whether a bus driver should hold and wait for a connecting service, or break the connection and reduce its own delay. Our results show that the DQR-model performs overall best for the 80%, 90% and 95% prediction intervals, both for a 15 minute time horizon into the future (t + 1), but also for the 30 and 45 minutes time horizon (t + 2 and t + 3), with a constant, but very small underestimation of the uncertainty interval (1-4 pp.). However, we also show, that the BRNN model still can outperform the DQR for specific cases. Lastly, we demonstrate how a simple decision support system can take advantage of our uncertainty-aware travel time models to prioritize the difference in travel time uncertainty for bus holding at strategic points, thus reducing the introduced delay for the connection assurance application.
When Non-Elitism Meets Time-Linkage Problems
Venue: arXiv cs.NE 2021
Year: 2021

Many real-world applications have the time-linkage property, and the only theoretical analysis is recently given by Zheng, et al. (TEVC 2021) on their proposed time-linkage OneMax problem, OneMax$_{(0,1^n)}$. However, only two elitist algorithms (1+1)EA and ($μ$+1)EA are analyzed, and it is unknown whether the non-elitism mechanism could help to escape the local optima existed in OneMax$_{(0,1^n)}$. In general, there are few theoretical results on the benefits of the non-elitism in evolutionary algorithms. In this work, we analyze on the influence of the non-elitism via comparing the performance of the elitist (1+$λ$)EA and its non-elitist counterpart (1,$λ$)EA. We prove that with probability $1-o(1)$ (1+$λ$)EA will get stuck in the local optima and cannot find the global optimum, but with probability $1$, (1,$λ$)EA can reach the global optimum and its expected runtime is $O(n^{3+c}\log n)$ with $λ=c \log_{\frac{e}{e-1}} n$ for the constant $c\ge 1$. Noting that a smaller offspring size is helpful for escaping from the local optima, we further resort to the compact genetic algorithm where only two individuals are sampled to update the probabilistic model, and prove its expected runtime of $O(n^3\log n)$. Our computational experiments also verify the efficiency of the two non-elitist algorithms.
Monitoring Open Access at a national level: French case study
Venue: arXiv cs.DL 2021
Year: 2021

After the launch of multiple plans for Open Science, there is now a need for an accurate method or tool to monitor the Open Science trends and in particular Open Access (OA) trends. We address this requirement with a methodology that we developed and tested for France, but that could be extended to other countries. Only open data and information available on the Web are used, leveraging as much as we can large-scale systems such as Unpaywall, HAL (the main open repository in France, part of the CNRS), ORCID and IDRef (referential for French Higher Education and Research). We used rule-based and machine learning techniques to enrich the metadata of the publications. We estimate that the overall OA rate for French affiliated publications ranges from 39% to 42% between 2013 and 2017. The trend is slightly up, except for the last year, but we gather evidence that shows this is a consequence of the moving nature of the OA status. Therefore these figures should be seen as a snapshot rather than definitive. For the last observed year (2017), we show that the OA rate varies according to the publication type, the publisher and the discipline (more than 60% in Mathematics while it is about 30% in Medical research which represents the largest share in the number of publications). We describe the main challenges of our method (detection of the publications with a French affiliation, metadata enrichment with machine learning, open access status) and evaluate the errors of each step. Most of the method is not country-specific and could be applied for another perimeter. Our implementation is open sourced on the repository https://github.com/MinistereSupRecherche/bso .