OpenRaiser/Intern-Atlas huggingface dataset:
Exploring the Research Landscape of Pakistan: A Data-driven Analysis of Scopus Indexed Scientific Literature
Venue: arXiv cs.DL 2022
Year: 2022

Scientific contribution and research performance of a university, research group, or institute needs to be evaluated all the more with the increasing volume and fast-developing disciplines of research. The need of the time is to develop tools for strategic planning and management that will help research bodies to rank and benchmark themselves against international standards. This will enable them to invest appropriately in research areas of promising strength and gain maximally from them, thus fulfilling the ultimate purpose of positive impact of research on society. Our tool is capable of rating and benchmarking universities as well as research institutes in not only the major disciplines and sub-disciplines, but at the finest level of niche areas of science and technology too with the help of its innovative bibliometric indicators based on publications and citation analysis. The tool accepts inputs like discipline/subject area, university, and country and time window while using data retrieved from bibliography database, Scopus, to benchmark and rate the research body under consideration. We have evaluated that there are many niche subject areas in which small or medium size universities are performing good in comparison to the large universities. Most of these subject areas are of more significance in the present day and the future. Government and funds allocating bodies should take this factor in account that investing the right money at right place will give far better results than we they are having right now
Perception-Aware Perching on Powerlines with Multirotors
Venue: arXiv cs.RO 2022
Year: 2022

Multirotor aerial robots are becoming widely used for the inspection of powerlines. To enable continuous, robust inspection without human intervention, the robots must be able to perch on the powerlines to recharge their batteries. Highly versatile perching capabilities are necessary to adapt to the variety of configurations and constraints that are present in real powerline systems. This paper presents a novel perching trajectory generation framework that computes perception-aware, collision-free, and dynamically-feasible maneuvers to guide the robot to the desired final state. Trajectory generation is achieved via solving a Nonlinear Programming problem using the Primal-Dual Interior Point method. The problem considers the full dynamic model of the robot down to its single rotor thrusts and minimizes the final pose and velocity errors while avoiding collisions and maximizing the visibility of the powerline during the maneuver. The generated maneuvers consider both the perching and the posterior recovery trajectories. The framework adopts costs and constraints defined by efficient mathematical representations of powerlines, enabling online onboard execution in resource-constrained hardware. The method is validated on-board an agile quadrotor conducting powerline inspection and various perching maneuvers with final pitch values of up to 180 degrees. The developed code is available online at: https://github.com/grvcPerception/pa_powerline_perching
Learning from Randomly Initialized Neural Network Features
Venue: arXiv cs.LG 2022
Year: 2022

We present the surprising result that randomly initialized neural networks are good feature extractors in expectation. These random features correspond to finite-sample realizations of what we call Neural Network Prior Kernel (NNPK), which is inherently infinite-dimensional. We conduct ablations across multiple architectures of varying sizes as well as initializations and activation functions. Our analysis suggests that certain structures that manifest in a trained model are already present at initialization. Therefore, NNPK may provide further insight into why neural networks are so effective in learning such structures.
Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations
Venue: arXiv cs.LG 2022
Year: 2022

Real world recommendation systems influence a constantly growing set of domains. With deep networks, that now drive such systems, recommendations have been more relevant to the user's interests and tasks. However, they may not always be reproducible even if produced by the same system for the same user, recommendation sequence, request, or query. This problem received almost no attention in academic publications, but is, in fact, very realistic and critical in real production systems. We consider reproducibility of real large scale deep models, whose predictions determine such recommendations. We demonstrate that the celebrated Rectified Linear Unit (ReLU) activation, used in deep models, can be a major contributor to irreproducibility. We propose the use of smooth activations to improve recommendation reproducibility. We describe a novel family of smooth activations; Smooth ReLU (SmeLU), designed to improve reproducibility with mathematical simplicity, with potentially cheaper implementation. SmeLU is a member of a wider family of smooth activations. While other techniques that improve reproducibility in real systems usually come at accuracy costs, smooth activations not only improve reproducibility, but can even give accuracy gains. We report metrics from real systems in which we were able to productionalize SmeLU with substantial reproducibility gains and better accuracy-reproducibility trade-offs. These include click-through-rate (CTR) prediction systems, content, and application recommendation systems.
Adaptive Graph Convolutional Networks for Weakly Supervised Anomaly Detection in Videos
Venue: arXiv cs.CV 2022
Year: 2022

For weakly supervised anomaly detection, most existing work is limited to the problem of inadequate video representation due to the inability of modeling long-term contextual information. To solve this, we propose a novel weakly supervised adaptive graph convolutional network (WAGCN) to model the complex contextual relationship among video segments. By which, we fully consider the influence of other video segments on the current one when generating the anomaly probability score for each segment. Firstly, we combine the temporal consistency as well as feature similarity of video segments to construct a global graph, which makes full use of the association information among spatial-temporal features of anomalous events in videos. Secondly, we propose a graph learning layer in order to break the limitation of setting topology manually, which can extract graph adjacency matrix based on data adaptively and effectively. Extensive experiments on two public datasets (i.e., UCF-Crime dataset and ShanghaiTech dataset) demonstrate the effectiveness of our approach which achieves state-of-the-art performance.
A formal algebraic approach for the quantitative modeling of connectors in architectures
Venue: arXiv cs.LO 2022
Year: 2022

In this paper we propose an algebraic formalization of connectors in the quantitative setting, in order to address their non-functional features in architectures of component-based systems. We firstly present a weighted Algebra of Interactions over a set of ports and a commutative and idempotent semiring, which is proved sufficient for modeling well-known coordination schemes in the weighted setup. In turn, we study a weighted Algebra of Connectors over a set of ports and a commutative and idempotent semiring, which extends the weighted Algebra of Interactions with types that encode Rendezvous and Broadcast synchronization. We show the expressiveness of the algebra by modeling the weighted connectors of several coordination schemes. Moreover, we derive two subalgebras, namely the weighted Algebra of Synchrons and the weighted Algebra of Triggers, and study their properties. Finally, we introduce a concept of congruence relation for connectors in the weighted setup and we provide conditions for proving such a congruence.
Secure-by-Construction Synthesis of Cyber-Physical Systems
Venue: arXiv cs.CR 2022
Year: 2022

Correct-by-construction synthesis is a cornerstone of the confluence of formal methods and control theory towards designing safety-critical systems. Instead of following the time-tested, albeit laborious (re)design-verify-validate loop, correct-by-construction methodology advocates the use of continual refinements of formal requirements -- connected by chains of formal proofs -- to build a system that assures the correctness by design. A remarkable progress has been made in scaling the scope of applicability of correct-by-construction synthesis -- with a focus on cyber-physical systems that tie discrete-event control with continuous environment -- to enlarge control systems by combining symbolic approaches with principled state-space reduction techniques. Unfortunately, in the security-critical control systems, the security properties are verified ex post facto the design process in a way that undermines the correct-by-construction paradigm. We posit that, to truly realize the dream of correct-by-construction synthesis for security-critical systems, security considerations must take center-stage with the safety considerations. Moreover, catalyzed by the recent progress on the opacity sub-classes of security properties and the notion of hyperproperties capable of combining security with safety properties, we believe that the time is ripe for the research community to holistically target the challenge of secure-by-construction synthesis. This paper details our vision by highlighting the recent progress and open challenges that may serve as bricks for providing a solid foundation for secure-by-construction synthesis of cyber-physical systems.
Overhead Management in Multi-Core Environment
Venue: arXiv cs.DC 2022
Year: 2022

In multi-core systems, various factors like inter-process communication, dependency, resource sharing and scheduling, level of parallelism, synchronization, number of available cores etc. influence the extent of possible High Performance Computing parallelization. These parameters if not managed to the root level, later surface as overheads during execution. This paper emphasizes on these parameters of parallelism, their overheads of parallelization and its effective management for optimal parallel execution under any domain. As a whole, we focus on the Dense Linear Algebra (DLA) domain and specifically on Matrix Multiplication and sorting domains. These domains are chosen as they find application in various sectors of scientific and mathematical applications. The comparative analysis of results obtained clarifies the trade-off between serial and parallel execution of DLA problems the surfacing overheads and their possible and effective management.
A Subpolynomial Approximation Algorithm for Graph Crossing Number in Low-Degree Graphs
Venue: arXiv cs.DS 2022
Year: 2022

We consider the classical Minimum Crossing Number problem: given an $n$-vertex graph $G$, compute a drawing of $G$ in the plane, while minimizing the number of crossings between the images of its edges. This is a fundamental and extensively studied problem, whose approximability status is widely open. In all currently known approximation algorithms, the approximation factor depends polynomially on $Δ$ -- the maximum vertex degree in $G$. The best current approximation algorithm achieves an $O(n^{1/2-\varepsilon}\cdot \text{poly}(Δ\cdot\log n))$-approximation, for a small fixed constant $ε$, while the best negative result is APX-hardness, leaving a large gap in our understanding of this basic problem. In this paper we design a randomized $O\left(2^{O((\log n)^{7/8}\log\log n)}\cdot\text{poly}(Δ)\right )$-approximation algorithm for Minimum Crossing Number. This is the first approximation algorithm for the problem that achieves a subpolynomial in $n$ approximation factor (albeit only in graphs whose maximum vertex degree is subpolynomial in $n$). In order to achieve this approximation factor, we design a new algorithm for a closely related problem called Crossing Number with Rotation System, in which, for every vertex $v\in V(G)$, the circular ordering, in which the images of the edges incident to $v$ must enter the image of $v$ in the drawing is fixed as part of the input. Combining this result with the recent reduction of [Chuzhoy, Mahabadi, Tan '20] immediately yields the improved approximation algorithm for Minimum Crossing Number. We introduce several new technical tools, that we hope will be helpful in obtaining better algorithms for the problem in the future.
Problems hard for treewidth but easy for stable gonality
Venue: arXiv cs.DS 2022
Year: 2022

We show that some natural problems that are XNLP-hard (which implies W[t]-hardness for all t) when parameterized by pathwidth or treewidth, become FPT when parameterized by stable gonality, a novel graph parameter based on optimal maps from graphs to trees. The problems we consider are classical flow and orientation problems, such as Undirected Flow with Lower Bounds (which is strongly NP-complete, as shown by Itai), Minimum Maximum Outdegree (for which W[1]-hardness for treewidth was proven by Szeider), and capacitated optimization problems such as Capacitated (Red-Blue) Dominating Set (for which W[1]-hardness was proven by Dom, Lokshtanov, Saurabh and Villanger). Our hardness proofs (that beat existing results) use reduction to a recent XNLP-complete problem (Accepting Non-deterministic Checking Counter Machine). The new easy parameterized algorithms use a novel notion of weighted tree partition with an associated parameter that we call treebreadth, inspired by Seese's notion of tree-partite graphs, as well as techniques from dynamical programming and integer linear programming.
What Do They Capture? -- A Structural Analysis of Pre-Trained Language Models for Source Code
Venue: arXiv cs.SE 2022
Year: 2022

Recently, many pre-trained language models for source code have been proposed to model the context of code and serve as a basis for downstream code intelligence tasks such as code completion, code search, and code summarization. These models leverage masked pre-training and Transformer and have achieved promising results. However, currently there is still little progress regarding interpretability of existing pre-trained code models. It is not clear why these models work and what feature correlations they can capture. In this paper, we conduct a thorough structural analysis aiming to provide an interpretation of pre-trained language models for source code (e.g., CodeBERT, and GraphCodeBERT) from three distinctive perspectives: (1) attention analysis, (2) probing on the word embedding, and (3) syntax tree induction. Through comprehensive analysis, this paper reveals several insightful findings that may inspire future studies: (1) Attention aligns strongly with the syntax structure of code. (2) Pre-training language models of code can preserve the syntax structure of code in the intermediate representations of each Transformer layer. (3) The pre-trained models of code have the ability of inducing syntax trees of code. Theses findings suggest that it may be helpful to incorporate the syntax structure of code into the process of pre-training for better code representations.
Enhancing expressivity of checked corecursive streams (extended version)
Venue: arXiv cs.PL 2022
Year: 2022

We propose a novel approach to stream definition and manipulation. Our solution is based on two key ideas. Regular corecursion, which avoids non termination by detecting cyclic calls, is enhanced, by allowing in equations defining streams other operators besides the stream constructor. In this way, some non-regular streams are definable. Furthermore, execution includes a runtime check to ensure that the stream generated by a function call is well-defined, in the sense that access to an arbitrary index always succeeds. We extend the technique beyond the simple stream operators considered in previous work, notably by adding an interleaving combinator which has a non-trivial recursion scheme.
FOLD-RM: A Scalable, Efficient, and Explainable Inductive Learning Algorithm for Multi-Category Classification of Mixed Data
Venue: arXiv cs.LG 2022
Year: 2022

FOLD-RM is an automated inductive learning algorithm for learning default rules for mixed (numerical and categorical) data. It generates an (explainable) answer set programming (ASP) rule set for multi-category classification tasks while maintaining efficiency and scalability. The FOLD-RM algorithm is competitive in performance with the widely-used, state-of-the-art algorithms such as XGBoost and multi-layer perceptrons (MLPs), however, unlike these algorithms, the FOLD-RM algorithm produces an explainable model. FOLD-RM outperforms XGBoost on some datasets, particularly large ones. FOLD-RM also provides human-friendly explanations for predictions.
A Generic Self-Supervised Framework of Learning Invariant Discriminative Features
Venue: arXiv cs.LG 2022
Year: 2022

Self-supervised learning (SSL) has become a popular method for generating invariant representations without the need for human annotations. Nonetheless, the desired invariant representation is achieved by utilising prior online transformation functions on the input data. As a result, each SSL framework is customised for a particular data type, e.g., visual data, and further modifications are required if it is used for other dataset types. On the other hand, autoencoder (AE), which is a generic and widely applicable framework, mainly focuses on dimension reduction and is not suited for learning invariant representation. This paper proposes a generic SSL framework based on a constrained self-labelling assignment process that prevents degenerate solutions. Specifically, the prior transformation functions are replaced with a self-transformation mechanism, derived through an unsupervised training process of adversarial training, for imposing invariant representations. Via the self-transformation mechanism, pairs of augmented instances can be generated from the same input data. Finally, a training objective based on contrastive learning is designed by leveraging both the self-labelling assignment and the self-transformation mechanism. Despite the fact that the self-transformation process is very generic, the proposed training strategy outperforms a majority of state-of-the-art representation learning methods based on AE structures. To validate the performance of our method, we conduct experiments on four types of data, namely visual, audio, text, and mass spectrometry data, and compare them in terms of four quantitative metrics. Our comparison results indicate that the proposed method demonstrate robustness and successfully identify patterns within the datasets.
Slicing Aided Hyper Inference and Fine-tuning for Small Object Detection
Venue: arXiv cs.CV 2022
Year: 2022

Detection of small objects and objects far away in the scene is a major challenge in surveillance applications. Such objects are represented by small number of pixels in the image and lack sufficient details, making them difficult to detect using conventional detectors. In this work, an open-source framework called Slicing Aided Hyper Inference (SAHI) is proposed that provides a generic slicing aided inference and fine-tuning pipeline for small object detection. The proposed technique is generic in the sense that it can be applied on top of any available object detector without any fine-tuning. Experimental evaluations, using object detection baselines on the Visdrone and xView aerial object detection datasets show that the proposed inference method can increase object detection AP by 6.8%, 5.1% and 5.3% for FCOS, VFNet and TOOD detectors, respectively. Moreover, the detection accuracy can be further increased with a slicing aided fine-tuning, resulting in a cumulative increase of 12.7%, 13.4% and 14.5% AP in the same order. Proposed technique has been integrated with Detectron2, MMDetection and YOLOv5 models and it is publicly available at https://github.com/obss/sahi.git .
LiDiTE: a Full-Fledged and Featherweight Digital Twin Framework
Venue: arXiv cs.DC 2022
Year: 2022

The rising of the Cyber-Physical System (CPS) and the Industry 4.0 paradigms demands the design and the implementation of Digital Twin Frameworks (DTFs) that may support the quick build of reliable Digital Twins (DTs) for experimental and testing purposes. Most of the current DTF proposals allow generating DTs at a good pace but affect generality, scalability, portability, and completeness. As a consequence, current DTF are mostly domain-specific and hardly span several application domains (e.g., from simple IoT deployments to the modeling of complex critical infrastructures). Furthermore, the generated DTs often requires a high amount of computational resource to run. In this paper, we present LiDiTE, a novel DTF that overcomes the previous limitations by, on the one hand, supporting the building of general-purpose DTs at a fine-grained level, but, on the other hand, with a reduced resource footprint w.r.t. the current state of the art. We show the characteristics of the LiDiTE by building the DT of a complex and real critical infrastructure (i.e., the Smart Poligeneration Microgrid of the Savona Campus) and evaluating its resource consumption. The source code of LiDiTE, as well as the experimental dataset, is publicly available.
Building Inspection Toolkit: Unified Evaluation and Strong Baselines for Damage Recognition
Venue: arXiv cs.CV 2022
Year: 2022

In recent years, several companies and researchers have started to tackle the problem of damage recognition within the scope of automated inspection of built structures. While companies are neither willing to publish associated data nor models, researchers are facing the problem of data shortage on one hand and inconsistent dataset splitting with the absence of consistent metrics on the other hand. This leads to incomparable results. Therefore, we introduce the building inspection toolkit -- bikit -- which acts as a simple to use data hub containing relevant open-source datasets in the field of damage recognition. The datasets are enriched with evaluation splits and predefined metrics, suiting the specific task and their data distribution. For the sake of compatibility and to motivate researchers in this domain, we also provide a leaderboard and the possibility to share model weights with the community. As starting point we provide strong baselines for multi-target classification tasks utilizing extensive hyperparameter search using three transfer learning approaches for state-of-the-art algorithms. The toolkit and the leaderboard are available online.
Strategy Discovery and Mixture in Lifelong Learning from Heterogeneous Demonstration
Venue: arXiv cs.LG 2022
Year: 2022

Learning from Demonstration (LfD) approaches empower end-users to teach robots novel tasks via demonstrations of the desired behaviors, democratizing access to robotics. A key challenge in LfD research is that users tend to provide heterogeneous demonstrations for the same task due to various strategies and preferences. Therefore, it is essential to develop LfD algorithms that ensure \textit{flexibility} (the robot adapts to personalized strategies), \textit{efficiency} (the robot achieves sample-efficient adaptation), and \textit{scalability} (robot reuses a concise set of strategies to represent a large amount of behaviors). In this paper, we propose a novel algorithm, Dynamic Multi-Strategy Reward Distillation (DMSRD), which distills common knowledge between heterogeneous demonstrations, leverages learned strategies to construct mixture policies, and continues to improve by learning from all available data. Our personalized, federated, and lifelong LfD architecture surpasses benchmarks in two continuous control problems with an average 77\% improvement in policy returns and 42\% improvement in log likelihood, alongside stronger task reward correlation and more precise strategy rewards.