OpenRaiser/Intern-Atlas huggingface dataset:
Knowledge Prompting for Few-shot Action Recognition
Venue: arXiv cs.CV 2022
Year: 2022

Few-shot action recognition in videos is challenging for its lack of supervision and difficulty in generalizing to unseen actions. To address this task, we propose a simple yet effective method, called knowledge prompting, which leverages commonsense knowledge of actions from external resources to prompt a powerful pre-trained vision-language model for few-shot classification. We first collect large-scale language descriptions of actions, defined as text proposals, to build an action knowledge base. The collection of text proposals is done by filling in handcraft sentence templates with external action-related corpus or by extracting action-related phrases from captions of Web instruction videos.Then we feed these text proposals into the pre-trained vision-language model along with video frames to generate matching scores of the proposals to each frame, and the scores can be treated as action semantics with strong generalization. Finally, we design a lightweight temporal modeling network to capture the temporal evolution of action semantics for classification.Extensive experiments on six benchmark datasets demonstrate that our method generally achieves the state-of-the-art performance while reducing the training overhead to 0.001 of existing methods.
Efficient Sampling Algorithms for Approximate Motif Counting in Temporal Graph Streams
Venue: arXiv cs.SI 2022
Year: 2022

A great variety of complex systems, from user interactions in communication networks to transactions in financial markets, can be modeled as temporal graphs consisting of a set of vertices and a series of timestamped and directed edges. Temporal motifs are generalized from subgraph patterns in static graphs which consider edge orderings and durations in addition to topologies. Counting the number of occurrences of temporal motifs is a fundamental problem for temporal network analysis. However, existing methods either cannot support temporal motifs or suffer from performance issues. Moreover, they cannot work in the streaming model where edges are observed incrementally over time. In this paper, we focus on approximate temporal motif counting via random sampling. We first propose two sampling algorithms for temporal motif counting in the offline setting. The first is an edge sampling (ES) algorithm for estimating the number of instances of any temporal motif. The second is an improved edge-wedge sampling (EWS) algorithm that hybridizes edge sampling with wedge sampling for counting temporal motifs with $3$ vertices and $3$ edges. Furthermore, we propose two algorithms to count temporal motifs incrementally in temporal graph streams by extending the ES and EWS algorithms referred to as SES and SEWS. We provide comprehensive analyses of the theoretical bounds and complexities of our proposed algorithms. Finally, we perform extensive experimental evaluations of our proposed algorithms on several real-world temporal graphs. The results show that ES and EWS have higher efficiency, better accuracy, and greater scalability than state-of-the-art sampling methods for temporal motif counting in the offline setting. Moreover, SES and SEWS achieve up to three orders of magnitude speedups over ES and EWS while having comparable estimation errors for temporal motif counting in the streaming setting.
HaRiM$^+$: Evaluating Summary Quality with Hallucination Risk
Venue: arXiv cs.CL 2022
Year: 2022

One of the challenges of developing a summarization model arises from the difficulty in measuring the factual inconsistency of the generated text. In this study, we reinterpret the decoder overconfidence-regularizing objective suggested in (Miao et al., 2021) as a hallucination risk measurement to better estimate the quality of generated summaries. We propose a reference-free metric, HaRiM+, which only requires an off-the-shelf summarization model to compute the hallucination risk based on token likelihoods. Deploying it requires no additional training of models or ad-hoc modules, which usually need alignment to human judgments. For summary-quality estimation, HaRiM+ records state-of-the-art correlation to human judgment on three summary-quality annotation sets: FRANK, QAGS, and SummEval. We hope that our work, which merits the use of summarization models, facilitates the progress of both automated evaluation and generation of summary.
The Monocular Depth Estimation Challenge
Venue: arXiv cs.CV 2022
Year: 2022

This paper summarizes the results of the first Monocular Depth Estimation Challenge (MDEC) organized at WACV2023. This challenge evaluated the progress of self-supervised monocular depth estimation on the challenging SYNS-Patches dataset. The challenge was organized on CodaLab and received submissions from 4 valid teams. Participants were provided a devkit containing updated reference implementations for 16 State-of-the-Art algorithms and 4 novel techniques. The threshold for acceptance for novel techniques was to outperform every one of the 16 SotA baselines. All participants outperformed the baseline in traditional metrics such as MAE or AbsRel. However, pointcloud reconstruction metrics were challenging to improve upon. We found predictions were characterized by interpolation artefacts at object boundaries and errors in relative object positioning. We hope this challenge is a valuable contribution to the community and encourage authors to participate in future editions.
Analysis of the DoIP Protocol for Security Vulnerabilities
Venue: arXiv cs.CR 2022
Year: 2022

DoIP, which is defined in ISO 13400, is a transport protocol stack for diagnostic data. Diagnostic data is a potential attack vector at vehicles, so secure transmission must be guaranteed to protect sensitive data and the vehicle. Previous work analyzed a draft version and earlier versions of the DoIP protocol without Transport Layer Security (TLS). No formal analysis exists for the DoIP protocol. The goal of this work is to investigate the DoIP protocol for design flaws that may lead to security vulnerabilities and possible attacks to exploit them. For this purpose, we deductively analyze the DoIP protocol in a first step and subsequently confirm our conclusions formally. For the formal analysis, we use the prover Tamarin. Based on the results, we propose countermeasures to improve the DoIP's security.We showthat the DoIP protocol cannot be considered secure mainly because the security mechanisms TLS and client authentication in the DoIP protocol are not mandatory. We propose measures to mitigate the vulnerabilities thatwe confirm to remain after activating TLS. These require only a minor redesign of the protocol.
A Combined Approach of Process Mining and Rule-based AI for Study Planning and Monitoring in Higher Education
Venue: arXiv cs.AI 2022
Year: 2022

This paper presents an approach of using methods of process mining and rule-based artificial intelligence to analyze and understand study paths of students based on campus management system data and study program models. Process mining techniques are used to characterize successful study paths, as well as to detect and visualize deviations from expected plans. These insights are combined with recommendations and requirements of the corresponding study programs extracted from examination regulations. Here, event calculus and answer set programming are used to provide models of the study programs which support planning and conformance checking while providing feedback on possible study plan violations. In its combination, process mining and rule-based artificial intelligence are used to support study planning and monitoring by deriving rules and recommendations for guiding students to more suitable study paths with higher success rates. Two applications will be implemented, one for students and one for study program designers.
Virtual Reality in University Teaching: Experiences from a Computer Science Seminar
Venue: arXiv cs.HC 2022
Year: 2022

Due to the corona pandemic, numerous courses were held using digital solutions in order to be able to continue teaching. Conventional collaboration tools (Zoom, Big Blue Button, etc.) were used in particular to digitally map a synchronous session for teaching and learning purposes. While these conventional collaboration tools offer a solid basis for communication between learners and teachers, aspects such as presence or a realistic type of interaction are neglected. In this work, we report on the experiences from a computer science seminar where virtual reality (VR) technology was used as an alternative solution for teaching and group work. The benefits of VR compared to conventional collaboration tools were examined using questionnaires and interviews with the participants. On the one hand, the results show the high potential of VR to increase the clarity and experienceability of learning content and to promote cooperation through social presence. On the other hand, the use of VR brings with it some technical and organizational difficulties that should be taken into account in the didactic implementation.
Sustainable Wireless Services with UAV Swarms Tailored to Renewable Energy Sources
Venue: arXiv cs.NI 2022
Year: 2022

Unmanned Aerial Vehicle (UAV) swarms are often required in off-grid scenarios, such as disaster-struck, war-torn or rural areas, where the UAVs have no access to the power grid and instead rely on renewable energy. Considering a main battery fed from two renewable sources, wind and solar, we scale such a system based on the financial budget, environmental characteristics, and seasonal variations. Interestingly, the source of energy is correlated with the energy expenditure of the UAVs, since strong winds cause UAV hovering to become increasingly energy-hungry. The aim is to maximize the cost efficiency of coverage at a particular location, which is a combinatorial optimization problem for dimensioning of the multivariate energy generation system under non-convex criteria. We have devised a customized algorithm by lowering the processing complexity and reducing the solution space through sampling. Evaluation is done with condensed real-world data on wind, solar energy, and traffic load per unit area, driven by vendor-provided prices. The implementation was tested in four locations, with varying wind or solar intensity. The best results were achieved in locations with mild wind presence and strong solar irradiation, while locations with strong winds and low solar intensity require higher Capital Expenditure (CAPEX) allocation.
The Security Protocol Verifier ProVerif and its Horn Clause Resolution Algorithm
Venue: arXiv cs.CR 2022
Year: 2022

ProVerif is a widely used security protocol verifier. Internally, ProVerif uses an abstract representation of the protocol by Horn clauses and a resolution algorithm on these clauses, in order to prove security properties of the protocol or to find attacks. In this paper, we present an overview of ProVerif and discuss some specificities of its resolution algorithm, related to the particular application domain and the particular clauses that ProVerif generates. This paper is a short summary that gives pointers to publications on ProVerif in which the reader will find more details.
Generalized Random Gilbert-Varshamov Codes: Typical Error Exponent and Concentration Properties
Venue: arXiv cs.IT 2022
Year: 2022

We find the exact typical error exponent of constant composition generalized random Gilbert-Varshamov (RGV) codes over DMCs channels with generalized likelihood decoding. We show that the typical error exponent of the RGV ensemble is equal to the expurgated error exponent, provided that the RGV codebook parameters are chosen appropriately. We also prove that the random coding exponent converges in probability to the typical error exponent, and the corresponding non-asymptotic concentration rates are derived. Our results show that the decay rate of the lower tail is exponential while that of the upper tail is double exponential above the expurgated error exponent. The explicit dependence of the decay rates on the RGV distance functions is characterized.
FE-Fusion-VPR: Attention-based Multi-Scale Network Architecture for Visual Place Recognition by Fusing Frames and Events
Venue: arXiv cs.CV 2022
Year: 2022

Traditional visual place recognition (VPR), usually using standard cameras, is easy to fail due to glare or high-speed motion. By contrast, event cameras have the advantages of low latency, high temporal resolution, and high dynamic range, which can deal with the above issues. Nevertheless, event cameras are prone to failure in weakly textured or motionless scenes, while standard cameras can still provide appearance information in this case. Thus, exploiting the complementarity of standard cameras and event cameras can effectively improve the performance of VPR algorithms. In the paper, we propose FE-Fusion-VPR, an attention-based multi-scale network architecture for VPR by fusing frames and events. First, the intensity frame and event volume are fed into the two-stream feature extraction network for shallow feature fusion. Next, the three-scale features are obtained through the multi-scale fusion network and aggregated into three sub-descriptors using the VLAD layer. Finally, the weight of each sub-descriptor is learned through the descriptor re-weighting network to obtain the final refined descriptor. Experimental results show that on the Brisbane-Event-VPR and DDD20 datasets, the Recall@1 of our FE-Fusion-VPR is 29.26% and 33.59% higher than Event-VPR and Ensemble-EventVPR, and is 7.00% and 14.15% higher than MultiRes-NetVLAD and NetVLAD. To our knowledge, this is the first end-to-end network that goes beyond the existing event-based and frame-based SOTA methods to fuse frame and events directly for VPR.
A Cross-Residual Learning for Image Recognition
Venue: arXiv cs.CV 2022
Year: 2022

ResNets and its variants play an important role in various fields of image recognition. This paper gives another variant of ResNets, a kind of cross-residual learning networks called C-ResNets, which has less computation and parameters than ResNets. C-ResNets increases the information interaction between modules by densifying jumpers and enriches the role of jumpers. In addition, some meticulous designs on jumpers and channels counts can further reduce the resource consumption of C-ResNets and increase its classification performance. In order to test the effectiveness of C-ResNets, we use the same hyperparameter settings as fine-tuned ResNets in the experiments. We test our C-ResNets on datasets MNIST, FashionMnist, CIFAR-10, CIFAR-100, CALTECH-101 and SVHN. Compared with fine-tuned ResNets, C-ResNets not only maintains the classification performance, but also enormously reduces the amount of calculations and parameters which greatly save the utilization rate of GPUs and GPU memory resources. Therefore, our C-ResNets is competitive and viable alternatives to ResNets in various scenarios. Code is available at https://github.com/liangjunhello/C-ResNet
Method for Determining the Similarity of Text Documents for the Kazakh language, Taking Into Account Synonyms: Extension to TF-IDF
Venue: arXiv cs.IR 2022
Year: 2022

The task of determining the similarity of text documents has received considerable attention in many areas such as Information Retrieval, Text Mining, Natural Language Processing (NLP) and Computational Linguistics. Transferring data to numeric vectors is a complex task where algorithms such as tokenization, stopword filtering, stemming, and weighting of terms are used. The term frequency - inverse document frequency (TF-IDF) is the most widely used term weighting method to facilitate the search for relevant documents. To improve the weighting of terms, a large number of TF-IDF extensions are made. In this paper, another extension of the TF-IDF method is proposed where synonyms are taken into account. The effectiveness of the method is confirmed by experiments on functions such as Cosine, Dice and Jaccard to measure the similarity of text documents for the Kazakh language.
An Emotion-Aware Multi-Task Approach to Fake News and Rumour Detection using Transfer Learning
Venue: arXiv cs.CL 2022
Year: 2022

Social networking sites, blogs, and online articles are instant sources of news for internet users globally. However, in the absence of strict regulations mandating the genuineness of every text on social media, it is probable that some of these texts are fake news or rumours. Their deceptive nature and ability to propagate instantly can have an adverse effect on society. This necessitates the need for more effective detection of fake news and rumours on the web. In this work, we annotate four fake news detection and rumour detection datasets with their emotion class labels using transfer learning. We show the correlation between the legitimacy of a text with its intrinsic emotion for fake news and rumour detection, and prove that even within the same emotion class, fake and real news are often represented differently, which can be used for improved feature extraction. Based on this, we propose a multi-task framework for fake news and rumour detection, predicting both the emotion and legitimacy of the text. We train a variety of deep learning models in single-task and multi-task settings for a more comprehensive comparison. We further analyze the performance of our multi-task approach for fake news detection in cross-domain settings to verify its efficacy for better generalization across datasets, and to verify that emotions act as a domain-independent feature. Experimental results verify that our multi-task models consistently outperform their single-task counterparts in terms of accuracy, precision, recall, and F1 score, both for in-domain and cross-domain settings. We also qualitatively analyze the difference in performance in single-task and multi-task learning models.
PiRL: Participant-Invariant Representation Learning for Healthcare
Venue: arXiv cs.LG 2022
Year: 2022

Due to individual heterogeneity, performance gaps are observed between generic (one-size-fits-all) models and person-specific models in data-driven health applications. However, in real-world applications, generic models are usually more favorable due to new-user-adaptation issues and system complexities, etc. To improve the performance of the generic model, we propose a representation learning framework that learns participant-invariant representations, named PiRL. The proposed framework utilizes maximum mean discrepancy (MMD) loss and domain-adversarial training to encourage the model to learn participant-invariant representations. Further, a triplet loss, which constrains the model for inter-class alignment of the representations, is utilized to optimize the learned representations for downstream health applications. We evaluated our frameworks on two public datasets related to physical and mental health, for detecting sleep apnea and stress, respectively. As preliminary results, we found the proposed approach shows around a 5% increase in accuracy compared to the baseline.
Fault-Tolerant Dispersion of Mobile Robots
Venue: arXiv cs.DC 2022
Year: 2022

We consider the mobile robot dispersion problem in the presence of faulty robots (crash-fault). Mobile robot dispersion consists of $k\leq n$ robots in an $n$-node anonymous graph. The goal is to ensure that regardless of the initial placement of the robots over the nodes, the final configuration consists of having at most one robot at each node. In a crash-fault setting, up to $f \leq k$ robots may fail by crashing arbitrarily and subsequently lose all the information stored at the robots, rendering them unable to communicate. In this paper, we solve the dispersion problem in a crash-fault setting by considering two different initial configurations: i) the rooted configuration, and ii) the arbitrary configuration. In the rooted case, all robots are placed together at a single node at the start. The arbitrary configuration is a general configuration (a.k.a. arbitrary configuration in the literature) where the robots are placed in some $l<k$ clusters arbitrarily across the graph. For the first case, we develop an algorithm solving dispersion in the presence of faulty robots in $O(k^2)$ rounds, which improves over the previous $O(f\cdot\text{min}(m,kΔ))$-round result by \cite{PS021}. For the arbitrary configuration, we present an algorithm solving dispersion in $O((f+l)\cdot\text{min}(m, k Δ, k^2))$ rounds, when the number of edges $m$ and the maximum degree $Δ$ of the graph is known to the robots.
A Deep Reinforcement Learning Approach to Rare Event Estimation
Venue: arXiv cs.LG 2022
Year: 2022

An important step in the design of autonomous systems is to evaluate the probability that a failure will occur. In safety-critical domains, the failure probability is extremely small so that the evaluation of a policy through Monte Carlo sampling is inefficient. Adaptive importance sampling approaches have been developed for rare event estimation but do not scale well to sequential systems with long horizons. In this work, we develop two adaptive importance sampling algorithms that can efficiently estimate the probability of rare events for sequential decision making systems. The basis for these algorithms is the minimization of the Kullback-Leibler divergence between a state-dependent proposal distribution and a target distribution over trajectories, but the resulting algorithms resemble policy gradient and value-based reinforcement learning. We apply multiple importance sampling to reduce the variance of our estimate and to address the issue of multi-modality in the optimal proposal distribution. We demonstrate our approach on a control task with both continuous and discrete actions spaces and show accuracy improvements over several baselines.
NLP meets psychotherapy: Using predicted client emotions and self-reported client emotions to measure emotional coherence
Venue: arXiv cs.CL 2022
Year: 2022

Emotions are experienced and expressed through various response systems. Coherence between emotional experience and emotional expression is considered important to clients' well being. To date, emotional coherence (EC) has been studied at a single time point using lab-based tasks with relatively small datasets. No study has examined EC between the subjective experience of emotions and emotion expression in therapy or whether this coherence is associated with clients' well being. Natural language Processing (NLP) approaches have been applied to identify emotions from psychotherapy dialogue, which can be implemented to study emotional processes on a larger scale. However, these methods have yet to be used to study coherence between emotional experience and emotional expression over the course of therapy and whether it relates to clients' well-being. This work presents an end-to-end approach where we use emotion predictions from our transformer based emotion recognition model to study emotional coherence and its diagnostic potential in psychotherapy research. We first employ our transformer based approach on a Hebrew psychotherapy dataset to automatically label clients' emotions at utterance level in psychotherapy dialogues. We subsequently investigate the emotional coherence between clients' self-reported emotional states and our model-based emotion predictions. We also examine the association between emotional coherence and clients' well being. Our findings indicate a significant correlation between clients' self-reported emotions and positive and negative emotions expressed verbally during psychotherapy sessions. Coherence in positive emotions was also highly correlated with clients well-being. These results illustrate how NLP can be applied to identify important emotional processes in psychotherapy to improve diagnosis and treatment for clients suffering from mental-health problems.