OpenRaiser/Intern-Atlas huggingface dataset:
TEDB System Description to a Shared Task on Euphemism Detection 2022
Venue: arXiv cs.CL 2023
Year: 2023

In this report, we describe our Transformers for euphemism detection baseline (TEDB) submissions to a shared task on euphemism detection 2022. We cast the task of predicting euphemism as text classification. We considered Transformer-based models which are the current state-of-the-art methods for text classification. We explored different training schemes, pretrained models, and model architectures. Our best result of 0.816 F1-score (0.818 precision and 0.814 recall) consists of a euphemism-detection-finetuned TweetEval/TimeLMs-pretrained RoBERTa model as a feature extractor frontend with a KimCNN classifier backend trained end-to-end using a cosine annealing scheduler. We observed pretrained models on sentiment analysis and offensiveness detection to correlate with more F1-score while pretraining on other tasks, such as sarcasm detection, produces less F1-scores. Also, putting more word vector channels does not improve the performance in our experiments.
Does Spending More Always Ensure Higher Cooperation? An Analysis of Institutional Incentives on Heterogeneous Networks
Venue: arXiv cs.MA 2023
Year: 2023

Humans have developed considerable machinery used at scale to create policies and to distribute incentives, yet we are forever seeking ways in which to improve upon these, our institutions. Especially when funding is limited, it is imperative to optimise spending without sacrificing positive outcomes, a challenge which has often been approached within several areas of social, life and engineering sciences. These studies often neglect the availability of information, cost restraints, or the underlying complex network structures, which define real-world populations. Here, we have extended these models, including the aforementioned concerns, but also tested the robustness of their findings to stochastic social learning paradigms. Akin to real-world decisions on how best to distribute endowments, we study several incentive schemes, which consider information about the overall population, local neighbourhoods, or the level of influence which a cooperative node has in the network, selectively rewarding cooperative behaviour if certain criteria are met. Following a transition towards a more realistic network setting and stochastic behavioural update rule, we found that carelessly promoting cooperators can often lead to their downfall in socially diverse settings. These emergent cyclic patterns not only damage cooperation, but also decimate the budgets of external investors. Our findings highlight the complexity of designing effective and cogent investment policies in socially diverse populations.
$Ae^2I$: A Double Autoencoder for Imputation of Missing Values
Venue: arXiv cs.LG 2023
Year: 2023

The most common strategy of imputing missing values in a table is to study either the column-column relationship or the row-row relationship of the data table, then use the relationship to impute the missing values based on the non-missing values from other columns of the same row, or from the other rows of the same column. This paper introduces a double autoencoder for imputation ($Ae^2I$) that simultaneously and collaboratively uses both row-row relationship and column-column relationship to impute the missing values. Empirical tests on Movielens 1M dataset demonstrated that $Ae^2I$ outperforms the current state-of-the-art models for recommender systems by a significant margin.
Artificial intelligence as a gateway to scientific discovery: Uncovering features in retinal fundus images
Venue: arXiv cs.CV 2023
Year: 2023

Purpose: Convolutional neural networks can be trained to detect various conditions or patient traits based on retinal fundus photographs, some of which, such as the patient sex, are invisible to the expert human eye. Here we propose a methodology for explainable classification of fundus images to uncover the mechanism(s) by which CNNs successfully predict the labels. We used patient sex as a case study to validate our proposed methodology. Approach: First, we used a set of 4746 fundus images, including training, validation and test partitions, to fine-tune a pre-trained CNN on the sex classification task. Next, we utilized deep learning explainability tools to hypothesize possible ways sex differences in the retina manifest. We measured numerous retinal properties relevant to our hypotheses through image segmentation to identify those significantly different between males and females. To tackle the multiple comparisons problem, we shortlisted the parameters by testing them on a set of 100 fundus images distinct from the images used for fine-tuning. Finally, we used an additional 400 images, not included in any previous set, to reveal significant sex differences in the retina. Results: We observed that the peripapillary area is darker in males compared to females ($p=.023, d=.243$). We also observed that males have richer retinal vasculature networks by showing a higher number of branches ($p=.016, d=.272$) and nodes ($p=.014, d=.299$) and a larger total length of branches ($p=.045, d=.206$) in the vessel graph. Also, vessels cover a greater area in the superior temporal quadrant of the retina in males compared to females ($p=0.048, d=.194$). Conclusions: Our methodology reveals retinal features in fundus photographs that allow CNNs to predict traits currently unknown, but meaningful to experts.
Distribution Aligned Feature Clustering for Zero-Shot Sketch-Based Image Retrieval
Venue: arXiv cs.CV 2023
Year: 2023

Zero-Shot Sketch-Based Image Retrieval (ZS-SBIR) is a challenging cross-modal retrieval task. In prior arts, the retrieval is conducted by sorting the distance between the query sketch and each image in the gallery. However, the domain gap and the zero-shot setting make neural networks hard to generalize. This paper tackles the challenges from a new perspective: utilizing gallery image features. We propose a Cluster-then-Retrieve (ClusterRetri) method that performs clustering on the gallery images and uses the cluster centroids as proxies for retrieval. Furthermore, a distribution alignment loss is proposed to align the image and sketch features with a common Gaussian distribution, reducing the domain gap. Despite its simplicity, our proposed method outperforms the state-of-the-art methods by a large margin on popular datasets, e.g., up to 31% and 39% relative improvement of mAP@all on the Sketchy and TU-Berlin datasets.
Ember-IO: Effective Firmware Fuzzing with Model-Free Memory Mapped IO
Venue: arXiv cs.CR 2023
Year: 2023

Exponential growth in embedded systems is driving the research imperative to develop fuzzers to automate firmware testing to uncover software bugs and security vulnerabilities. But, employing fuzzing techniques in this context present a uniquely challenging proposition; a key problem is the need to deal with the diverse and large number of peripheral communications in an automated testing framework. Recent fuzzing approaches: i) employ re-hosting methods by executing code in an emulator because fuzzing on resource limited embedded systems is slow and unscalable; and ii) integrate models of hardware behaviour to overcome the challenges faced by the massive input-space to be explored created by peripheral devices and to generate inputs that are effective in aiding a fuzzer to make progress. Our efforts expounds upon program execution behaviours unique to firmware to address the resulting input-space search problem. The techniques we propose improve the fuzzer's ability to generate values likely to progress execution and avoids time consumed on mutating inputs that are functionally equivalent to other test cases. We demonstrate the methods are highly efficient and effective at overcoming the input-space search problem. Our emulation-based implementation, Ember-IO, when compared to the existing state-of-the-art fuzzing framework across 21 firmware binaries, demonstrates up to 255% improvement in blocks covered. Further Ember-IO discovered 6 new bugs in the real-world firmware, previously not identified by state-of-the-art fuzzing frameworks. Importantly, Ember-IO integrated with the state-of-the-art fuzzer, Fuzzware, demonstrates similar or improved coverage across all firmware binaries whilst reproducing 3 of the 6 new bugs discovered by Ember-IO.
Open RAN: Evolution of Architecture, Deployment Aspects, and Future Directions
Venue: arXiv cs.NI 2023
Year: 2023

The Open Radio Access Network (Open RAN) aims to enable disaggregated, virtualized, programmable, and data-driven intelligent network with open interfaces to support various real-time and non-real-time applications for different classes of users and multiple industry verticals in beyond 5G and 6G networks while providing interoperability among multi-vendor network functions and components. In this article, we first discuss the evolution of RAN and then the O-RAN Alliance standardization activities and objectives to provide a comprehensive overview of O-RAN from a standardization point of view. Then, we discuss the O-RAN security aspects, use cases, deployment aspects, and open source projects and related activities in other forums. Finally, we summarize the open issues, challenges, and future research directions to explore further for in-depth study and analysis.
Real-time, low latency virtual DBA hypervisor for SLA-compliant multi-service operations over shared Passive Optical Networks
Venue: arXiv cs.NI 2023
Year: 2023

We present a heuristic algorithm for a PON upstream scheduling hypervisor,supporting low latency services with strict service-level agreement. The algorithm achieves near-optimal performance while running in only 3.5 us, thus operating in real-time.
Utilization of Impedance Disparity Incurred from Switching Activities to Monitor and Characterize Firmware Activities
Venue: arXiv cs.CR 2023
Year: 2023

The massive trend toward embedded systems introduces new security threats to prevent. Malicious firmware makes it easier to launch cyberattacks against embedded systems. Systems infected with malicious firmware maintain the appearance of normal firmware operation but execute undesirable activities, which is usually a security risk. Traditionally, cybercriminals use malicious firmware to develop possible back-doors for future attacks. Due to the restricted resources of embedded systems, it is difficult to thwart these attacks using the majority of contemporary standard security protocols. In addition, monitoring the firmware operations using existing side channels from outside the processing unit, such as electromagnetic radiation, necessitates a complicated hardware configuration and in-depth technical understanding. In this paper, we propose a physical side channel that is formed by detecting the overall impedance changes induced by the firmware actions of a central processing unit. To demonstrate how this side channel can be exploited for detecting firmware activities, we experimentally validate it using impedance measurements to distinguish between distinct firmware operations with an accuracy of greater than 90%. These findings are the product of classifiers that are trained via machine learning. The implementation of our proposed methodology also leaves room for the use of hardware authentication.
Follow Us and Become Famous! Insights and Guidelines From Instagram Engagement Mechanisms
Venue: arXiv cs.SI 2023
Year: 2023

With 1.3 billion users, Instagram (IG) has also become a business tool. IG influencer marketing, expected to generate $33.25 billion in 2022, encourages companies and influencers to create trending content. Various methods have been proposed for predicting a post's popularity, i.e., how much engagement (e.g., Likes) it will generate. However, these methods are limited: first, they focus on forecasting the likes, ignoring the number of comments, which became crucial in 2021. Secondly, studies often use biased or limited data. Third, researchers focused on Deep Learning models to increase predictive performance, which are difficult to interpret. As a result, end-users can only estimate engagement after a post is created, which is inefficient and expensive. A better approach is to generate a post based on what people and IG like, e.g., by following guidelines. In this work, we uncover part of the underlying mechanisms driving IG engagement. To achieve this goal, we rely on statistical analysis and interpretable models rather than Deep Learning (black-box) approaches. We conduct extensive experiments using a worldwide dataset of 10 million posts created by 34K global influencers in nine different categories. With our simple yet powerful algorithms, we can predict engagement up to 94% of F1-Score, making us comparable and even superior to Deep Learning-based method. Furthermore, we propose a novel unsupervised algorithm for finding highly engaging topics on IG. Thanks to our interpretable approaches, we conclude by outlining guidelines for creating successful posts.
USER: Unified Semantic Enhancement with Momentum Contrast for Image-Text Retrieval
Venue: arXiv cs.CV 2023
Year: 2023

As a fundamental and challenging task in bridging language and vision domains, Image-Text Retrieval (ITR) aims at searching for the target instances that are semantically relevant to the given query from the other modality, and its key challenge is to measure the semantic similarity across different modalities. Although significant progress has been achieved, existing approaches typically suffer from two major limitations: (1) It hurts the accuracy of the representation by directly exploiting the bottom-up attention based region-level features where each region is equally treated. (2) It limits the scale of negative sample pairs by employing the mini-batch based end-to-end training mechanism. To address these limitations, we propose a Unified Semantic Enhancement Momentum Contrastive Learning (USER) method for ITR. Specifically, we delicately design two simple but effective Global representation based Semantic Enhancement (GSE) modules. One learns the global representation via the self-attention algorithm, noted as Self-Guided Enhancement (SGE) module. The other module benefits from the pre-trained CLIP module, which provides a novel scheme to exploit and transfer the knowledge from an off-the-shelf model, noted as CLIP-Guided Enhancement (CGE) module. Moreover, we incorporate the training mechanism of MoCo into ITR, in which two dynamic queues are employed to enrich and enlarge the scale of negative sample pairs. Meanwhile, a Unified Training Objective (UTO) is developed to learn from mini-batch based and dynamic queue based samples. Extensive experiments on the benchmark MSCOCO and Flickr30K datasets demonstrate the superiority of both retrieval accuracy and inference efficiency. Our source code will be released at https://github.com/zhangy0822/USER.
Algorithms for Acyclic Weighted Finite-State Automata with Failure Arcs
Venue: arXiv cs.DS 2023
Year: 2023

Weighted finite-state automata (WSFAs) are commonly used in NLP. Failure transitions are a useful extension for compactly representing backoffs or interpolation in $n$-gram models and CRFs, which are special cases of WFSAs. The pathsum in ordinary acyclic WFSAs is efficiently computed by the backward algorithm in time $O(|E|)$, where $E$ is the set of transitions. However, this does not allow failure transitions, and preprocessing the WFSA to eliminate failure transitions could greatly increase $|E|$. We extend the backward algorithm to handle failure transitions directly. Our approach is efficient when the average state has outgoing arcs for only a small fraction $s \ll 1$ of the alphabet $Σ$. We propose an algorithm for general acyclic WFSAs which runs in $O{\left(|E| + s |Σ| |Q| T_\text{max} \log{|Σ|}\right)}$, where $Q$ is the set of states and $T_\text{max}$ is the size of the largest connected component of failure transitions. When the failure transition topology satisfies a condition exemplified by CRFs, the $T_\text{max}$ factor can be dropped, and when the weight semiring is a ring, the $\log{|Σ|}$ factor can be dropped. In the latter case (ring-weighted acyclic WFSAs), we also give an alternative algorithm with complexity $\displaystyle O{\left(|E| + |Σ| |Q| \min(1,sπ_\text{max}) \right)}$, where $π_\text{max}$ is the size of the longest failure path.
Building Scalable Video Understanding Benchmarks through Sports
Venue: arXiv cs.CV 2023
Year: 2023

Existing benchmarks for evaluating long video understanding falls short on two critical aspects, either lacking in scale or quality of annotations. These limitations arise from the difficulty in collecting dense annotations for long videos, which often require manually labeling each frame. In this work, we introduce an automated Annotation and Video Stream Alignment Pipeline (abbreviated ASAP). We demonstrate the generality of ASAP by aligning unlabeled videos of four different sports with corresponding freely available dense web annotations (i.e. commentary). We then leverage ASAP scalability to create LCric, a large-scale long video understanding benchmark, with over 1000 hours of densely annotated long Cricket videos (with an average sample length of ~50 mins) collected at virtually zero annotation cost. We benchmark and analyze state-of-the-art video understanding models on LCric through a large set of compositional multi-choice and regression queries. We establish a human baseline that indicates significant room for new research to explore. Our human studies indicate that ASAP can align videos and annotations with high fidelity, precision, and speed. The dataset along with the code for ASAP and baselines can be accessed here: https://asap-benchmark.github.io/.
Multi-Biometric Fuzzy Vault based on Face and Fingerprints
Venue: arXiv cs.CR 2023
Year: 2023

The fuzzy vault scheme has been established as cryptographic primitive suitable for privacy-preserving biometric authentication. To improve accuracy and privacy protection, biometric information of multiple characteristics can be fused at feature level prior to locking it in a fuzzy vault. We construct a multi-biometric fuzzy vault based on face and multiple fingerprints. On a multi-biometric database constructed from the FRGCv2 face and the MCYT-100 fingerprint databases, a perfect recognition accuracy is achieved at a false accept security above 30 bits. Further, we provide a formalisation of feature-level fusion in multi-biometric fuzzy vaults, on the basis of which relevant security issues are elaborated. Said security issues, for which we define countermeasures, are commonly ignored and may impair the overall system's security.
Computer Science for Future -- Sustainability and Climate Protection in the Computer Science Courses of the HAW Hamburg
Venue: arXiv cs.CY 2023
Year: 2023

Computer Science for Future (CS4F) is an initiative in the Department of Computer Science at HAW Hamburg. The aim of the initiative is a paradigm shift in the discipline of computer science, thus establishing sustainability goals as a primary leitmotif for teaching and research. The focus is on teaching since the most promising multipliers are the students of a university. The change in teaching influences our research, the transfer to business and civil society as well as the change in our own institution. In this article, we present the initiative CS4F and reflect primarily on the role of students as amplifiers in the transformation process of computer science.
Cooperation Learning Enhanced Colonic Polyp Segmentation Based on Transformer-CNN Fusion
Venue: arXiv cs.CV 2023
Year: 2023

Traditional segmentation methods for colonic polyps are mainly designed based on low-level features. They could not accurately extract the location of small colonic polyps. Although the existing deep learning methods can improve the segmentation accuracy, their effects are still unsatisfied. To meet the above challenges, we propose a hybrid network called Fusion-Transformer-HardNetMSEG (i.e., Fu-TransHNet) in this study. Fu-TransHNet uses deep learning of different mechanisms to fuse each other and is enhanced with multi-view collaborative learning techniques. Firstly, the Fu-TransHNet utilizes the Transformer branch and the CNN branch to realize the global feature learning and local feature learning, respectively. Secondly, a fusion module is designed to integrate the features from two branches. The fusion module consists of two parts: 1) the Global-Local Feature Fusion (GLFF) part and 2) the Dense Fusion of Multi-scale features (DFM) part. The former is built to compensate the feature information mission from two branches at the same scale; the latter is constructed to enhance the feature representation. Thirdly, the above two branches and fusion modules utilize multi-view cooperative learning techniques to obtain their respective weights that denote their importance and then make a final decision comprehensively. Experimental results showed that the Fu-TransHNet network was superior to the existing methods on five widely used benchmark datasets. In particular, on the ETIS-LaribPolypDB dataset containing many small-target colonic polyps, the mDice obtained by Fu-TransHNet were 12.4% and 6.2% higher than the state-of-the-art methods HardNet-MSEG and TransFuse-s, respectively.
BSNet: Lane Detection via Draw B-spline Curves Nearby
Venue: arXiv cs.CV 2023
Year: 2023

Curve-based methods are one of the classic lane detection methods. They learn the holistic representation of lane lines, which is intuitive and concise. However, their performance lags behind the recent state-of-the-art methods due to the limitation of their lane representation and optimization. In this paper, we revisit the curve-based lane detection methods from the perspectives of the lane representations' globality and locality. The globality of lane representation is the ability to complete invisible parts of lanes with visible parts. The locality of lane representation is the ability to modify lanes locally which can simplify parameter optimization. Specifically, we first propose to exploit the b-spline curve to fit lane lines since it meets the locality and globality. Second, we design a simple yet efficient network BSNet to ensure the acquisition of global and local features. Third, we propose a new curve distance to make the lane detection optimization objective more reasonable and alleviate ill-conditioned problems. The proposed methods achieve state-of-the-art performance on the Tusimple, CULane, and LLAMAS datasets, which dramatically improved the accuracy of curve-based methods in the lane detection task while running far beyond real-time (197FPS).
DR-WLC: Dimensionality Reduction cognition for object detection and pose estimation by Watching, Learning and Checking
Venue: arXiv cs.CV 2023
Year: 2023

Object detection and pose estimation are difficult tasks in robotics and autonomous driving. Existing object detection and pose estimation methods mostly adopt the same-dimensional data for training. For example, 2D object detection usually requires a large amount of 2D annotation data with high cost. Using high-dimensional information to supervise lower-dimensional tasks is a feasible way to reduce datasets size. In this work, the DR-WLC, a dimensionality reduction cognitive model, which can perform both object detection and pose estimation tasks at the same time is proposed. The model only requires 3D model of objects and unlabeled environment images (with or without objects) to finish the training. In addition, a bounding boxes generation strategy is also proposed to build the relationship between 3D model and 2D object detection task. Experiments show that our method can qualify the work without any manual annotations and it is easy to deploy for practical applications. Source code is at https://github.com/IN2-ViAUn/DR-WLC.