OpenRaiser/Intern-Atlas huggingface dataset:
LoadLord: Loading on the Fly to Defend Against Code-Reuse Attacks
Venue: arXiv cs.CR 2023
Year: 2023

Code-reuse attacks have become a kind of common attack method, in which attackers use the existing code in the program to hijack the control flow. Most existing defenses focus on control flow integrity (CFI), code randomization, and software debloating. However, most fine-grained schemes of those that ensure such high security suffer from significant performance overhead, and only reduce attack surfaces such as software debloating can not defend against code-reuse attacks completely. In this paper, from the perspective of shrinking the available code space at runtime, we propose LoadLord, which dynamically loads, and timely unloads functions during program running to defend against code-reuse attacks. LoadLord can reduce the number of gadgets in memory, especially high-risk gadgets. Moreover, LoadLord ensures the control flow integrity of the loading process and breaks the necessary conditions to build a gadget chain. We implemented LoadLord on Linux operating system and experimented that when limiting only 1/16 of the original function. As a result, LoadLord can defend against code-reuse attacks and has an average runtime overhead of 1.7% on the SPEC CPU 2006, reducing gadgets by 94.02%.
Multi-view Feature Extraction based on Triple Contrastive Heads
Venue: arXiv cs.CV 2023
Year: 2023

Multi-view feature extraction is an efficient approach for alleviating the issue of dimensionality in highdimensional multi-view data. Contrastive learning (CL), which is a popular self-supervised learning method, has recently attracted considerable attention. In this study, we propose a novel multi-view feature extraction method based on triple contrastive heads, which combines the sample-, recovery- , and feature-level contrastive losses to extract the sufficient yet minimal subspace discriminative information in compliance with information bottleneck principle. In MFETCH, we construct the feature-level contrastive loss, which removes the redundent information in the consistency information to achieve the minimality of the subspace discriminative information. Moreover, the recovery-level contrastive loss is also constructed in MFETCH, which captures the view-specific discriminative information to achieve the sufficiency of the subspace discriminative information.The numerical experiments demonstrate that the proposed method offers a strong advantage for multi-view feature extraction.
An Extended Study of Human-like Behavior under Adversarial Training
Venue: arXiv cs.CV 2023
Year: 2023

Neural networks have a number of shortcomings. Amongst the severest ones is the sensitivity to distribution shifts which allows models to be easily fooled into wrong predictions by small perturbations to inputs that are often imperceivable to humans and do not have to carry semantic meaning. Adversarial training poses a partial solution to address this issue by training models on worst-case perturbations. Yet, recent work has also pointed out that the reasoning in neural networks is different from humans. Humans identify objects by shape, while neural nets mainly employ texture cues. Exemplarily, a model trained on photographs will likely fail to generalize to datasets containing sketches. Interestingly, it was also shown that adversarial training seems to favorably increase the shift toward shape bias. In this work, we revisit this observation and provide an extensive analysis of this effect on various architectures, the common $\ell_2$- and $\ell_\infty$-training, and Transformer-based models. Further, we provide a possible explanation for this phenomenon from a frequency perspective.
Visual motion analysis of the player's finger
Venue: arXiv cs.CV 2023
Year: 2023

This work is about the extraction of the motion of fingers, in their three articulations, of a keyboard player from a video sequence. The relevance of the problem involves several aspects, in fact, the extraction of the movements of the fingers may be used to compute the keystroke efficiency and individual joint contributions, as showed by Werner Goebl and Caroline Palmer in the paper 'Temporal Control and Hand Movement Efficiency in Skilled Music Performance'. Those measures are directly related to the precision in timing and force measures. A very good approach to the hand gesture recognition problem has been presented in the paper ' Real-Time Hand Gesture Recognition Using Finger Segmentation'. Detecting the keys pressed on a keyboard is a task that can be complex because of the shadows that can degrade the quality of the result and possibly cause the detection of not pressed keys. Among the several approaches that already exist, a great amount of them is based on the subtraction of frames in order to detect the movements of the keys caused by their pressure. Detecting the keys that are pressed could be useful to automatically evaluate the performance of a pianist or to automatically write sheet music of the melody that is being played.
Longitudinal Performance of Iris Recognition in Children: Time Intervals up to Six years
Venue: arXiv cs.CV 2023
Year: 2023

The temporal stability of iris recognition performance is core to its success as a biometric modality. With the expanding horizon of applications for children, gaps in the knowledge base on the temporal stability of iris recognition performance in children have impacted decision-making during applications at the global scale. This report presents the most extensive analysis of longitudinal iris recognition performance in children with data from the same 230 children over 6.5 years between enrollment and query for ages 4 to 17 years. Assessment of match scores, statistical modelling of variability factors impacting match scores and in-depth assessment of the root causes of the false rejections concludes no impact on iris recognition performance due to aging.
LocalEyenet: Deep Attention framework for Localization of Eyes
Venue: arXiv cs.CV 2023
Year: 2023

Development of human machine interface has become a necessity for modern day machines to catalyze more autonomy and more efficiency. Gaze driven human intervention is an effective and convenient option for creating an interface to alleviate human errors. Facial landmark detection is very crucial for designing a robust gaze detection system. Regression based methods capacitate good spatial localization of the landmarks corresponding to different parts of the faces. But there are still scope of improvements which have been addressed by incorporating attention. In this paper, we have proposed a deep coarse-to-fine architecture called LocalEyenet for localization of only the eye regions that can be trained end-to-end. The model architecture, build on stacked hourglass backbone, learns the self-attention in feature maps which aids in preserving global as well as local spatial dependencies in face image. We have incorporated deep layer aggregation in each hourglass to minimize the loss of attention over the depth of architecture. Our model shows good generalization ability in cross-dataset evaluation and in real-time localization of eyes.
Toward Data-Driven Glare Classification and Prediction for Marine Megafauna Survey
Venue: arXiv cs.CV 2023
Year: 2023

Critically endangered species in Canadian North Atlantic waters are systematically surveyed to estimate species populations which influence governing policies. Due to its impact on policy, population accuracy is important. This paper lays the foundation towards a data-driven glare modelling system, which will allow surveyors to preemptively minimize glare. Surveyors use a detection function to estimate megafauna populations which are not explicitly seen. A goal of the research is to maximize useful imagery collected, to that end we will use our glare model to predict glare and optimize for glare-free data collection. To build this model, we leverage a small labelled dataset to perform semi-supervised learning. The large dataset is labelled with a Cascading Random Forest Model using a naïve pseudo-labelling approach. A reflectance model is used, which pinpoints features of interest, to populate our datasets which allows for context-aware machine learning models. The pseudo-labelled dataset is used on two models: a Multilayer Perceptron and a Recurrent Neural Network. With this paper, we lay the foundation for data-driven mission planning; a glare modelling system which allows surveyors to preemptively minimize glare and reduces survey reliance on the detection function as an estimator of whale populations during periods of poor subsurface visibility.
DPPMask: Masked Image Modeling with Determinantal Point Processes
Venue: arXiv cs.CV 2023
Year: 2023

Masked Image Modeling (MIM) has achieved impressive representative performance with the aim of reconstructing randomly masked images. Despite the empirical success, most previous works have neglected the important fact that it is unreasonable to force the model to reconstruct something beyond recovery, such as those masked objects. In this work, we show that uniformly random masking widely used in previous works unavoidably loses some key objects and changes original semantic information, resulting in a misalignment problem and hurting the representative learning eventually. To address this issue, we augment MIM with a new masking strategy namely the DPPMask by substituting the random process with Determinantal Point Process (DPPs) to reduce the semantic change of the image after masking. Our method is simple yet effective and requires no extra learnable parameters when implemented within various frameworks. In particular, we evaluate our method on two representative MIM frameworks, MAE and iBOT. We show that DPPMask surpassed random sampling under both lower and higher masking ratios, indicating that DPPMask makes the reconstruction task more reasonable. We further test our method on the background challenge and multi-class classification tasks, showing that our method is more robust at various tasks.
A Method for Animating Children's Drawings of the Human Figure
Venue: arXiv cs.CV 2023
Year: 2023

Children's drawings have a wonderful inventiveness, creativity, and variety to them. We present a system that automatically animates children's drawings of the human figure, is robust to the variance inherent in these depictions, and is simple and straightforward enough for anyone to use. We demonstrate the value and broad appeal of our approach by building and releasing the Animated Drawings Demo, a freely available public website that has been used by millions of people around the world. We present a set of experiments exploring the amount of training data needed for fine-tuning, as well as a perceptual study demonstrating the appeal of a novel twisted perspective retargeting technique. Finally, we introduce the Amateur Drawings Dataset, a first-of-its-kind annotated dataset, collected via the public demo, containing over 178,000 amateur drawings and corresponding user-accepted character bounding boxes, segmentation masks, and joint location annotations.
DR.CPO: Diversified and Realistic 3D Augmentation via Iterative Construction, Random Placement, and HPR Occlusion
Venue: arXiv cs.CV 2023
Year: 2023

In autonomous driving, data augmentation is commonly used for improving 3D object detection. The most basic methods include insertion of copied objects and rotation and scaling of the entire training frame. Numerous variants have been developed as well. The existing methods, however, are considerably limited when compared to the variety of the real world possibilities. In this work, we develop a diversified and realistic augmentation method that can flexibly construct a whole-body object, freely locate and rotate the object, and apply self-occlusion and external-occlusion accordingly. To improve the diversity of the whole-body object construction, we develop an iterative method that stochastically combines multiple objects observed from the real world into a single object. Unlike the existing augmentation methods, the constructed objects can be randomly located and rotated in the training frame because proper occlusions can be reflected to the whole-body objects in the final step. Finally, proper self-occlusion at each local object level and external-occlusion at the global frame level are applied using the Hidden Point Removal (HPR) algorithm that is computationally efficient. HPR is also used for adaptively controlling the point density of each object according to the object's distance from the LiDAR. Experiment results show that the proposed DR.CPO algorithm is data-efficient and model-agnostic without incurring any computational overhead. Also, DR.CPO can improve mAP performance by 2.08% when compared to the best 3D detection result known for KITTI dataset. The code is available at https://github.com/SNU-DRL/DRCPO.git
On-Device Unsupervised Image Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Along with the breakthrough of convolutional neural networks, learning-based segmentation has emerged in many research works. Most of them are based on supervised learning, requiring plenty of annotated data; however, to support segmentation, a label for each pixel is required, which is obviously expensive. As a result, the issue of lacking annotated segmentation data commonly exists. Continuous learning is a promising way to deal with this issue; however, it still has high demands on human labor for annotation. What's more, privacy is highly required in segmentation data for real-world applications, which further calls for on-device learning. In this paper, we aim to resolve the above issue in an alternative way: Instead of supervised segmentation, we propose to develop efficient unsupervised segmentation that can be executed on edge devices. Based on our observation that segmentation can obtain high performance when pixels are mapped to a high-dimension space, we for the first time bring brain-inspired hyperdimensional computing (HDC) to the segmentation task. We build the HDC-based unsupervised segmentation framework, namely "SegHDC". In SegHDC, we devise a novel encoding approach that follows the Manhattan distance. A clustering algorithm is further developed on top of the encoded high-dimension vectors to obtain segmentation results. Experimental results show SegHDC can significantly surpass neural network-based unsupervised segmentation. On a standard segmentation dataset, DSB2018, SegHDC can achieve a 28.0% improvement in Intersection over Union (IoU) score; meanwhile, it achieves over 300x speedup on Raspberry PI. What's more, for a larger size image in the BBBC005 dataset, the existing approach cannot be accommodated to Raspberry PI due to out of memory; on the other hand, SegHDC can obtain segmentation results within 3 minutes while achieving a 0.9587 IoU score.
A temporally quantized distribution of pupil diameters as a new feature for cognitive load classification
Venue: arXiv cs.CV 2023
Year: 2023

In this paper, we present a new feature that can be used to classify cognitive load based on pupil information. The feature consists of a temporal segmentation of the eye tracking recordings. For each segment of the temporal partition, a probability distribution of pupil size is computed and stored. These probability distributions can then be used to classify the cognitive load. The presented feature significantly improves the classification accuracy of the cognitive load compared to other statistical values obtained from eye tracking data, which represent the state of the art in this field. The applications of determining Cognitive Load from pupil data are numerous and could lead, for example, to pre-warning systems for burnouts. Link: https://es-cloud.cs.uni-tuebingen.de/d/8e2ab8c3fdd444e1a135/?p=%2FCognitiveLoadFeature&mode=list
Interpretable Bangla Sarcasm Detection using BERT and Explainable AI
Venue: arXiv cs.CL 2023
Year: 2023

A positive phrase or a sentence with an underlying negative motive is usually defined as sarcasm that is widely used in today's social media platforms such as Facebook, Twitter, Reddit, etc. In recent times active users in social media platforms are increasing dramatically which raises the need for an automated NLP-based system that can be utilized in various tasks such as determining market demand, sentiment analysis, threat detection, etc. However, since sarcasm usually implies the opposite meaning and its detection is frequently a challenging issue, data meaning extraction through an NLP-based model becomes more complicated. As a result, there has been a lot of study on sarcasm detection in English over the past several years, and there's been a noticeable improvement and yet sarcasm detection in the Bangla language's state remains the same. In this article, we present a BERT-based system that can achieve 99.60\% while the utilized traditional machine learning algorithms are only capable of achieving 89.93\%. Additionally, we have employed Local Interpretable Model-Agnostic Explanations that introduce explainability to our system. Moreover, we have utilized a newly collected bangla sarcasm dataset, BanglaSarc that was constructed specifically for the evaluation of this study. This dataset consists of fresh records of sarcastic and non-sarcastic comments, the majority of which are acquired from Facebook and YouTube comment sections.
The Complexity of Why-Provenance for Datalog Queries
Venue: arXiv cs.DB 2023
Year: 2023

Explaining why a database query result is obtained is an essential task towards the goal of Explainable AI, especially nowadays where expressive database query languages such as Datalog play a critical role in the development of ontology-based applications. A standard way of explaining a query result is the so-called why-provenance, which essentially provides information about the witnesses to a query result in the form of subsets of the input database that are sufficient to derive that result. To our surprise, despite the fact that the notion of why-provenance for Datalog queries has been around for decades and intensively studied, its computational complexity remains unexplored. The goal of this work is to fill this apparent gap in the why-provenance literature. Towards this end, we pinpoint the data complexity of why-provenance for Datalog queries and key subclasses thereof. The takeaway of our work is that why-provenance for recursive queries, even if the recursion is limited to be linear, is an intractable problem, whereas for non-recursive queries is highly tractable. Having said that, we experimentally confirm, by exploiting SAT solvers, that making why-provenance for (recursive) Datalog queries work in practice is not an unrealistic goal.
Test-time Detection and Repair of Adversarial Samples via Masked Autoencoder
Venue: arXiv cs.CV 2023
Year: 2023

Training-time defenses, known as adversarial training, incur high training costs and do not generalize to unseen attacks. Test-time defenses solve these issues but most existing test-time defenses require adapting the model weights, therefore they do not work on frozen models and complicate model memory management. The only test-time defense that does not adapt model weights aims to adapt the input with self-supervision tasks. However, we empirically found these self-supervision tasks are not sensitive enough to detect adversarial attacks accurately. In this paper, we propose DRAM, a novel defense method to detect and repair adversarial samples at test time via Masked autoencoder (MAE). We demonstrate how to use MAE losses to build a Kolmogorov-Smirnov test to detect adversarial samples. Moreover, we use the MAE losses to calculate input reversal vectors that repair adversarial samples resulting from previously unseen attacks. Results on large-scale ImageNet dataset show that, compared to all detection baselines evaluated, DRAM achieves the best detection rate (82% on average) on all eight adversarial attacks evaluated. For attack repair, DRAM improves the robust accuracy by 6% ~ 41% for standard ResNet50 and 3% ~ 8% for robust ResNet50 compared with the baselines that use contrastive learning and rotation prediction.
Consensus on an Unknown Torus with Dense Byzantine Faults
Venue: arXiv cs.DC 2023
Year: 2023

We present a solution to consensus on a torus with Byzantine faults. Any solution to classic consensus that is tolerant to $f$ Byzantine faults requires $2f+1$ node-disjoint paths. Due to limited torus connectivity, this bound necessitates spatial separation between faults. Our solution does not require this many disjoint paths and tolerates dense faults. Specifically, we consider the case where all faults are in one column. We address the version of consensus where only processes in fault-free columns must agree. We prove that even this weaker version is not solvable if the column may be completely faulty. We then present a solution for the case where at least one row is fault-free. The correct processes share orientation but do not know the identities of other processes or the torus dimensions. The communication is synchronous. To achieve our solution, we build and prove correct an all-to-all broadcast algorithm $\mathcal{BAT}$ that guarantees delivery to all processes in fault-free columns. We use this algorithm to solve our weak consensus problem. Our solution, $\mathcal{CBAT}$, runs in $O(H+W)$ rounds, where $H$ and $W$ are torus height and width respectively. We extend our consensus solution to the fixed message size model where it runs in $O(H^3W^2)$ rounds. Our results are immediately applicable if the faults are located in a single row, rather than a column.
Overcoming Algorithm Aversion: A Comparison between Process and Outcome Control
Venue: arXiv cs.HC 2023
Year: 2023

Algorithm aversion occurs when humans are reluctant to use algorithms despite their superior performance. Studies show that giving users outcome control by providing agency over how models' predictions are incorporated into decision-making mitigates algorithm aversion. We study whether algorithm aversion is mitigated by process control, wherein users can decide what input factors and algorithms to use in model training. We conduct a replication study of outcome control, and test novel process control study conditions on Amazon Mechanical Turk (MTurk) and Prolific. Our results partly confirm prior findings on the mitigating effects of outcome control, while also forefronting reproducibility challenges. We find that process control in the form of choosing the training algorithm mitigates algorithm aversion, but changing inputs does not. Furthermore, giving users both outcome and process control does not reduce algorithm aversion more than outcome or process control alone. This study contributes to design considerations around mitigating algorithm aversion.