OpenRaiser/Intern-Atlas huggingface dataset:
Towards Scalable EM-based Anomaly Detection For Embedded Devices Through Synthetic Fingerprinting
Venue: arXiv cs.CR 2023
Year: 2023

Embedded devices are omnipresent in modern networks including the ones operating inside critical environments. However, due to their constrained nature, novel mechanisms are required to provide external, and non-intrusive anomaly detection. Among such approaches, one that has gained traction is based on the analysis of the electromagnetic (EM) signals that get emanated during a device's operation. However, one of the most neglected challenges of this approach is the requirement for manually gathering and fingerprinting the signals that correspond to each execution path of the software/firmware. Indeed, even simple programs are comprised of hundreds if not thousands of branches thus, making the fingerprinting stage an extremely time-consuming process that involves the manual labor of a human specialist. To address this issue, we propose a framework for generating synthetic EM signals directly from the machine code. The synthetic signals can be used to train a Machine Learning based (ML) system for anomaly detection. The main advantage of the proposed approach is that it completely removes the need for an elaborate and error-prone fingerprinting stage, thus, dramatically increasing the scalability of the corresponding protection mechanisms. The experimental evaluations indicate that our method provides high detection accuracy (above 90% AUC score) when employed for the detection of injection attacks. Moreover, the proposed methodology inflicts only a small penalty (-1.3%) in accuracy for the detection of the injection of as little as four malicious instructions when compared to the same methods if real signals were to be used.
VuLASTE: Long Sequence Model with Abstract Syntax Tree Embedding for vulnerability Detection
Venue: arXiv cs.SE 2023
Year: 2023

In this paper, we build a model named VuLASTE, which regards vulnerability detection as a special text classification task. To solve the vocabulary explosion problem, VuLASTE uses a byte level BPE algorithm from natural language processing. In VuLASTE, a new AST path embedding is added to represent source code nesting information. We also use a combination of global and dilated window attention from Longformer to extract long sequence semantic from source code. To solve the data imbalance problem, which is a common problem in vulnerability detection datasets, focal loss is used as loss function to make model focus on poorly classified cases during training. To test our model performance on real-world source code, we build a cross-language and multi-repository vulnerability dataset from Github Security Advisory Database. On this dataset, VuLASTE achieved top 50, top 100, top 200, top 500 hits of 29, 51, 86, 228, which are higher than state-of-art researches.
Towards Precision in Appearance-based Gaze Estimation in the Wild
Venue: arXiv cs.CV 2023
Year: 2023

Appearance-based gaze estimation systems have shown great progress recently, yet the performance of these techniques depend on the datasets used for training. Most of the existing gaze estimation datasets setup in interactive settings were recorded in laboratory conditions and those recorded in the wild conditions display limited head pose and illumination variations. Further, we observed little attention so far towards precision evaluations of existing gaze estimation approaches. In this work, we present a large gaze estimation dataset, PARKS-Gaze, with wider head pose and illumination variation and with multiple samples for a single Point of Gaze (PoG). The dataset contains 974 minutes of data from 28 participants with a head pose range of 60 degrees in both yaw and pitch directions. Our within-dataset and cross-dataset evaluations and precision evaluations indicate that the proposed dataset is more challenging and enable models to generalize on unseen participants better than the existing in-the-wild datasets. The project page can be accessed here: https://github.com/lrdmurthy/PARKS-Gaze
Neural networks meet hyperelasticity: A guide to enforcing physics
Venue: arXiv cs.CE 2023
Year: 2023

In the present work, a hyperelastic constitutive model based on neural networks is proposed which fulfills all common constitutive conditions by construction, and in particular, is applicable to compressible material behavior. Using different sets of invariants as inputs, a hyperelastic potential is formulated as a convex neural network, thus fulfilling symmetry of the stress tensor, objectivity, material symmetry, polyconvexity, and thermodynamic consistency. In addition, a physically sensible stress behavior of the model is ensured by using analytical growth terms, as well as normalization terms which ensure the undeformed state to be stress free and with zero energy. In particular, polyconvex, invariant-based stress normalization terms are formulated for both isotropic and transversely isotropic material behavior. By fulfilling all of these conditions in an exact way, the proposed physics-augmented model combines a sound mechanical basis with the extraordinary flexibility that neural networks offer. Thus, it harmonizes the theory of hyperelasticity developed in the last decades with the up-to-date techniques of machine learning. Furthermore, the non-negativity of the hyperelastic neural network-based potentials is numerically examined by sampling the space of admissible deformations states, which, to the best of the authors' knowledge, is the only possibility for the considered nonlinear compressible models. For the isotropic neural network model, the sampling space required for that is reduced by analytical considerations. In addition, a proof for the non-negativity of the compressible Neo-Hooke potential is presented. The applicability of the model is demonstrated by calibrating it on data generated with analytical potentials, which is followed by an application of the model to finite element simulations. In addition, an adaption of the model to noisy data is shown and its [...]
FineDeb: A Debiasing Framework for Language Models
Venue: arXiv cs.CL 2023
Year: 2023

As language models are increasingly included in human-facing machine learning tools, bias against demographic subgroups has gained attention. We propose FineDeb, a two-phase debiasing framework for language models that starts with contextual debiasing of embeddings learned by pretrained language models. The model is then fine-tuned on a language modeling objective. Our results show that FineDeb offers stronger debiasing in comparison to other methods which often result in models as biased as the original language model. Our framework is generalizable for demographics with multiple classes, and we demonstrate its effectiveness through extensive experiments and comparisons with state of the art techniques. We release our code and data on GitHub.
Distributed Computation Offloading of an application from mobile/IoT device to cloud
Venue: arXiv cs.DC 2023
Year: 2023

In Covid-19 pandemic, the number of users connecting to the Internet using mobile devices increased. People are doing there every task using mobile phones [16]. These devices are battery-powered and have limited computation capabilities. Their computational capabilities can be enhanced by computation offloading in which required computation is to be done on a third-party server on a cloud instead of the device itself. The cloud offers virtually infinite computation and storage. We proposed that by exploiting parallelism within an application call hierarchy we can decrease the execution time of off-loadable parts and minimize data resend in case of VM crash. We determine function call paths that are independent of each other within an application and schedule each of them on separate VMs in a distributed way. Wherever such independent paths merge, we collapse to a single VM and whenever the paths diverge again, we schedule multiple VMs. If any single VM fails another copy will be created. However, only the code and data associated with the crashed VM needs to be re-transmitted from the client device. In the case of face reorganization application and montage application we decrease execution time to 27.5% and 43.43% respectively. Whereas the data resend in case if any of both VMs crash will be the portion of the application that had been offloaded to respective VM at depending upon the level of parallelism they have which save mobile battery in case of Resend. We will also discuss the energy consumption effect of using multiple Vms for a job VS single Vm for the same job.
Multiuser Offloading with Cloud Server Data
Venue: arXiv cs.IT 2023
Year: 2023

Computation offloading becomes useful for users of limited computing power and mobile edge computing (MEC) can help mobile users perform their tasks more effectively. In this paper, we consider MEC when users perform tasks with local data as well as data at cloud storage servers, since users can often keep their data at cloud storage servers or have tasks that need to use public datasets. An optimization problem to minimize the total consumed energy of users as well as a base station (BS) is formulated with a total transmission time constraint, which is solved by converting into multiple binary integer linear programming problems.
TempEL: Linking Dynamically Evolving and Newly Emerging Entities
Venue: arXiv cs.CL 2023
Year: 2023

In our continuously evolving world, entities change over time and new, previously non-existing or unknown, entities appear. We study how this evolutionary scenario impacts the performance on a well established entity linking (EL) task. For that study, we introduce TempEL, an entity linking dataset that consists of time-stratified English Wikipedia snapshots from 2013 to 2022, from which we collect both anchor mentions of entities, and these target entities' descriptions. By capturing such temporal aspects, our newly introduced TempEL resource contrasts with currently existing entity linking datasets, which are composed of fixed mentions linked to a single static version of a target Knowledge Base (e.g., Wikipedia 2010 for CoNLL-AIDA). Indeed, for each of our collected temporal snapshots, TempEL contains links to entities that are continual, i.e., occur in all of the years, as well as completely new entities that appear for the first time at some point. Thus, we enable to quantify the performance of current state-of-the-art EL models for: (i) entities that are subject to changes over time in their Knowledge Base descriptions as well as their mentions' contexts, and (ii) newly created entities that were previously non-existing (e.g., at the time the EL model was trained). Our experimental results show that in terms of temporal performance degradation, (i) continual entities suffer a decrease of up to 3.1% EL accuracy, while (ii) for new entities this accuracy drop is up to 17.9%. This highlights the challenge of the introduced TempEL dataset and opens new research prospects in the area of time-evolving entity disambiguation.
Timed Partial Order Inference Algorithm
Venue: arXiv cs.FL 2023
Year: 2023

In this work, we propose the model of timed partial orders (TPOs) for specifying workflow schedules, especially for modeling manufacturing processes. TPOs integrate partial orders over events in a workflow, specifying ``happens-before'' relations, with timing constraints specified using guards and resets on clocks -- an idea borrowed from timed-automata specifications. TPOs naturally allow us to capture event ordering, along with a restricted but useful class of timing relationships. Next, we consider the problem of mining TPO schedules from workflow logs, which include events along with their time stamps. We demonstrate a relationship between formulating TPOs and the graph-coloring problem, and present an algorithm for learning TPOs with correctness guarantees. We demonstrate our approach on synthetic datasets, including two datasets inspired by real-life applications of aircraft turnaround and gameplay videos of the Overcooked computer game. Our TPO mining algorithm can infer TPOs involving hundreds of events from thousands of data-points within a few seconds. We show that the resulting TPOs provide useful insights into the dependencies and timing constraints for workflows.
Generating Dispatching Rules for the Interrupting Swap-Allowed Blocking Job Shop Problem Using Graph Neural Network and Reinforcement Learning
Venue: arXiv cs.LG 2023
Year: 2023

The interrupting swap-allowed blocking job shop problem (ISBJSSP) is a complex scheduling problem that is able to model many manufacturing planning and logistics applications realistically by addressing both the lack of storage capacity and unforeseen production interruptions. Subjected to random disruptions due to machine malfunction or maintenance, industry production settings often choose to adopt dispatching rules to enable adaptive, real-time re-scheduling, rather than traditional methods that require costly re-computation on the new configuration every time the problem condition changes dynamically. To generate dispatching rules for the ISBJSSP problem, we introduce a dynamic disjunctive graph formulation characterized by nodes and edges subjected to continuous deletions and additions. This formulation enables the training of an adaptive scheduler utilizing graph neural networks and reinforcement learning. Furthermore, a simulator is developed to simulate interruption, swapping, and blocking in the ISBJSSP setting. Employing a set of reported benchmark instances, we conduct a detailed experimental study on ISBJSSP instances with a range of machine shutdown probabilities to show that the scheduling policies generated can outperform or are at least as competitive as existing dispatching rules with predetermined priority. This study shows that the ISBJSSP, which requires real-time adaptive solutions, can be scheduled efficiently with the proposed method when production interruptions occur with random machine shutdowns.
Connectivity Enhanced Safe Neural Network Planner for Lane Changing in Mixed Traffic
Venue: arXiv cs.RO 2023
Year: 2023

Connectivity technology has shown great potentials in improving the safety and efficiency of transportation systems by providing information beyond the perception and prediction capabilities of individual vehicles. However, it is expected that human-driven and autonomous vehicles, and connected and non-connected vehicles need to share the transportation network during the transition period to fully connected and automated transportation systems. Such mixed traffic scenarios significantly increase the complexity in analyzing system behavior and quantifying uncertainty for highly interactive scenarios, e.g., lane changing. It is even harder to ensure system safety when neural network based planners are leveraged to further improve efficiency. In this work, we propose a connectivity-enhanced neural network based lane changing planner. By cooperating with surrounding connected vehicles in dynamic environment, our proposed planner will adapt its planned trajectory according to the analysis of a safe evasion trajectory. We demonstrate the strength of our planner design in improving efficiency and ensuring safety in various mixed traffic scenarios with extensive simulations. We also analyze the system robustness when the communication or coordination is not perfect.
Optimal LZ-End Parsing is Hard
Venue: arXiv cs.DS 2023
Year: 2023

LZ-End is a variant of the well-known Lempel-Ziv parsing family such that each phrase of the parsing has a previous occurrence, with the additional constraint that the previous occurrence must end at the end of a previous phrase. LZ-End was initially proposed as a greedy parsing, where each phrase is determined greedily from left to right, as the longest factor that satisfies the above constraint~[Kreft & Navarro, 2010]. In this work, we consider an optimal LZ-End parsing that has the minimum number of phrases in such parsings. We show that a decision version of computing the optimal LZ-End parsing is NP-complete by showing a reduction from the vertex cover problem. Moreover, we give a MAX-SAT formulation for the optimal LZ-End parsing adapting an approach for computing various NP-hard repetitiveness measures recently presented by [Bannai et al., 2022]. We also consider the approximation ratio of the size of greedy LZ-End parsing to the size of the optimal LZ-End parsing, and give a lower bound of the ratio which asymptotically approaches $2$.
Linking data separation, visual separation, and classifier performance using pseudo-labeling by contrastive learning
Venue: arXiv cs.LG 2023
Year: 2023

Lacking supervised data is an issue while training deep neural networks (DNNs), mainly when considering medical and biological data where supervision is expensive. Recently, Embedded Pseudo-Labeling (EPL) addressed this problem by using a non-linear projection (t-SNE) from a feature space of the DNN to a 2D space, followed by semi-supervised label propagation using a connectivity-based method (OPFSemi). We argue that the performance of the final classifier depends on the data separation present in the latent space and visual separation present in the projection. We address this by first proposing to use contrastive learning to produce the latent space for EPL by two methods (SimCLR and SupCon) and by their combination, and secondly by showing, via an extensive set of experiments, the aforementioned correlations between data separation, visual separation, and classifier performance. We demonstrate our results by the classification of five real-world challenging image datasets of human intestinal parasites with only 1% supervised samples.
Chain of Hindsight Aligns Language Models with Feedback
Venue: arXiv cs.LG 2023
Year: 2023

Learning from human preferences is important for language models to match human needs and to align with human and social values. Prior works have achieved remarkable successes by learning from human feedback to understand and follow instructions. Nonetheless, these methods are either founded on hand-picked model generations that are favored by human annotators, rendering them inefficient in terms of data utilization and challenging to apply in general, or they depend on reinforcement learning, which often suffers from imperfect reward functions and relies on extremely challenging optimizations. In this work, we propose a novel technique, Chain of Hindsight, that is easy to optimize and can learn from any form of feedback, regardless of its polarity. Our idea is inspired by how humans learn from extensive feedback presented in the form of languages. We convert all types of feedback into sequences of sentences, which are then used to fine-tune the model, allowing us to take advantage of the language comprehension capabilities of language models. We condition the model on a sequence of model generations paired with feedback. By doing so, the model is trained to generate outputs based on feedback, while learning to identify and correct negative attributes or errors. Applying our method to large language models, we observed that Chain of Hindsight significantly surpasses previous methods in aligning language models with human preferences. We report significant improvements on summarization and dialogue benchmarks, with our approach markedly preferred in human evaluations.
On Exact Sampling in the Two-Variable Fragment of First-Order Logic
Venue: arXiv cs.AI 2023
Year: 2023

In this paper, we study the sampling problem for first-order logic proposed recently by Wang et al. -- how to efficiently sample a model of a given first-order sentence on a finite domain? We extend their result for the universally-quantified subfragment of two-variable logic $\mathbf{FO}^2$ ($\mathbf{UFO}^2$) to the entire fragment of $\mathbf{FO}^2$. Specifically, we prove the domain-liftability under sampling of $\mathbf{FO}^2$, meaning that there exists a sampling algorithm for $\mathbf{FO}^2$ that runs in time polynomial in the domain size. We then further show that this result continues to hold even in the presence of counting constraints, such as $\forall x\exists_{=k} y: φ(x,y)$ and $\exists_{=k} x\forall y: φ(x,y)$, for some quantifier-free formula $φ(x,y)$. Our proposed method is constructive, and the resulting sampling algorithms have potential applications in various areas, including the uniform generation of combinatorial structures and sampling in statistical-relational models such as Markov logic networks and probabilistic logic programs.
AuthentiSense: A Scalable Behavioral Biometrics Authentication Scheme using Few-Shot Learning for Mobile Platforms
Venue: arXiv cs.CR 2023
Year: 2023

Mobile applications are widely used for online services sharing a large amount of personal data online. One-time authentication techniques such as passwords and physiological biometrics (e.g., fingerprint, face, and iris) have their own advantages but also disadvantages since they can be stolen or emulated, and do not prevent access to the underlying device, once it is unlocked. To address these challenges, complementary authentication systems based on behavioural biometrics have emerged. The goal is to continuously profile users based on their interaction with the mobile device. However, existing behavioural authentication schemes are not (i) user-agnostic meaning that they cannot dynamically handle changes in the user-base without model re-training, or (ii) do not scale well to authenticate millions of users. In this paper, we present AuthentiSense, a user-agnostic, scalable, and efficient behavioural biometrics authentication system that enables continuous authentication and utilizes only motion patterns (i.e., accelerometer, gyroscope and magnetometer data) while users interact with mobile apps. Our approach requires neither manually engineered features nor a significant amount of data for model training. We leverage a few-shot learning technique, called Siamese network, to authenticate users at a large scale. We perform a systematic measurement study and report the impact of the parameters such as interaction time needed for authentication and n-shot verification (comparison with enrollment samples) at the recognition stage. Remarkably, AuthentiSense achieves high accuracy of up to 97% in terms of F1-score even when evaluated in a few-shot fashion that requires only a few behaviour samples per user (3 shots). Our approach accurately authenticates users only after 1 second of user interaction. For AuthentiSense, we report a FAR and FRR of 0.023 and 0.057, respectively.
Challenges and Opportunities of Content Optimization for Freeform User Interfaces
Venue: arXiv cs.HC 2023
Year: 2023

While recent innovations on shape technologies allow for the creation of displays with almost unlimited form factors, current graphical user interfaces still rely on rectangular layouts and contents. This rectangular legacy hinders the progress of freeform displays, which are particularly relevant for pervasive scenarios to display interactive dynamic content where and when needed. By challenging the prevailing layout tradition on rectangular displays, freeform user interfaces raise design challenges which call for exploring the interlink between computational approaches and user interface generation and adaptation. In this position paper we report on previous work on content optimization for freeform user interfaces and anticipate the upcoming challenges and opportunities.