OpenRaiser/Intern-Atlas huggingface dataset:
Thwarting Piracy: Anti-debugging Using GPU-assisted Self-healing Codes
Venue: arXiv cs.CR 2022
Year: 2022

Software piracy is one of the concerns in the IT sector. Pirates leverage the debugger tools to reverse engineer the logic that verifies the license keys or bypass the entire verification process. Anti-debugging techniques are used to defeat piracy using self-healing codes. However, anti-debugging methods can be defeated when the licensing protections are limited to CPU-based implementation by writing custom codes to deactivate the anti-debugging methods. In the paper, we demonstrate how GPU implementation can prevent pirates from deactivating the anti-debugging methods by using the limitations of debugging on GPU. Generally, GPUs do not support debugging directly on the hardware, and therefore all the debugging is limited to CPU-based emulation. Also, a process running on CPU generally does not have any visibility on codes running on GPU, which comes as an added benefit for our work. We provide an implementation on GPU to show the feasibility of our method. As GPUs are getting widespread with the raise in popularity of gaming software, our technique provides a method to protect against piracy. Our method thwarts any attempts to bypass the license verification step thus offering a better anti-piracy mechanism.
Vertical Federated Linear Contextual Bandits
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we investigate a novel problem of building contextual bandits in the vertical federated setting, i.e., contextual information is vertically distributed over different departments. This problem remains largely unexplored in the research community. To this end, we carefully design a customized encryption scheme named orthogonal matrix-based mask mechanism(O3M) for encrypting local contextual information while avoiding expensive conventional cryptographic techniques. We further apply the mechanism to two commonly-used bandit algorithms, LinUCB and LinTS, and instantiate two practical protocols for online recommendation under the vertical federated setting. The proposed protocols can perfectly recover the service quality of centralized bandit algorithms while achieving a satisfactory runtime efficiency, which is theoretically proved and analyzed in this paper. By conducting extensive experiments on both synthetic and real-world datasets, we show the superiority of the proposed method in terms of privacy protection and recommendation performance.
MovieCLIP: Visual Scene Recognition in Movies
Venue: arXiv cs.CV 2022
Year: 2022

Longform media such as movies have complex narrative structures, with events spanning a rich variety of ambient visual scenes. Domain specific challenges associated with visual scenes in movies include transitions, person coverage, and a wide array of real-life and fictional scenarios. Existing visual scene datasets in movies have limited taxonomies and don't consider the visual scene transition within movie clips. In this work, we address the problem of visual scene recognition in movies by first automatically curating a new and extensive movie-centric taxonomy of 179 scene labels derived from movie scripts and auxiliary web-based video datasets. Instead of manual annotations which can be expensive, we use CLIP to weakly label 1.12 million shots from 32K movie clips based on our proposed taxonomy. We provide baseline visual models trained on the weakly labeled dataset called MovieCLIP and evaluate them on an independent dataset verified by human raters. We show that leveraging features from models pretrained on MovieCLIP benefits downstream tasks such as multi-label scene and genre classification of web videos and movie trailers.
Iterative collaborative routing among equivariant capsules for transformation-robust capsule networks
Venue: arXiv cs.CV 2022
Year: 2022

Transformation-robustness is an important feature for machine learning models that perform image classification. Many methods aim to bestow this property to models by the use of data augmentation strategies, while more formal guarantees are obtained via the use of equivariant models. We recognise that compositional, or part-whole structure is also an important aspect of images that has to be considered for building transformation-robust models. Thus, we propose a capsule network model that is, at once, equivariant and compositionality-aware. Equivariance of our capsule network model comes from the use of equivariant convolutions in a carefully-chosen novel architecture. The awareness of compositionality comes from the use of our proposed novel, iterative, graph-based routing algorithm, termed Iterative collaborative routing (ICR). ICR, the core of our contribution, weights the predictions made for capsules based on an iteratively averaged score of the degree-centralities of its nearest neighbours. Experiments on transformed image classification on FashionMNIST, CIFAR-10, and CIFAR-100 show that our model that uses ICR outperforms convolutional and capsule baselines to achieve state-of-the-art performance.
Execution Time Program Verification With Tight Bounds
Venue: arXiv cs.PL 2022
Year: 2022

This paper presents a proof system for reasoning about execution time bounds for a core imperative programming language. Proof systems are defined for three different scenarios: approximations of the worst-case execution time, exact time reasoning, and less pessimistic execution time estimation using amortized analysis. We define a Hoare logic for the three cases and prove its soundness with respect to an annotated cost-aware operational semantics. Finally, we define a verification conditions (VC) generator that generates the goals needed to prove program correctness, cost, and termination. Those goals are then sent to the Easycrypt toolset for validation. The practicality of the proof system is demonstrated with an implementation in OCaml of the different modules needed to apply it to example programs. Our case studies are motivated by real-time and cryptographic software.
General Image Descriptors for Open World Image Retrieval using ViT CLIP
Venue: arXiv cs.CV 2022
Year: 2022

The Google Universal Image Embedding (GUIE) Challenge is one of the first competitions in multi-domain image representations in the wild, covering a wide distribution of objects: landmarks, artwork, food, etc. This is a fundamental computer vision problem with notable applications in image retrieval, search engines and e-commerce. In this work, we explain our 4th place solution to the GUIE Challenge, and our "bag of tricks" to fine-tune zero-shot Vision Transformers (ViT) pre-trained using CLIP.
Graph Neural Networks with Trainable Adjacency Matrices for Fault Diagnosis on Multivariate Sensor Data
Venue: arXiv cs.AI 2022
Year: 2022

Timely detected anomalies in the chemical technological processes, as well as the earliest detection of the cause of the fault, significantly reduce the production cost in the industrial factories. Data on the state of the technological process and the operation of production equipment are received by a large number of different sensors. To better predict the behavior of the process and equipment, it is necessary not only to consider the behavior of the signals in each sensor separately, but also to take into account their correlation and hidden relationships with each other. Graph-based data representation helps with this. The graph nodes can be represented as data from the different sensors, and the edges can display the influence of these data on each other. In this work, the possibility of applying graph neural networks to the problem of fault diagnosis in a chemical process is studied. It was proposed to construct a graph during the training of graph neural network. This allows to train models on data where the dependencies between the sensors are not known in advance. In this work, several methods for obtaining adjacency matrices were considered, as well as their quality was studied. It has also been proposed to use multiple adjacency matrices in one model. We showed state-of-the-art performance on the fault diagnosis task with the Tennessee Eastman Process dataset. The proposed graph neural networks outperformed the results of recurrent neural networks.
On the Automation, Optimization, and In-Orbit Validation of Intelligent Satellite Constellation Operations
Venue: arXiv cs.NI 2022
Year: 2022

Recent breakthroughs in technology have led to a thriving "new space" culture in low-Earth orbit (LEO) in which performance and cost considerations dominate over resilience and reliability as mission goals. These advances create a manifold of opportunities for new research and business models but come with a number of striking new challenges. In particular, the size and weight limitations of low-Earth orbit small satellites make their successful operation rest on a fine balance between solar power infeed and the power demands of the mission payload and supporting platform technologies, buffered by on-board battery storage. At the same time, these satellites are being rolled out as part of ever-larger constellations and mega-constellations. Altogether, this induces a number of challenging computational problems related to the recurring need to make decisions about which task each satellite is to effectuate next. Against this background, GomSpace and Saarland University have joined forces to develop highly sophisticated software-based automated solutions rooted in optimal algorithmic and self-improving learning techniques, all this validated in modern nanosatellite networked missions operating in orbit.
Comparing Machine Learning Techniques for Alfalfa Biomass Yield Prediction
Venue: arXiv cs.LG 2022
Year: 2022

The alfalfa crop is globally important as livestock feed, so highly efficient planting and harvesting could benefit many industries, especially as the global climate changes and traditional methods become less accurate. Recent work using machine learning (ML) to predict yields for alfalfa and other crops has shown promise. Previous efforts used remote sensing, weather, planting, and soil data to train machine learning models for yield prediction. However, while remote sensing works well, the models require large amounts of data and cannot make predictions until the harvesting season begins. Using weather and planting data from alfalfa variety trials in Kentucky and Georgia, our previous work compared feature selection techniques to find the best technique and best feature set. In this work, we trained a variety of machine learning models, using cross validation for hyperparameter optimization, to predict biomass yields, and we showed better accuracy than similar work that employed more complex techniques. Our best individual model was a random forest with a mean absolute error of 0.081 tons/acre and R{$^2$} of 0.941. Next, we expanded this dataset to include Wisconsin and Mississippi, and we repeated our experiments, obtaining a higher best R{$^2$} of 0.982 with a regression tree. We then isolated our testing datasets by state to explore this problem's eligibility for domain adaptation (DA), as we trained on multiple source states and tested on one target state. This Trivial DA (TDA) approach leaves plenty of room for improvement through exploring more complex DA techniques in forthcoming work.
Application of Explainable Machine Learning in Detecting and Classifying Ransomware Families Based on API Call Analysis
Venue: arXiv cs.CR 2022
Year: 2022

Ransomware has appeared as one of the major global threats in recent days. The alarming increasing rate of ransomware attacks and new ransomware variants intrigue the researchers to constantly examine the distinguishing traits of ransomware and refine their detection strategies. Application Programming Interface (API) is a way for one program to collaborate with another; API calls are the medium by which they communicate. Ransomware uses this strategy to interact with the OS and makes a significantly higher number of calls in different sequences to ask for taking action. This research work utilizes the frequencies of different API calls to detect and classify ransomware families. First, a Web-Crawler is developed to automate collecting the Windows Portable Executable (PE) files of 15 different ransomware families. By extracting different frequencies of 68 API calls, we develop our dataset in the first phase of the two-phase feature engineering process. After selecting the most significant features in the second phase of the feature engineering process, we deploy six Supervised Machine Learning models: Na"ive Bayes, Logistic Regression, Random Forest, Stochastic Gradient Descent, K-Nearest Neighbor, and Support Vector Machine. Then, the performances of all the classifiers are compared to select the best model. The results reveal that Logistic Regression can efficiently classify ransomware into their corresponding families securing 99.15% overall accuracy. Finally, instead of relying on the 'Black box' characteristic of the Machine Learning models, we present the post-hoc analysis of our best-performing model using 'SHapley Additive exPlanations' or SHAP values to ascertain the transparency and trustworthiness of the model's prediction.
A note on diffusion limits for stochastic gradient descent
Venue: arXiv cs.LG 2022
Year: 2022

In the machine learning literature stochastic gradient descent has recently been widely discussed for its purported implicit regularization properties. Much of the theory, that attempts to clarify the role of noise in stochastic gradient algorithms, has widely approximated stochastic gradient descent by a stochastic differential equation with Gaussian noise. We provide a novel rigorous theoretical justification for this practice that showcases how the Gaussianity of the noise arises naturally.
Low-Latency Hybrid NOMA-TDMA: QoS-Driven Design Framework
Venue: arXiv cs.IT 2022
Year: 2022

Enabling ultra-reliable and low-latency communication services while providing massive connectivity is one of the major goals to be accomplished in future wireless communication networks. In this paper, we investigate the performance of a hybrid multi-access scheme in the finite blocklength (FBL) regime that combines the advantages of both non-orthogonal multiple access (NOMA) and time-division multiple access (TDMA) schemes. Two latency-sensitive application scenarios are studied, distinguished by whether the queuing behaviour has an influence on the transmission performance or not. In particular, for the latency-critical case with one-shot transmission, we aim at a certain physical-layer quality-of-service (QoS) performance, namely the optimization of the reliability. And for the case in which queuing behaviour plays a role, we focus on the link-layer QoS performance and provide a design that maximizes the effective capacity. For both designs, we leverage the characterizations in the FBL regime to provide the optimal framework by jointly allocating the blocklength and transmit power of each user. In particular, for the reliability-oriented design, the original problem is decomposed and the joint convexity of sub-problems is shown via a variable substitution method. For the effective-capacity-oriented design, we exploit the method of Lagrange multipliers to formulate a solvable dual problem with strong duality to the original problem. Via simulations, we validate our analytical results of convexity/concavity and show the advantage of our proposed approaches compared to other existing schemes.
Uncovering Fingerprinting Networks. An Analysis of In-Browser Tracking using a Behavior-based Approach
Venue: arXiv cs.CR 2022
Year: 2022

Throughout recent years, the importance of internet-privacy has continuously risen. [...] Browser fingerprinting is a technique that does not require cookies or persistent identifiers. It derives a sufficiently unique identifier from the various browser or device properties. Academic work has covered offensive and defensive fingerprinting methods for almost a decade, observing a rise in popularity. This thesis explores the current state of browser fingerprinting on the internet. For that, we implement FPNET - a scalable & reliable tool based on FPMON, to identify fingerprinting scripts on large sets of websites by observing their behavior. By scanning the Alexa Top 10,000 websites, we spot several hundred networks of equally behaving scripts. For each network, we determine the actor behind it. We track down companies like Google, Yandex, Maxmind, Sift, or FingerprintJS, to name a few. In three complementary studies, we further investigate the uncovered networks with regards to I) randomization of filenames or domains, II) behavior changes, III) security. Two consecutive scans reveal that only less than 12.5% of the pages do not change script files. With our behavior-based approach, we successfully re-identify almost 9,000 scripts whose filename or domain changed, and over 86% of the scripts without URL changes. The security analysis shows an adoption of TLS/SSL to over 98% and specific web security headers set for over 30% of the scripts. Finally, we voice concerns about the unavoidability of modern fingerprinting and its implications for internet users' privacy since we believe that many users are unaware of being fingerprinted or have insufficient possibilities to protect against it.
Hypernetworks in Meta-Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

Training a reinforcement learning (RL) agent on a real-world robotics task remains generally impractical due to sample inefficiency. Multi-task RL and meta-RL aim to improve sample efficiency by generalizing over a distribution of related tasks. However, doing so is difficult in practice: In multi-task RL, state of the art methods often fail to outperform a degenerate solution that simply learns each task separately. Hypernetworks are a promising path forward since they replicate the separate policies of the degenerate solution while also allowing for generalization across tasks, and are applicable to meta-RL. However, evidence from supervised learning suggests hypernetwork performance is highly sensitive to the initialization. In this paper, we 1) show that hypernetwork initialization is also a critical factor in meta-RL, and that naive initializations yield poor performance; 2) propose a novel hypernetwork initialization scheme that matches or exceeds the performance of a state-of-the-art approach proposed for supervised settings, as well as being simpler and more general; and 3) use this method to show that hypernetworks can improve performance in meta-RL by evaluating on multiple simulated robotics benchmarks.
Meeting Decision Tracker: Making Meeting Minutes with De-Contextualized Utterances
Venue: arXiv cs.CL 2022
Year: 2022

Meetings are a universal process to make decisions in business and project collaboration. The capability to automatically itemize the decisions in daily meetings allows for extensive tracking of past discussions. To that end, we developed Meeting Decision Tracker, a prototype system to construct decision items comprising decision utterance detector (DUD) and decision utterance rewriter (DUR). We show that DUR makes a sizable contribution to improving the user experience by dealing with utterance collapse in natural conversation. An introduction video of our system is also available at https://youtu.be/TG1pJJo0Iqo.
Solving Reasoning Tasks with a Slot Transformer
Venue: arXiv cs.LG 2022
Year: 2022

The ability to carve the world into useful abstractions in order to reason about time and space is a crucial component of intelligence. In order to successfully perceive and act effectively using senses we must parse and compress large amounts of information for further downstream reasoning to take place, allowing increasingly complex concepts to emerge. If there is any hope to scale representation learning methods to work with real world scenes and temporal dynamics then there must be a way to learn accurate, concise, and composable abstractions across time. We present the Slot Transformer, an architecture that leverages slot attention, transformers and iterative variational inference on video scene data to infer such representations. We evaluate the Slot Transformer on CLEVRER, Kinetics-600 and CATER datesets and demonstrate that the approach allows us to develop robust modeling and reasoning around complex behaviours as well as scores on these datasets that compare favourably to existing baselines. Finally we evaluate the effectiveness of key components of the architecture, the model's representational capacity and its ability to predict from incomplete input.
Learning Rationalizable Equilibria in Multiplayer Games
Venue: arXiv cs.LG 2022
Year: 2022

A natural goal in multiagent learning besides finding equilibria is to learn rationalizable behavior, where players learn to avoid iteratively dominated actions. However, even in the basic setting of multiplayer general-sum games, existing algorithms require a number of samples exponential in the number of players to learn rationalizable equilibria under bandit feedback. This paper develops the first line of efficient algorithms for learning rationalizable Coarse Correlated Equilibria (CCE) and Correlated Equilibria (CE) whose sample complexities are polynomial in all problem parameters including the number of players. To achieve this result, we also develop a new efficient algorithm for the simpler task of finding one rationalizable action profile (not necessarily an equilibrium), whose sample complexity substantially improves over the best existing results of Wu et al. (2021). Our algorithms incorporate several novel techniques to guarantee rationalizability and no (swap-)regret simultaneously, including a correlated exploration scheme and adaptive learning rates, which may be of independent interest. We complement our results with a sample complexity lower bound showing the sharpness of our guarantees.
Self-Supervised Learning with Masked Image Modeling for Teeth Numbering, Detection of Dental Restorations, and Instance Segmentation in Dental Panoramic Radiographs
Venue: arXiv cs.CV 2022
Year: 2022

The computer-assisted radiologic informative report is currently emerging in dental practice to facilitate dental care and reduce time consumption in manual panoramic radiographic interpretation. However, the amount of dental radiographs for training is very limited, particularly from the point of view of deep learning. This study aims to utilize recent self-supervised learning methods like SimMIM and UM-MAE to increase the model efficiency and understanding of the limited number of dental radiographs. We use the Swin Transformer for teeth numbering, detection of dental restorations, and instance segmentation tasks. To the best of our knowledge, this is the first study that applied self-supervised learning methods to Swin Transformer on dental panoramic radiographs. Our results show that the SimMIM method obtained the highest performance of 90.4% and 88.9% on detecting teeth and dental restorations and instance segmentation, respectively, increasing the average precision by 13.4 and 12.8 over the random initialization baseline. Moreover, we augment and correct the existing dataset of panoramic radiographs. The code and the dataset are available at https://github.com/AmaniHAlmalki/DentalMIM.