OpenRaiser/Intern-Atlas huggingface dataset:
A Generic Image Retrieval Method for Date Estimation of Historical Document Collections
Venue: arXiv cs.CV 2022
Year: 2022

Date estimation of historical document images is a challenging problem, with several contributions in the literature that lack of the ability to generalize from one dataset to others. This paper presents a robust date estimation system based in a retrieval approach that generalizes well in front of heterogeneous collections. we use a ranking loss function named smooth-nDCG to train a Convolutional Neural Network that learns an ordination of documents for each problem. One of the main usages of the presented approach is as a tool for historical contextual retrieval. It means that scholars could perform comparative analysis of historical images from big datasets in terms of the period where they were produced. We provide experimental evaluation on different types of documents from real datasets of manuscript and newspaper images.
Gone Quishing: A Field Study of Phishing with Malicious QR Codes
Venue: arXiv cs.CR 2022
Year: 2022

The COVID-19 pandemic enabled "quishing", or phishing with malicious QR codes, as they became a convenient go-between for sharing URLs, including malicious ones. To explore the quishing phenomenon, we conducted a 173-participant study where we used a COVID-19 digital passport sign-up trial with a malicious QR code as a pretext. We found that 67 % of the participants were happy to sign-up with their Google or Facebook credentials, 18.5% to create a new account, and only 14.5% to skip on the sign-up. Convenience was the single most cited factor for the willingness to yield participants' credentials. Reluctance of linking personal accounts with new services was the reason for creating a new account or skipping the registration. We also developed a Quishing Awareness Scale (QAS) and found a significant relationship between participants' QR code behavior and their sign-up choices: the ones choosing to sign-up with Facebook scored the lowest while the one choosing to skip the highest on average. We used our results to propose quishing awareness training guidelines and develop and test usable security indicators for warning users about the threat of quishing.
Single-level Adversarial Data Synthesis based on Neural Tangent Kernels
Venue: arXiv cs.LG 2022
Year: 2022

Abstract Generative adversarial networks (GANs) have achieved impressive performance in data synthesis and have driven the development of many applications. However, GANs are known to be hard to train due to their bilevel objective, which leads to the problems of convergence, mode collapse, and gradient vanishing. In this paper, we propose a new generative model called the generative adversarial NTK (GA-NTK) that has a single-level objective. The GA-NTK keeps the spirit of adversarial learning (which helps generate plausible data) while avoiding the training difficulties of GANs. This is done by modeling the discriminator as a Gaussian process with a neural tangent kernel (NTK-GP) whose training dynamics can be completely described by a closed-form formula. We analyze the convergence behavior of GA-NTK trained by gradient descent and give some sufficient conditions for convergence. We also conduct extensive experiments to study the advantages and limitations of GA-NTK and propose some techniques that make GA-NTK more practical.
Individually-Fair Auctions for Multi-Slot Sponsored Search
Venue: arXiv cs.GT 2022
Year: 2022

We design fair sponsored search auctions that achieve a near-optimal tradeoff between fairness and quality. Our work builds upon the model and auction design of Chawla and Jagadeesan \cite{CJ22}, who considered the special case of a single slot. We consider sponsored search settings with multiple slots and the standard model of click through rates that are multiplicatively separable into an advertiser-specific component and a slot-specific component. When similar users have similar advertiser-specific click through rates, our auctions achieve the same near-optimal tradeoff between fairness and quality as in \cite{CJ22}. When similar users can have different advertiser-specific preferences, we show that a preference-based fairness guarantee holds. Finally, we provide a computationally efficient algorithm for computing payments for our auctions as well as those in previous work, resolving another open direction from \cite{CJ22}.
A Video Anomaly Detection Framework based on Appearance-Motion Semantics Representation Consistency
Venue: arXiv cs.CV 2022
Year: 2022

Video anomaly detection refers to the identification of events that deviate from the expected behavior. Due to the lack of anomalous samples in training, video anomaly detection becomes a very challenging task. Existing methods almost follow a reconstruction or future frame prediction mode. However, these methods ignore the consistency between appearance and motion information of samples, which limits their anomaly detection performance. Anomalies only occur in the moving foreground of surveillance videos, so the semantics expressed by video frame sequences and optical flow without background information in anomaly detection should be highly consistent and significant for anomaly detection. Based on this idea, we propose Appearance-Motion Semantics Representation Consistency (AMSRC), a framework that uses normal data's appearance and motion semantic representation consistency to handle anomaly detection. Firstly, we design a two-stream encoder to encode the appearance and motion information representations of normal samples and introduce constraints to further enhance the consistency of the feature semantics between appearance and motion information of normal samples so that abnormal samples with low consistency appearance and motion feature representation can be identified. Moreover, the lower consistency of appearance and motion features of anomalous samples can be used to generate predicted frames with larger reconstruction error, which makes anomalies easier to spot. Experimental results demonstrate the effectiveness of the proposed method.
Uncertain Case Identifiers in Process Mining: A User Study of the Event-Case Correlation Problem on Click Data
Venue: arXiv cs.DB 2022
Year: 2022

Among the many sources of event data available today, a prominent one is user interaction data. User activity may be recorded during the use of an application or website, resulting in a type of user interaction data often called click data. An obstacle to the analysis of click data using process mining is the lack of a case identifier in the data. In this paper, we show a case and user study for event-case correlation on click data, in the context of user interaction events from a mobility sharing company. To reconstruct the case notion of the process, we apply a novel method to aggregate user interaction data in separate user sessions-interpreted as cases-based on neural networks. To validate our findings, we qualitatively discuss the impact of process mining analyses on the resulting well-formed event log through interviews with process experts.
Internet of Things Protection and Encryption: A Survey
Venue: arXiv cs.CR 2022
Year: 2022

The Internet of Things (IoT) has enabled a wide range of sectors to interact effectively with their consumers in order to deliver seamless services and products. Despite the widespread availability of (IoT) devices and their Internet connectivity, they have a low level of information security integrity. A number of security methods were proposed and evaluated in our research, and comparisons were made in terms of energy and time in the encryption and decryption processes. A ratification procedure is also performed on the devices in the main manager, which is regarded as a full firewall for IoT devices. The suggested algorithm's success has been shown utilizing low-cost Adriano Uno and Raspberry Pi devices. Arduous Uno has been used to demonstrate the encryption process in low-energy devices using a variety of algorithms, including Enhanced Algorithm for Data Integrity and Authentication (EDAI) and raspberry, which serves as a safety manager in low-energy device molecules. A variety of enhanced algorithms used in conjunction with Blockchain software have also assured the security and integrity of the information. These findings and discussions are presented at the conclusion of the paper.
CookieEnforcer: Automated Cookie Notice Analysis and Enforcement
Venue: arXiv cs.CR 2022
Year: 2022

Online websites use cookie notices to elicit consent from the users, as required by recent privacy regulations like the GDPR and the CCPA. Prior work has shown that these notices use dark patterns to manipulate users into making website-friendly choices which put users' privacy at risk. In this work, we develop CookieEnforcer, a new system for automatically discovering cookie notices and deciding on the options that result in disabling all non-essential cookies. In order to achieve this, we first build an automatic cookie notice detector that utilizes the rendering pattern of the HTML elements to identify the cookie notices. Next, CookieEnforcer analyzes the cookie notices and predicts the set of actions required to disable all unnecessary cookies. This is done by modeling the problem as a sequence-to-sequence task, where the input is a machine-readable cookie notice and the output is the set of clicks to make. We demonstrate the efficacy of CookieEnforcer via an end-to-end accuracy evaluation, showing that it can generate the required steps in 91% of the cases. Via a user study, we show that CookieEnforcer can significantly reduce the user effort. Finally, we use our system to perform several measurements on the top 5k websites from the Tranco list (as accessed from the US and the UK), drawing comparisons and observations at scale.
How does online teamwork change student communication patterns in programming courses?
Venue: arXiv cs.CY 2022
Year: 2022

Online teaching has become a new reality due to the COVID-19 pandemic raising a lot of questions about its learning outcomes. Recent studies have shown that peer communication positively affects learning outcomes of online teaching. However, it is not clear how collaborative programming tasks change peer communication patterns in the learning process. In this study, we compare communication patterns in MOOCs where peer communication is limited with those of a blended course in which students are involved in online peer instruction. We used a mixed-method approach comprising automated text analysis and community extraction with further qualitative analysis. The results show that students prefer to seek help in programming from peers and not the teacher. Team assignment helped to support this habit. Students communicated more positively and intensively with each other, while only team leaders communicated with the instructor reducing teacher overload. This shift could explain how peer communication improves learning outcomes, as has been shown in previous studies on MOOCs.
Towards Cognitive Robots That People Accept in Their Home
Venue: arXiv cs.RO 2022
Year: 2022

It is intractable for assistive robots to have all functionalities pre-programmed prior to deployment. Rather, it is more realistic for robots to perform supplemental, on-site learning about user's needs and preferences, and particularities of the environment. This additional learning is especially helpful for care robots that assist with individualized caregiver activities in residential or assisted living facilities. As each care receiver has unique needs and those needs may change over time, robots require the ability to adapt and learn on-site. In this work, we propose the study design to investigate the impacts on end-users of observing robot learning. We will assess user attitudes towards robots that conduct some learning in the home as compared to a baseline condition where the robot is delivered fully capable. We will additionally compare different modes of learning to determine whether some are more likely to instill trust.
On Improving Cross-dataset Generalization of Deepfake Detectors
Venue: arXiv cs.CV 2022
Year: 2022

Facial manipulation by deep fake has caused major security risks and raised severe societal concerns. As a countermeasure, a number of deep fake detection methods have been proposed recently. Most of them model deep fake detection as a binary classification problem using a backbone convolutional neural network (CNN) architecture pretrained for the task. These CNN-based methods have demonstrated very high efficacy in deep fake detection with the Area under the Curve (AUC) as high as 0.99. However, the performance of these methods degrades significantly when evaluated across datasets. In this paper, we formulate deep fake detection as a hybrid combination of supervised and reinforcement learning (RL) to improve its cross-dataset generalization performance. The proposed method chooses the top-k augmentations for each test sample by an RL agent in an image-specific manner. The classification scores, obtained using CNN, of all the augmentations of each test image are averaged together for final real or fake classification. Through extensive experimental validation, we demonstrate the superiority of our method over existing published research in cross-dataset generalization of deep fake detectors, thus obtaining state-of-the-art performance.
Machine learning model to predict solar radiation, based on the integration of meteorological data and data obtained from satellite images
Venue: arXiv cs.LG 2022
Year: 2022

Knowing the behavior of solar radiation at a geographic location is essential for the use of energy from the sun using photovoltaic systems; however, the number of stations for measuring meteorological parameters and for determining the size of solar fields in remote areas is limited. In this work, images obtained from the GOES-13 satellite were used, from which variables were extracted that could be integrated into datasets from meteorological stations. From this, 3 different models were built, on which the performance of 5 machine learning algorithms in predicting solar radiation was evaluated. The neural networks had the highest performance in the model that integrated the meteorological variables and the variables obtained from the images, according to an analysis carried out using four evaluation metrics; although if the rRMSE is considered, all results obtained were higher than 20%, which classified the performance of the algorithms as fair. In the 2012 dataset, the estimation results according to the metrics MBE, R2, RMSE, and rRMSE corresponded to -0.051, 0.880, 90.99 and 26.7%, respectively. In the 2017 dataset, the results of MBE, R2, RMSE, and rRMSE were -0.146, 0.917, 40.97 and 22.3%, respectively. Although it is possible to calculate solar radiation from satellite images, it is also true that some statistical methods depend on radiation data and sunshine captured by ground-based instruments, which is not always possible given that the number of measurement stations on the surface is limited.
Towards Understanding Barriers and Mitigation Strategies of Software Engineers with Non-traditional Educational and Occupational Backgrounds
Venue: arXiv cs.SE 2022
Year: 2022

The traditional path to a software engineering career involves a post-secondary diploma in Software Engineering, Computer Science, or a related field. However, many software engineers take a non-traditional path to their career, starting from other industries or fields of study. This paper proposes a study on barriers faced by software engineers with non-traditional educational and occupational backgrounds, and possible mitigation strategies for those barriers. We propose a two-stage methodology, consisting of an exploratory study, followed by a validation study. The exploratory study will involve a grounded-theory-based qualitative analysis of relevant Reddit data to yield a framework around the barriers and possible mitigation strategies. These findings will then be validated using a survey in the validation study. Making software engineering more accessible to those with non-traditional backgrounds will not only bring about the benefits of functional diversity, but also serves as a method of filling in the labour shortages of the software engineering industry.
An Adaptive Black-box Backdoor Detection Method for Deep Neural Networks
Venue: arXiv cs.CR 2022
Year: 2022

With the surge of Machine Learning (ML), An emerging amount of intelligent applications have been developed. Deep Neural Networks (DNNs) have demonstrated unprecedented performance across various fields such as medical diagnosis and autonomous driving. While DNNs are widely employed in security-sensitive fields, they are identified to be vulnerable to Neural Trojan (NT) attacks that are controlled and activated by stealthy triggers. In this paper, we target to design a robust and adaptive Trojan detection scheme that inspects whether a pre-trained model has been Trojaned before its deployment. Prior works are oblivious of the intrinsic property of trigger distribution and try to reconstruct the trigger pattern using simple heuristics, i.e., stimulating the given model to incorrect outputs. As a result, their detection time and effectiveness are limited. We leverage the observation that the pixel trigger typically features spatial dependency and propose the first trigger approximation based black-box Trojan detection framework that enables a fast and scalable search of the trigger in the input space. Furthermore, our approach can also detect Trojans embedded in the feature space where certain filter transformations are used to activate the Trojan. We perform extensive experiments to investigate the performance of our approach across various datasets and ML models. Empirical results show that our approach achieves a ROC-AUC score of 0.93 on the public TrojAI dataset. Our code can be found at https://github.com/xinqiaozhang/adatrojan
A Little Too Personal: Effects of Standardization versus Personalization on Job Acquisition, Work Completion, and Revenue for Online Freelancers
Venue: arXiv cs.HC 2022
Year: 2022

As more individuals consider permanently working from home, the online labor market continues to grow as an alternative working environment. While the flexibility and autonomy of these online gigs attracts many workers, success depends critically upon self-management and workers' efficient allocation of scarce resources. To achieve this, freelancers may develop alternative work strategies, employing highly standardized schedules and communication patterns while taking on large work volumes, or engaging in smaller numbers of jobs whilst tailoring their activities to build relationships with individual employers. In this study, we consider this contrast in relation to worker communication patterns. We demonstrate the heterogeneous effects of standardization versus personalization across different stages of a project and examine the relative impact on job acquisition, project completion, and earnings. Our findings can inform the design of platforms and various worker support tools for the gig economy.
Deep neural network goes lighter: A case study of deep compression techniques on automatic RF modulation recognition for Beyond 5G networks
Venue: arXiv cs.LG 2022
Year: 2022

Automatic RF modulation recognition is a primary signal intelligence (SIGINT) technique that serves as a physical layer authentication enabler and automated signal processing scheme for the beyond 5G and military networks. Most existing works rely on adopting deep neural network architectures to enable RF modulation recognition. The application of deep compression for the wireless domain, especially automatic RF modulation classification, is still in its infancy. Lightweight neural networks are key to sustain edge computation capability on resource-constrained platforms. In this letter, we provide an in-depth view of the state-of-the-art deep compression and acceleration techniques with an emphasis on edge deployment for beyond 5G networks. Finally, we present an extensive analysis of the representative acceleration approaches as a case study on automatic radar modulation classification and evaluate them in terms of the computational metrics.
Generalised Mathematical Formulations for Non-Linear Optimized Scheduling
Venue: arXiv cs.NI 2022
Year: 2022

In practice, most of the optimization problems are non-linear requiring certain interactive solutions and approaches to model. In 5G Advanced and Beyond network slicing, mathematically modeling the users, type of service distributions and its adaptive SLAs are complex due to several dependencies. To facilitate the above, in this paper, we present novel Non-linear mathematical formulations and results that will form the base to achieve Optimized Scheduling.
Crime Patterns in Los Angeles County Before and After Covid19 (2018-2021)
Venue: arXiv cs.DC 2022
Year: 2022

The objective of our research is to present the change in crime rates in Los Angeles post-Covid19. Using data analysis with Geo-Mapping, bubbles, Marimekko, and a time series charts, we can illustrate which areas have the largest crime rate, and how it has changed. Through regression modeling, we can interpret which locations may also have a correlation to crime versus income, race, type of crime, and gender. The story will help to uncover whether the areas associated with crime are due to demographic or income variance. In showing the details of crimes in Los Angeles along with the factors at play we hope to see a compelling relationship between crime rates and recent events from 2020 to the present, along with changes in crime type trends during these periods. We use Excel to clean the data for SAP SAC to model effectively, as well as resources from other studies a comparison.
Improve Generalization of Driving Policy at Signalized Intersections with Adversarial Learning
Venue: arXiv cs.RO 2022
Year: 2022

Intersections are quite challenging among various driving scenes wherein the interaction of signal lights and distinct traffic actors poses great difficulty to learn a wise and robust driving policy. Current research rarely considers the diversity of intersections and stochastic behaviors of traffic participants. For practical applications, the randomness usually leads to some devastating events, which should be the focus of autonomous driving. This paper introduces an adversarial learning paradigm to boost the intelligence and robustness of driving policy for signalized intersections with dense traffic flow. Firstly, we design a static path planner which is capable of generating trackable candidate paths for multiple intersections with diversified topology. Next, a constrained optimal control problem (COCP) is built based on these candidate paths wherein the bounded uncertainty of dynamic models is considered to capture the randomness of driving environment. We propose adversarial policy gradient (APG) to solve the COCP wherein the adversarial policy is introduced to provide disturbances by seeking the most severe uncertainty while the driving policy learns to handle this situation by competition. Finally, a comprehensive system is established to conduct training and testing wherein the perception module is introduced and the human experience is incorporated to solve the yellow light dilemma. Experiments indicate that the trained policy can handle the signal lights flexibly meanwhile realizing the smooth and efficient passing with a humanoid paradigm. Besides, APG enables a large-margin improvement of the resistance to the abnormal behaviors and thus ensures a high safety level for the autonomous vehicle.