OpenRaiser/Intern-Atlas huggingface dataset:
Hide me Behind the Noise: Local Differential Privacy for Indoor Location Privacy
Venue: arXiv cs.CR 2022
Year: 2022

The advent of numerous indoor location-based services (LBSs) and the widespread use of many types of mobile devices in indoor environments have resulted in generating a massive amount of people's location data. While geo-spatial data contains sensitive information about personal activities, collecting it in its raw form may lead to the leak of personal information relating to the people, violating their privacy. This paper proposes a novel privacy-aware framework for aggregating the indoor location data employing the Local Differential Privacy (LDP) technique, in which the user location data is changed locally in the user's device and is sent to the aggregator afterward. Therefore, the users' locations are kept hidden from a server or any attackers. The practical feasibility of applying the proposed framework is verified by two real-world datasets. The impact of dataset properties, the privacy mechanisms, and the privacy level on our framework are also investigated. The experimental results indicate that the presented framework can protect the location information of users, and the accuracy of the population frequency of different zones in the indoor area is close to that of the original population frequency with no knowledge about the location of people indoors.
Human-Assisted Robotic Detection of Foreign Object Debris Inside Confined Spaces of Marine Vessels Using Probabilistic Mapping
Venue: arXiv cs.RO 2022
Year: 2022

Many complex vehicular systems, such as large marine vessels, contain confined spaces like water tanks, which are critical for the safe functioning of the vehicles. It is particularly hazardous for humans to inspect such spaces due to limited accessibility, poor visibility, and unstructured configuration. While robots provide a viable alternative, they encounter the same set of challenges in realizing robust autonomy. In this work, we specifically address the problem of detecting foreign object debris (FODs) left inside the confined spaces using a visual mapping-based system that relies on Mahalanobis distance-driven comparisons between the nominal and online maps for local outlier identification. Simulation trials show extremely high recall but low precision for the outlier identification method. The assistance of remote humans is, therefore, taken to deal with the precision problem by going over the close-up robot camera images of the outlier regions. An online survey is conducted to show the usefulness of this assistance process. Physical experiments are also reported on a GPU-enabled mobile robot platform inside a scaled-down, prototype tank to demonstrate the feasibility of the FOD detection system.
Contrastive Cross-Modal Knowledge Sharing Pre-training for Vision-Language Representation Learning and Retrieval
Venue: arXiv cs.CV 2022
Year: 2022

Recently, the cross-modal pre-training task has been a hotspot because of its wide application in various down-streaming researches including retrieval, captioning, question answering and so on. However, exiting methods adopt a one-stream pre-training model to explore the united vision-language representation for conducting cross-modal retrieval, which easily suffer from the calculation explosion. Moreover, although the conventional double-stream structures are quite efficient, they still lack the vital cross-modal interactions, resulting in low performances. Motivated by these challenges, we put forward a Contrastive Cross-Modal Knowledge Sharing Pre-training (COOKIE) to grasp the joint text-image representations. Structurally, COOKIE adopts the traditional double-stream structure because of the acceptable time consumption. To overcome the inherent defects of double-stream structure as mentioned above, we elaborately design two effective modules. Concretely, the first module is a weight-sharing transformer that builds on the head of the visual and textual encoders, aiming to semantically align text and image. This design enables visual and textual paths focus on the same semantics. The other one is three specially designed contrastive learning, aiming to share knowledge between different models. The shared cross-modal knowledge develops the study of unimodal representation greatly, promoting the single-modal retrieval tasks. Extensive experimental results on multi-modal matching researches that includes cross-modal retrieval, text matching, and image retrieval reveal the superiors in calculation efficiency and statistical indicators of our pre-training model.
Sum-of-Max Partition under a Knapsack Constraint
Venue: arXiv cs.DS 2022
Year: 2022

Sequence partition problems arise in many fields, such as sequential data analysis, information transmission, and parallel computing. In this paper, we study the following partition problem variant: given a sequence of $n$ items $1,\ldots,n$, where each item $i$ is associated with weight $w_i$ and another parameter $s_i$, partition the sequence into several consecutive subsequences, so that the total weight of each subsequence is no more than a threshold $w_0$, and the sum of the largest $s_i$ in each subsequence is minimized. This problem admits a straightforward solution based on dynamic programming, which costs $O(n^2)$ time and can be improved to $O(n\log n)$ time easily. Our contribution is an $O(n)$ time algorithm, which is nontrivial yet easy to implement. We also study the corresponding tree partition problem. We prove that the problem on the tree is NP-complete and we present an $O(w_0 n^2)$ time ($O(w_0^2n^2)$ time, respectively) algorithm for the unit weight (integer weight, respectively) case.
Enabling Harmonious Human-Machine Interaction with Visual-Context Augmented Dialogue System: A Review
Venue: arXiv cs.AI 2022
Year: 2022

The intelligent dialogue system, aiming at communicating with humans harmoniously with natural language, is brilliant for promoting the advancement of human-machine interaction in the era of artificial intelligence. With the gradually complex human-computer interaction requirements (e.g., multimodal inputs, time sensitivity), it is difficult for traditional text-based dialogue system to meet the demands for more vivid and convenient interaction. Consequently, Visual Context Augmented Dialogue System (VAD), which has the potential to communicate with humans by perceiving and understanding multimodal information (i.e., visual context in images or videos, textual dialogue history), has become a predominant research paradigm. Benefiting from the consistency and complementarity between visual and textual context, VAD possesses the potential to generate engaging and context-aware responses. For depicting the development of VAD, we first characterize the concepts and unique features of VAD, and then present its generic system architecture to illustrate the system workflow. Subsequently, several research challenges and representative works are detailed investigated, followed by the summary of authoritative benchmarks. We conclude this paper by putting forward some open issues and promising research trends for VAD, e.g., the cognitive mechanisms of human-machine dialogue under cross-modal dialogue context, and knowledge-enhanced cross-modal semantic interaction.
An AIoT-enabled Autonomous Dementia Monitoring System
Venue: arXiv cs.LG 2022
Year: 2022

An autonomous Artificial Internet of Things (AIoT) system for elderly dementia patients monitoring in a smart home is presented. The system mainly implements two functions based on the activity inference of the sensor data, which are real time abnormal activity monitoring and trend prediction of disease related activities. Specifically, CASAS dataset is employed to train a Random Forest (RF) model for activity inference. Then, another RF model trained by the output data of activity inference is used for abnormal activity monitoring. Particularly, RF is chosen for these tasks because of its balanced trade offs between accuracy, time efficiency, flexibility, and interpretability. Moreover, Long Short Term Memory (LSTM) is utilised to forecast the disease related activity trend of a patient. Consequently, the accuracy of two RF classifiers designed for activity inference and abnormal activity detection is greater than 99 percent and 94 percent, respectively. Furthermore, using the duration of sleep as an example, the LSTM model achieves accurate and evident future trends prediction.
Geometric Learning of Hidden Markov Models via a Method of Moments Algorithm
Venue: arXiv cs.LG 2022
Year: 2022

We present a novel algorithm for learning the parameters of hidden Markov models (HMMs) in a geometric setting where the observations take values in Riemannian manifolds. In particular, we elevate a recent second-order method of moments algorithm that incorporates non-consecutive correlations to a more general setting where observations take place in a Riemannian symmetric space of non-positive curvature and the observation likelihoods are Riemannian Gaussians. The resulting algorithm decouples into a Riemannian Gaussian mixture model estimation algorithm followed by a sequence of convex optimization procedures. We demonstrate through examples that the learner can result in significantly improved speed and numerical accuracy compared to existing learners.
Lane-GNN: Integrating GNN for Predicting Drivers' Lane Change Intention
Venue: arXiv cs.LG 2022
Year: 2022

Nowadays, intelligent highway traffic network is playing an important role in modern transportation infrastructures. A variable speed limit (VSL) system can be facilitated in the highway traffic network to provide useful and dynamic speed limit information for drivers to travel with enhanced safety. Such system is usually designed with a steady advisory speed in mind so that traffic can move smoothly when drivers follow the speed, rather than speeding up whenever there is a gap and slowing down at congestion. However, little attention has been given to the research of vehicles' behaviours when drivers left the road network governed by a VSL system, which may largely involve unexpected acceleration, deceleration and frequent lane changes, resulting in chaos for the subsequent highway road users. In this paper, we focus on the detection of traffic flow anomaly due to drivers' lane change intention on the highway traffic networks after a VSL system. More specifically, we apply graph modelling on the traffic flow data generated by a popular mobility simulator, SUMO, at road segment levels. We then evaluate the performance of lane changing detection using the proposed Lane-GNN scheme, an attention temporal graph convolutional neural network, and compare its performance with a temporal convolutional neural network (TCNN) as our baseline. Our experimental results show that the proposed Lane-GNN can detect drivers' lane change intention within 90 seconds with an accuracy of 99.42% under certain assumptions. Finally, some interpretation methods are applied to the trained models with a view to further illustrate our findings.
Firenze: Model Evaluation Using Weak Signals
Venue: arXiv cs.CR 2022
Year: 2022

Data labels in the security field are frequently noisy, limited, or biased towards a subset of the population. As a result, commonplace evaluation methods such as accuracy, precision and recall metrics, or analysis of performance curves computed from labeled datasets do not provide sufficient confidence in the real-world performance of a machine learning (ML) model. This has slowed the adoption of machine learning in the field. In the industry today, we rely on domain expertise and lengthy manual evaluation to build this confidence before shipping a new model for security applications. In this paper, we introduce Firenze, a novel framework for comparative evaluation of ML models' performance using domain expertise, encoded into scalable functions called markers. We show that markers computed and combined over select subsets of samples called regions of interest can provide a robust estimate of their real-world performances. Critically, we use statistical hypothesis testing to ensure that observed differences-and therefore conclusions emerging from our framework-are more prominent than that observable from the noise alone. Using simulations and two real-world datasets for malware and domain-name-service reputation detection, we illustrate our approach's effectiveness, limitations, and insights. Taken together, we propose Firenze as a resource for fast, interpretable, and collaborative model development and evaluation by mixed teams of researchers, domain experts, and business owners.
Balanced reconstruction codes for single edits
Venue: arXiv cs.IT 2022
Year: 2022

Motivated by the sequence reconstruction problem initiated by Levenshtein, reconstruction codes were introduced by Cai \emph{et al}. to combat errors when a fixed number of noisy channels are available. The central problem on this topic is to design codes with sizes as large as possible, such that every codeword can be uniquely reconstructed from any $N$ distinct noisy reads, where $N$ is fixed. In this paper, we study binary reconstruction codes with the constraint that every codeword is balanced, which is a common requirement in the technique of DNA-based storage. For all possible channels with a single edit error and their variants, we design asymptotically optimal balanced reconstruction codes for all $N$, and show that the number of their redundant symbols decreases from $\frac{3}{2}\log_2 n+O(1)$ to $\frac{1}{2}\log_2n+\log_2\log_2n+O(1)$, and finally to $\frac{1}{2}\log_2n+O(1)$ but with different speeds, where $n$ is the length of the code. Compared with the unbalanced case, our results imply that the balanced property does not reduce the rate of the reconstruction code in the corresponding codebook.
Combinatory Adjoints and Differentiation
Venue: arXiv cs.PL 2022
Year: 2022

We develop a compositional approach for automatic and symbolic differentiation based on categorical constructions in functional analysis where derivatives are linear functions on abstract vectors rather than being limited to scalars, vectors, matrices or tensors represented as multi-dimensional arrays. We show that both symbolic and automatic differentiation can be performed using a differential calculus for generating linear functions representing Fréchet derivatives based on rules for primitive, constant, linear and bilinear functions as well as their sequential and parallel composition. Linear functions are represented in a combinatory domain-specific language. Finally, we provide a calculus for symbolically computing the adjoint of a derivative without using matrices, which are too inefficient to use on high-dimensional spaces. The resulting symbolic representation of a derivative retains the data-parallel operations from the input program. The combination of combinatory differentiation and computing formal adjoints turns out to be behaviorally equivalent to reverse-mode automatic differentiation. In particular, it provides opportunities for optimizations where matrices are too inefficient to represent linear functions.
FL-Defender: Combating Targeted Attacks in Federated Learning
Venue: arXiv cs.LG 2022
Year: 2022

Federated learning (FL) enables learning a global machine learning model from local data distributed among a set of participating workers. This makes it possible i) to train more accurate models due to learning from rich joint training data, and ii) to improve privacy by not sharing the workers' local private data with others. However, the distributed nature of FL makes it vulnerable to targeted poisoning attacks that negatively impact the integrity of the learned model while, unfortunately, being difficult to detect. Existing defenses against those attacks are limited by assumptions on the workers' data distribution, may degrade the global model performance on the main task and/or are ill-suited to high-dimensional models. In this paper, we analyze targeted attacks against FL and find that the neurons in the last layer of a deep learning (DL) model that are related to the attacks exhibit a different behavior from the unrelated neurons, making the last-layer gradients valuable features for attack detection. Accordingly, we propose \textit{FL-Defender} as a method to combat FL targeted attacks. It consists of i) engineering more robust discriminative features by calculating the worker-wise angle similarity for the workers' last-layer gradients, ii) compressing the resulting similarity vectors using PCA to reduce redundant information, and iii) re-weighting the workers' updates based on their deviation from the centroid of the compressed similarity vectors. Experiments on three data sets with different DL model sizes and data distributions show the effectiveness of our method at defending against label-flipping and backdoor attacks. Compared to several state-of-the-art defenses, FL-Defender achieves the lowest attack success rates, maintains the performance of the global model on the main task and causes minimal computational overhead on the server.
The Professionalization of the Hacker Industry
Venue: arXiv cs.CY 2022
Year: 2022

Society is inextricably dependent on the Internet and other globally interconnected infrastructures used in the provisioning of information services. The growth of information technology (IT) and information systems (IS) over the past decades has created an unprecedented demand for access to information. The implication of wireless mobility are great, and the commercial possibilities of new and innovative wireless flexibility are just beginning to be realized through the emergence of the Internet of Things (IoT). This article takes a look the history of hacking and professionalization of the hacker industry. As the hacker industry becomes more fully professionalized, it is becoming much more adaptive and flexible, making it harder for intelligence and law enforcement to confront. Furthermore, the hacker industry is blurring the distinction between motivated crime and traditional computer security threats - including the disruption of critical infrastructures or the penetration of networks.
VocabulARy: Learning Vocabulary in AR Supported by Keyword Visualisations
Venue: arXiv cs.HC 2022
Year: 2022

Learning vocabulary in a primary or secondary language is enhanced when we encounter words in context. This context can be afforded by the place or activity we are engaged with. Existing learning environments include formal learning, mnemonics, flashcards, use of a dictionary or thesaurus, all leading to practice with new words in context. In this work, we propose an enhancement to the language learning process by providing the user with words and learning tools in context, with VocabulARy. VocabulARy visually annotates objects in AR, in the user's surroundings, with the corresponding English (first language) and Japanese (second language) words to enhance the language learning process. In addition to the written and audio description of each word, we also present the user with a keyword and its visualisation to enhance memory retention. We evaluate our prototype by comparing it to an alternate AR system that does not show an additional visualisation of the keyword, and, also, we compare it to two non-AR systems on a tablet, one with and one without visualising the keyword. Our results indicate that AR outperforms the tablet system regarding immediate recall, mental effort and task completion time. Additionally, the visualisation approach scored significantly higher than showing only the written keyword with respect to immediate and delayed recall and learning efficiency, mental effort and task-completion time.
SKIPP'D: a SKy Images and Photovoltaic Power Generation Dataset for Short-term Solar Forecasting
Venue: arXiv cs.CV 2022
Year: 2022

Large-scale integration of photovoltaics (PV) into electricity grids is challenged by the intermittent nature of solar power. Sky-image-based solar forecasting using deep learning has been recognized as a promising approach to predicting the short-term fluctuations. However, there are few publicly available standardized benchmark datasets for image-based solar forecasting, which limits the comparison of different forecasting models and the exploration of forecasting methods. To fill these gaps, we introduce SKIPP'D -- a SKy Images and Photovoltaic Power Generation Dataset. The dataset contains three years (2017-2019) of quality-controlled down-sampled sky images and PV power generation data that is ready-to-use for short-term solar forecasting using deep learning. In addition, to support the flexibility in research, we provide the high resolution, high frequency sky images and PV power generation data as well as the concurrent sky video footage. We also include a code base containing data processing scripts and baseline model implementations for researchers to reproduce our previous work and accelerate their research in solar forecasting.
The Impact of Partner Expressions on Felt Emotion in the Iterated Prisoner's Dilemma: An Event-level Analysis
Venue: arXiv cs.GT 2022
Year: 2022

Social games like the prisoner's dilemma are often used to develop models of the role of emotion in social decision-making. Here we examine an understudied aspect of emotion in such games: how an individual's feelings are shaped by their partner's expressions. Prior research has tended to focus on other aspects of emotion. Research on felt-emotion has focused on how an individual's feelings shape how they treat their partner, or whether these feelings are authentically expressed. Research on expressed-emotion has focused on how an individual's decisions are shaped by their partner's expressions, without regard for whether these expressions actually evoke feelings. Here, we use computer-generated characters to examine how an individual's moment-to-moment feelings are shaped by (1) how they are treated by their partner and (2) what their partner expresses during this treatment. Surprisingly, we find that partner expressions are far more important than actions in determining self-reported feelings. In other words, our partner can behave in a selfish and exploitive way, but if they show a collaborative pattern of expressions, we will feel greater pleasure collaborating with them. These results also emphasize the importance of context in determining how someone will feel in response to an expression (i.e., knowing a partner is happy is insufficient; we must know what they are happy-at). We discuss the implications of this work for cognitive-system design, emotion theory, and methodological practice in affective computing.