OpenRaiser/Intern-Atlas huggingface dataset:
Gloss Alignment Using Word Embeddings
Venue: arXiv cs.CL 2023
Year: 2023

Capturing and annotating Sign language datasets is a time consuming and costly process. Current datasets are orders of magnitude too small to successfully train unconstrained \acf{slt} models. As a result, research has turned to TV broadcast content as a source of large-scale training data, consisting of both the sign language interpreter and the associated audio subtitle. However, lack of sign language annotation limits the usability of this data and has led to the development of automatic annotation techniques such as sign spotting. These spottings are aligned to the video rather than the subtitle, which often results in a misalignment between the subtitle and spotted signs. In this paper we propose a method for aligning spottings with their corresponding subtitles using large spoken language models. Using a single modality means our method is computationally inexpensive and can be utilized in conjunction with existing alignment techniques. We quantitatively demonstrate the effectiveness of our method on the \acf{mdgs} and \acf{bobsl} datasets, recovering up to a 33.22 BLEU-1 score in word alignment.
MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusion
Venue: arXiv cs.CV 2023
Year: 2023

Reconstructing visual stimuli from brain recordings has been a meaningful and challenging task. Especially, the achievement of precise and controllable image reconstruction bears great significance in propelling the progress and utilization of brain-computer interfaces. Despite the advancements in complex image reconstruction techniques, the challenge persists in achieving a cohesive alignment of both semantic (concepts and objects) and structure (position, orientation, and size) with the image stimuli. To address the aforementioned issue, we propose a two-stage image reconstruction model called MindDiffuser. In Stage 1, the VQ-VAE latent representations and the CLIP text embeddings decoded from fMRI are put into Stable Diffusion, which yields a preliminary image that contains semantic information. In Stage 2, we utilize the CLIP visual feature decoded from fMRI as supervisory information, and continually adjust the two feature vectors decoded in Stage 1 through backpropagation to align the structural information. The results of both qualitative and quantitative analyses demonstrate that our model has surpassed the current state-of-the-art models on Natural Scenes Dataset (NSD). The subsequent experimental findings corroborate the neurobiological plausibility of the model, as evidenced by the interpretability of the multimodal feature employed, which align with the corresponding brain responses.
Novel Area-Efficient and Flexible Architectures for Optimal Ate Pairing on FPGA
Venue: arXiv cs.CR 2023
Year: 2023

While FPGA is a suitable platform for implementing cryptographic algorithms, there are several challenges associated with implementing Optimal Ate pairing on FPGA, such as security, limited computing resources, and high power consumption. To overcome these issues, this study introduces three approaches that can execute the optimal Ate pairing on Barreto-Naehrig curves using Jacobean coordinates with the goal of reaching 128-bit security on the Genesys board. The first approach is a pure software implementation utilizing the MicroBlaze processor. The second involves a combination of software and hardware, with key operations in $F_{p}$ and $F_{p^{2}}$ being transformed into IP cores for the MicroBlaze. The third approach builds on the second by incorporating parallelism to improve the pairing process. The utilization of multiple MicroBlaze processors within a single system offers both versatility and parallelism to speed up pairing calculations. A variety of methods and parameters are used to optimize the pairing computation, including Montgomery modular multiplication, the Karatsuba method, Jacobean coordinates, the Complex squaring method, sparse multiplication, squaring in $G_{ϕ6}F_{p^{12}}$, and the addition chain method. The proposed systems are designed to efficiently utilize limited resources in restricted environments, while still completing tasks in a timely manner.
The Model Inversion Eavesdropping Attack in Semantic Communication Systems
Venue: arXiv cs.IT 2023
Year: 2023

In recent years, semantic communication has been a popular research topic for its superiority in communication efficiency. As semantic communication relies on deep learning to extract meaning from raw messages, it is vulnerable to attacks targeting deep learning models. In this paper, we introduce the model inversion eavesdropping attack (MIEA) to reveal the risk of privacy leaks in the semantic communication system. In MIEA, the attacker first eavesdrops the signal being transmitted by the semantic communication system and then performs model inversion attack to reconstruct the raw message, where both the white-box and black-box settings are considered. Evaluation results show that MIEA can successfully reconstruct the raw message with good quality under different channel conditions. We then propose a defense method based on random permutation and substitution to defend against MIEA in order to achieve secure semantic communication. Our experimental results demonstrate the effectiveness of the proposed defense method in preventing MIEA.
Generative AI in Computing Education: Perspectives of Students and Instructors
Venue: arXiv cs.HC 2023
Year: 2023

Generative models are now capable of producing natural language text that is, in some cases, comparable in quality to the text produced by people. In the computing education context, these models are being used to generate code, code explanations, and programming exercises. The rapid adoption of these models has prompted multiple position papers and workshops which discuss the implications of these models for computing education, both positive and negative. This paper presents results from a series of semi-structured interviews with 12 students and 6 instructors about their awareness, experiences, and preferences regarding the use of tools powered by generative AI in computing classrooms. The results suggest that Generative AI (GAI) tools will play an increasingly significant role in computing education. However, students and instructors also raised numerous concerns about how these models should be integrated to best support the needs and learning goals of students. We also identified interesting tensions and alignments that emerged between how instructors and students prefer to engage with these models. We discuss these results and provide recommendations related to curriculum development, assessment methods, and pedagogical practice. As GAI tools become increasingly prevalent, it's important to understand educational stakeholders' preferences and values to ensure that these tools can be used for good and that potential harms can be mitigated.
All-pairs Consistency Learning for Weakly Supervised Semantic Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

In this work, we propose a new transformer-based regularization to better localize objects for Weakly supervised semantic segmentation (WSSS). In image-level WSSS, Class Activation Map (CAM) is adopted to generate object localization as pseudo segmentation labels. To address the partial activation issue of the CAMs, consistency regularization is employed to maintain activation intensity invariance across various image augmentations. However, such methods ignore pair-wise relations among regions within each CAM, which capture context and should also be invariant across image views. To this end, we propose a new all-pairs consistency regularization (ACR). Given a pair of augmented views, our approach regularizes the activation intensities between a pair of augmented views, while also ensuring that the affinity across regions within each view remains consistent. We adopt vision transformers as the self-attention mechanism naturally embeds pair-wise affinity. This enables us to simply regularize the distance between the attention matrices of augmented image pairs. Additionally, we introduce a novel class-wise localization method that leverages the gradients of the class token. Our method can be seamlessly integrated into existing WSSS methods using transformers without modifying the architectures. We evaluate our method on PASCAL VOC and MS COCO datasets. Our method produces noticeably better class localization maps (67.3% mIoU on PASCAL VOC train), resulting in superior WSSS performances.
On the concentration of the maximum degree in the duplication-divergence models
Venue: arXiv cs.DM 2023
Year: 2023

We present a rigorous and precise analysis of the maximum degree and the average degree in a dynamic duplication-divergence graph model introduced by Solé, Pastor-Satorras et al. in which the graph grows according to a duplication-divergence mechanism, i.e. by iteratively creating a copy of some node and then randomly alternating the neighborhood of a new node with probability $p$. This model captures the growth of some real-world processes e.g. biological or social networks. In this paper, we prove that for some $0 < p < 1$ the maximum degree and the average degree of a duplication-divergence graph on $t$ vertices are asymptotically concentrated with high probability around $t^p$ and $\max\{t^{2 p - 1}, 1\}$, respectively, i.e. they are within at most a polylogarithmic factor from these values with probability at least $1 - t^{-A}$ for any constant $A > 0$.
Learning Unbiased Image Segmentation: A Case Study with Plain Knee Radiographs
Venue: arXiv cs.CV 2023
Year: 2023

Automatic segmentation of knee bony anatomy is essential in orthopedics, and it has been around for several years in both pre-operative and post-operative settings. While deep learning algorithms have demonstrated exceptional performance in medical image analysis, the assessment of fairness and potential biases within these models remains limited. This study aims to revisit deep learning-powered knee-bony anatomy segmentation using plain radiographs to uncover visible gender and racial biases. The current contribution offers the potential to advance our understanding of biases, and it provides practical insights for researchers and practitioners in medical imaging. The proposed mitigation strategies mitigate gender and racial biases, ensuring fair and unbiased segmentation results. Furthermore, this work promotes equal access to accurate diagnoses and treatment outcomes for diverse patient populations, fostering equitable and inclusive healthcare provision.
Chrisimos: A useful Proof-of-Work for finding Minimal Dominating Set of a graph
Venue: arXiv cs.CR 2023
Year: 2023

Hash-based Proof-of-Work (PoW) used in the Bitcoin Blockchain leads to high energy consumption and resource wastage. In this paper, we aim to re-purpose the energy by replacing the hash function with real-life problems having commercial utility. We propose Chrisimos, a useful Proof-of-Work where miners are required to find a minimal dominating set for real-life graph instances. A miner who is able to output the smallest dominating set for the given graph within the block interval time wins the mining game. We also propose a new chain selection rule that ensures the security of the scheme. Thus our protocol also realizes a decentralized minimal dominating set solver for any graph instance. We provide formal proof of correctness and show via experimental results that the block interval time is within feasible bounds of hash-based PoW.
DiffCR: A Fast Conditional Diffusion Framework for Cloud Removal from Optical Satellite Images
Venue: arXiv cs.CV 2023
Year: 2023

Optical satellite images are a critical data source; however, cloud cover often compromises their quality, hindering image applications and analysis. Consequently, effectively removing clouds from optical satellite images has emerged as a prominent research direction. While recent advancements in cloud removal primarily rely on generative adversarial networks, which may yield suboptimal image quality, diffusion models have demonstrated remarkable success in diverse image-generation tasks, showcasing their potential in addressing this challenge. This paper presents a novel framework called DiffCR, which leverages conditional guided diffusion with deep convolutional networks for high-performance cloud removal for optical satellite imagery. Specifically, we introduce a decoupled encoder for conditional image feature extraction, providing a robust color representation to ensure the close similarity of appearance information between the conditional input and the synthesized output. Moreover, we propose a novel and efficient time and condition fusion block within the cloud removal model to accurately simulate the correspondence between the appearance in the conditional image and the target image at a low computational cost. Extensive experimental evaluations on two commonly used benchmark datasets demonstrate that DiffCR consistently achieves state-of-the-art performance on all metrics, with parameter and computational complexities amounting to only 5.1% and 5.4%, respectively, of those previous best methods. The source code, pre-trained models, and all the experimental results will be publicly available at https://github.com/XavierJiezou/DiffCR upon the paper's acceptance of this work.
UJI Probes: Dataset of Wi-Fi Probe Requests
Venue: arXiv cs.NI 2023
Year: 2023

This paper focuses on the creation of a new, publicly available Wi-Fi probe request dataset. Probe requests belong to the family of management frames used by the 802.11 (Wi-Fi) protocol. As the situation changes year by year, and technology improves probe request studies are necessary to be done on up-to-date data. We provide a month-long probe request capture in an office environment, including work days, weekends, and holidays consisting of over 1 400 000 probe requests. We provide a description of all the important aspects of the dataset. Apart from the raw packet capture we also provide a Radio Map (RM) of the office to ensure the users of the dataset have all the possible information about the environment. To protect privacy, user information in the dataset is anonymized. This anonymization is done in a way that protects the privacy of users while preserving the ability to analyze the dataset to almost the same level as raw data. Furthermore, we showcase several possible use cases for the dataset, like presence detection, temporal Received Signal Strength Indicator (RSSI) stability, and privacy protection evaluation.
Changes in Policy Preferences in German Tweets during the COVID Pandemic
Venue: arXiv cs.CY 2023
Year: 2023

Online social media have become an important forum for exchanging political opinions. In response to COVID measures citizens expressed their policy preferences directly on these platforms. Quantifying political preferences in online social media remains challenging: The vast amount of content requires scalable automated extraction of political preferences -- however fine grained political preference extraction is difficult with current machine learning (ML) technology, due to the lack of data sets. Here we present a novel data set of tweets with fine grained political preference annotations. A text classification model trained on this data is used to extract policy preferences in a German Twitter corpus ranging from 2019 to 2022. Our results indicate that in response to the COVID pandemic, expression of political opinions increased. Using a well established taxonomy of policy preferences we analyse fine grained political views and highlight changes in distinct political categories. These analyses suggest that the increase in policy preference expression is dominated by the categories pro-welfare, pro-education and pro-governmental administration efficiency. All training data and code used in this study are made publicly available to encourage other researchers to further improve automated policy preference extraction methods. We hope that our findings contribute to a better understanding of political statements in online social media and to a better assessment of how COVID measures impact political preferences.
Assessment of POS Owners Awareness of Cybersecurity and Insider Threats in POS Kiosks Related Financial Crimes
Venue: arXiv cs.CR 2023
Year: 2023

The introduction of point of sales POS technologies as a payment system was welcoming and constitutes one of the major breakthroughs in the efforts made to rejuvenate the global financial systems. However, like other information technology IT based financial systems, the POS also poses some cybersecurity security threats. The unique thing about the POS is that the main cybersecurity threats it poses to most users are not IT based which refers to unauthorized access and gaining information without the knowledge of the user. The threats are rather connected to the mode of operation of POS kiosks, particularly as experienced in most parts of Nigeria. The mode of operation exposes users cards to possible cloning.
High-Accuracy Prediction of Metal-Insulator-Metal Metasurface with Deep Learning
Venue: arXiv cs.LG 2023
Year: 2023

Deep learning prediction of electromagnetic software calculation results has been a widely discussed issue in recent years. But the prediction accuracy was still one of the challenges to be solved. In this work, we proposed that the ResNets-10 model was used for predicting plasmonic metasurface S11 parameters. The two-stage training was performed by the k-fold cross-validation and small learning rate. After the training was completed, the prediction loss for aluminum, gold, and silver metal-insulator-metal metasurfaces was -48.45, -46.47, and -35.54, respectively. Due to the ultralow error value, the proposed network can replace the traditional electromagnetic computing method for calculation within a certain structural range. Besides, this network can finish the training process less than 1,100 epochs. This means that the network training process can effectively lower the design process time. The ResNets-10 model we proposed can also be used to design meta-diffractive devices and biosensors, thereby reducing the time required for the calculation process. The ultralow error of the network indicates that this work contributes to the development of future artificial intelligence electromagnetic computing software.
Quarks: A Secure and Decentralized Blockchain-Based Messaging Network
Venue: arXiv cs.CR 2023
Year: 2023

In last two decades, messaging systems have gained widespread popularity both in the enterprise and consumer sectors. Many of these systems used secure protocols like end-to-end encryption to ensure strong security in one-to-one communication. However, the majority of them rely on centralized servers, which allows them to use their users' personal data. Also, it allows the government to track and regulate their citizens' activities, which poses significant threats to "digital freedom". Also, these systems have failed to achieve security attributes like confidentiality, integrity, and privacy for group communications. In this paper, we present a novel blockchain-based secure messaging system named Quarks that overcomes the security pitfalls of the existing systems and eliminates centralized control. We have analyzed our architecture with security models to demonstrate the system's reliability and usability. We have developed a Proof of Concept (PoC) of the Quarks system leveraging Distributed Ledger Technology (DLT) and conducted load testing on that. We noticed that our PoC system achieves all the desired attributes that are prevalent in a traditional centralized messaging scheme despite the limited capacity of the development and testing environment. Therefore, this assures us of the applicability of such systems in the near future if scaled up properly.
3D Scene Diffusion Guidance using Scene Graphs
Venue: arXiv cs.CV 2023
Year: 2023

Guided synthesis of high-quality 3D scenes is a challenging task. Diffusion models have shown promise in generating diverse data, including 3D scenes. However, current methods rely directly on text embeddings for controlling the generation, limiting the incorporation of complex spatial relationships between objects. We propose a novel approach for 3D scene diffusion guidance using scene graphs. To leverage the relative spatial information the scene graphs provide, we make use of relational graph convolutional blocks within our denoising network. We show that our approach significantly improves the alignment between scene description and generated scene.
Correlating Medi-Claim Service by Deep Learning Neural Networks
Venue: arXiv cs.LG 2023
Year: 2023

Medical insurance claims are of organized crimes related to patients, physicians, diagnostic centers, and insurance providers, forming a chain reaction that must be monitored constantly. These kinds of frauds affect the financial growth of both insured people and health insurance companies. The Convolution Neural Network architecture is used to detect fraudulent claims through a correlation study of regression models, which helps to detect money laundering on different claims given by different providers. Supervised and unsupervised classifiers are used to detect fraud and non-fraud claims.
A Kripke Semantics for Hajek's BL
Venue: arXiv cs.LO 2023
Year: 2023

We provide a generalisation of Kripke semantics for Petr Hajek's Basic Logic and prove soundness and completeness of the same with respect to our semantics. We find this semantics easily specialises to the linearly-ordered Kripke frames for Godel-Dummett logic, which BL properly contains. Our soundness, deduction theorem and completeness arguments further strengthen this analogy. This paper extends the insights of our previous paper, "A Kripke Semantics for Intuitionistic Lukasiewicz logic," to the case of Hajeks' BL.
Semantic Analysis of Subexponential Modalities in Distributive Non-commutative Linear Logic
Venue: arXiv cs.LO 2023
Year: 2023

In this paper, we consider the full Lambek calculus enriched with subexponential modalities in a distributive setting. We show that the distributive Lambek calculus with subexponentials is complete with respect to its Kripke frames via canonical extensions. In this approach, we consider subexponentials as S4-like modalities and each modality is interpreted with a reflexive and transitive relation similarly to usual Kripke semantics.
Determinants vs. Algebraic Branching Programs
Venue: arXiv cs.CC 2023
Year: 2023

We show that for every homogeneous polynomial of degree $d$, if it has determinantal complexity at most $s$, then it can be computed by a homogeneous algebraic branching program (ABP) of size at most $O(d^5s)$. Moreover, we show that for $\textit{most}$ homogeneous polynomials, the width of the resulting homogeneous ABP is just $s-1$ and the size is at most $O(ds)$. Thus, for constant degree homogeneous polynomials, their determinantal complexity and ABP complexity are within a constant factor of each other and hence, a super-linear lower bound for ABPs for any constant degree polynomial implies a super-linear lower bound on determinantal complexity; this relates two open problems of great interest in algebraic complexity. As of now, super-linear lower bounds for ABPs are known only for polynomials of growing degree, and for determinantal complexity the best lower bounds are larger than the number of variables only by a constant factor. While determinantal complexity and ABP complexity are classically known to be polynomially equivalent, the standard transformation from the former to the latter incurs a polynomial blow up in size in the process, and thus, it was unclear if a super-linear lower bound for ABPs implies a super-linear lower bound on determinantal complexity. In particular, a size preserving transformation from determinantal complexity to ABPs does not appear to have been known prior to this work, even for constant degree polynomials.