OpenRaiser/Intern-Atlas huggingface dataset:
fMRI-PTE: A Large-scale fMRI Pretrained Transformer Encoder for Multi-Subject Brain Activity Decoding
Venue: arXiv cs.CV 2023
Year: 2023

The exploration of brain activity and its decoding from fMRI data has been a longstanding pursuit, driven by its potential applications in brain-computer interfaces, medical diagnostics, and virtual reality. Previous approaches have primarily focused on individual subject analysis, highlighting the need for a more universal and adaptable framework, which is the core motivation behind our work. In this work, we propose fMRI-PTE, an innovative auto-encoder approach for fMRI pre-training, with a focus on addressing the challenges of varying fMRI data dimensions due to individual brain differences. Our approach involves transforming fMRI signals into unified 2D representations, ensuring consistency in dimensions and preserving distinct brain activity patterns. We introduce a novel learning strategy tailored for pre-training 2D fMRI images, enhancing the quality of reconstruction. fMRI-PTE's adaptability with image generators enables the generation of well-represented fMRI features, facilitating various downstream tasks, including within-subject and cross-subject brain activity decoding. Our contributions encompass introducing fMRI-PTE, innovative data transformation, efficient training, a novel learning strategy, and the universal applicability of our approach. Extensive experiments validate and support our claims, offering a promising foundation for further research in this domain.
Butson Hadamard matrices, bent sequences, and spherical codes
Venue: arXiv cs.CR 2023
Year: 2023

We explore a notion of bent sequence attached to the data consisting of an Hadamard matrix of order $n$ defined over the complex $q^{th}$ roots of unity, an eigenvalue of that matrix, and a Galois automorphism from the cyclotomic field of order $q.$ In particular we construct self-dual bent sequences for various $q\le 60$ and lengths $n\le 21.$ Computational construction methods comprise the resolution of polynomial systems by Groebner bases and eigenspace computations. Infinite families can be constructed from regular Hadamard matrices, Bush-type Hadamard matrices, and generalized Boolean bent functions.As an application, we estimate the covering radius of the code attached to that matrix over $\Z_q.$ We derive a lower bound on that quantity for the Chinese Euclidean metric when bent sequences exist. We give the Euclidean distance spectrum, and bound above the covering radius of an attached spherical code, depending on its strength as a spherical design.
Architecture of Data Anomaly Detection-Enhanced Decentralized Expert System for Early-Stage Alzheimer's Disease Prediction
Venue: arXiv cs.CR 2023
Year: 2023

Alzheimer's Disease is a global health challenge that requires early and accurate detection to improve patient outcomes. Magnetic Resonance Imaging (MRI) holds significant diagnostic potential, but its effective analysis remains a formidable task. This study introduces a groundbreaking decentralized expert system that cleverly combines blockchain technology with Artificial Intelligence (AI) to integrate robust anomaly detection for patient-submitted data. Traditional diagnostic methods often lead to delayed and imprecise predictions, especially in the early stages of the disease. Centralized data repositories struggle to manage the immense volumes of MRI data, and persistent privacy concerns hinder collaborative efforts. Our innovative solution harnesses decentralization to protect data integrity and patient privacy, facilitated by blockchain technology. It not only emphasizes AI-driven MRI analysis but also incorporates a sophisticated data anomaly detection architecture. These mechanisms scrutinize patient-contributed data for various issues, including data quality problems and atypical findings within MRI images. Conducting an exhaustive check of MRI image correctness and quality directly on the blockchain is impractical due to computational complexity and cost constraints. Typically, such checks are performed off-chain, and the blockchain securely records the results. This comprehensive approach empowers our decentralized app to provide more precise early-stage Alzheimer's Disease predictions. By merging the strengths of blockchain, AI, and anomaly detection, our system represents a pioneering step towards revolutionizing disease diagnostics.
On the Integration of Self-Sovereign Identity with TLS 1.3 Handshake to Build Trust in IoT Systems
Venue: arXiv cs.CR 2023
Year: 2023

The centralized PKI is not a suitable solution to provide identities in large-scale IoT systems. The main problem is the high cost of managing X.509 certificates throughout their lifecycle, from installation to regular updates and revocation. The Self-Sovereign Identity (SSI) is a decentralised option that reduces the need for human intervention, and therefore has the potential to significantly reduce the complexity and cost associated to identity management in large-scale IoT systems. However, to leverage the full potential of SSI, the authentication of IoT nodes needs to be moved from the application to the Transport Layer Security (TLS) level. This paper contributes to the adoption of SSI in large-scale IoT systems by addressing, for the first time, the extension of the original TLS 1.3 handshake to support two new SSI authentication modes while maintaining the interoperability with nodes implementing the original handshake protocol. The open source implementation of the new TLS 1.3 handshake protocol in OpenSSL is used to experimentally prove the feasibility of the approach.
Leveraging Hyperbolic Embeddings for Coarse-to-Fine Robot Design
Venue: arXiv cs.AI 2023
Year: 2023

Multi-cellular robot design aims to create robots comprised of numerous cells that can be efficiently controlled to perform diverse tasks. Previous research has demonstrated the ability to generate robots for various tasks, but these approaches often optimize robots directly in the vast design space, resulting in robots with complicated morphologies that are hard to control. In response, this paper presents a novel coarse-to-fine method for designing multi-cellular robots. Initially, this strategy seeks optimal coarse-grained robots and progressively refines them. To mitigate the challenge of determining the precise refinement juncture during the coarse-to-fine transition, we introduce the Hyperbolic Embeddings for Robot Design (HERD) framework. HERD unifies robots of various granularity within a shared hyperbolic space and leverages a refined Cross-Entropy Method for optimization. This framework enables our method to autonomously identify areas of exploration in hyperbolic space and concentrate on regions demonstrating promise. Finally, the extensive empirical studies on various challenging tasks sourced from EvoGym show our approach's superior efficiency and generalization capability.
Diffusion models for probabilistic programming
Venue: arXiv cs.LG 2023
Year: 2023

We propose Diffusion Model Variational Inference (DMVI), a novel method for automated approximate inference in probabilistic programming languages (PPLs). DMVI utilizes diffusion models as variational approximations to the true posterior distribution by deriving a novel bound to the marginal likelihood objective used in Bayesian modelling. DMVI is easy to implement, allows hassle-free inference in PPLs without the drawbacks of, e.g., variational inference using normalizing flows, and does not make any constraints on the underlying neural network model. We evaluate DMVI on a set of common Bayesian models and show that its posterior inferences are in general more accurate than those of contemporary methods used in PPLs while having a similar computational cost and requiring less manual tuning.
Style Locality for Controllable Generation with kNN Language Models
Venue: arXiv cs.CL 2023
Year: 2023

Recent language models have been improved by the addition of external memory. Nearest neighbor language models retrieve similar contexts to assist in word prediction. The addition of locality levels allows a model to learn how to weight neighbors based on their relative location to the current text in source documents, and have been shown to further improve model performance. Nearest neighbor models have been explored for controllable generation but have not examined the use of locality levels. We present a novel approach for this purpose and evaluate it using automatic and human evaluation on politeness, formality, supportiveness, and toxicity textual data. We find that our model is successfully able to control style and provides a better fluency-style trade-off than previous work.
Group Distributionally Robust Knowledge Distillation
Venue: arXiv cs.CV 2023
Year: 2023

Knowledge distillation enables fast and effective transfer of features learned from a bigger model to a smaller one. However, distillation objectives are susceptible to sub-population shifts, a common scenario in medical imaging analysis which refers to groups/domains of data that are underrepresented in the training set. For instance, training models on health data acquired from multiple scanners or hospitals can yield subpar performance for minority groups. In this paper, inspired by distributionally robust optimization (DRO) techniques, we address this shortcoming by proposing a group-aware distillation loss. During optimization, a set of weights is updated based on the per-group losses at a given iteration. This way, our method can dynamically focus on groups that have low performance during training. We empirically validate our method, GroupDistil on two benchmark datasets (natural images and cardiac MRIs) and show consistent improvement in terms of worst-group accuracy.
Rule-Based Error Classification for Analyzing Differences in Frequent Errors
Venue: arXiv cs.SE 2023
Year: 2023

Finding and fixing errors is a time-consuming task not only for novice programmers but also for expert programmers. Prior work has identified frequent error patterns among various levels of programmers. However, the differences in the tendencies between novices and experts have yet to be revealed. From the knowledge of the frequent errors in each level of programmers, instructors will be able to provide helpful advice for each level of learners. In this paper, we propose a rule-based error classification tool to classify errors in code pairs consisting of wrong and correct programs. We classify errors for 95,631 code pairs and identify 3.47 errors on average, which are submitted by various levels of programmers on an online judge system. The classified errors are used to analyze the differences in frequent errors between novice and expert programmers. The analyzed results show that, as for the same introductory problems, errors made by novices are due to the lack of knowledge in programming, and the mistakes are considered an essential part of the learning process. On the other hand, errors made by experts are due to misunderstandings caused by the carelessness of reading problems or the challenges of solving problems differently than usual. The proposed tool can be used to create error-labeled datasets and for further code-related educational research.
A Leakage-based Method for Mitigation of Faulty Reconfigurable Intelligent Surfaces
Venue: arXiv cs.IT 2023
Year: 2023

Reconfigurable Intelligent Surfaces (RISs) are expected to be massively deployed in future beyond-5th generation wireless networks, thanks to their ability to programmatically alter the propagation environment, inherent low-cost and low-maintenance nature. Indeed, they are envisioned to be implemented on the facades of buildings or on moving objects. However, such an innovative characteristic may potentially turn into an involuntary negative behavior that needs to be addressed: an undesired signal scattering. In particular, RIS elements may be prone to experience failures due to lack of proper maintenance or external environmental factors. While the resulting Signal-to-Noise-Ratio (SNR) at the intended User Equipment (UE) may not be significantly degraded, we demonstrate the potential risks in terms of unwanted spreading of the transmit signal to non-intended UE. In this regard, we consider the problem of mitigating such undesired effect by proposing two simple yet effective algorithms, which are based on maximizing the Signal-to-Leakage- and-Noise-Ratio (SLNR) over a predefined two-dimensional (2D) area and are applicable in the case of perfect channel-state-information (CSI) and partial CSI, respectively. Numerical and full-wave simulations demonstrate the added gains compared to leakage-unaware and reference schemes.
Active Noise Control Portable Device Design
Venue: arXiv cs.SD 2023
Year: 2023

While our world is filled with its own natural sounds that we can't resist enjoying, it is also chock-full of other sounds that can be irritating, this is noise. Noise not only influences the working efficiency but also the human's health. The problem of reducing noise is one of great importance and great difficulty. The problem has been addressed in many ways over the years. The current methods for noise reducing mostly rely on the materials and transmission medium, which are only effective to some extent for the high frequency noise. However, the effective reduction noise method especially for low frequency noise is very limited. Here we come up with a noise reduction system consist of a sensor to detect the noise in the environment. Then the noise will be sent to an electronic control system to process the noise, which will generate a reverse phase frequency signal to counteract the disturbance. Finally, the processed smaller noise will be broadcasted by the speaker. Through this smart noise reduction system, even the noise with low-frequency can be eliminated. The system is also integrated with sleep tracking and music player applications. It can also remember and store settings for the same environment, sense temperature, and smart control of home furniture, fire alarm, etc. This smart system can transfer data easily by Wi-Fi or Bluetooth and controlled by its APP. In this project, we will present a model of the above technology which can be used in various environments to prevent noise pollution and provide a solution to the people who have difficulties finding a peaceful and quiet environment for sleep, work or study.
PAUMER: Patch Pausing Transformer for Semantic Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

We study the problem of improving the efficiency of segmentation transformers by using disparate amounts of computation for different parts of the image. Our method, PAUMER, accomplishes this by pausing computation for patches that are deemed to not need any more computation before the final decoder. We use the entropy of predictions computed from intermediate activations as the pausing criterion, and find this aligns well with semantics of the image. Our method has a unique advantage that a single network trained with the proposed strategy can be effortlessly adapted at inference to various run-time requirements by modulating its pausing parameters. On two standard segmentation datasets, Cityscapes and ADE20K, we show that our method operates with about a $50\%$ higher throughput with an mIoU drop of about $0.65\%$ and $4.6\%$ respectively.
De-Diffusion Makes Text a Strong Cross-Modal Interface
Venue: arXiv cs.CV 2023
Year: 2023

We demonstrate text as a strong cross-modal interface. Rather than relying on deep embeddings to connect image and language as the interface representation, our approach represents an image as text, from which we enjoy the interpretability and flexibility inherent to natural language. We employ an autoencoder that uses a pre-trained text-to-image diffusion model for decoding. The encoder is trained to transform an input image into text, which is then fed into the fixed text-to-image diffusion decoder to reconstruct the original input -- a process we term De-Diffusion. Experiments validate both the precision and comprehensiveness of De-Diffusion text representing images, such that it can be readily ingested by off-the-shelf text-to-image tools and LLMs for diverse multi-modal tasks. For example, a single De-Diffusion model can generalize to provide transferable prompts for different text-to-image tools, and also achieves a new state of the art on open-ended vision-language tasks by simply prompting large language models with few-shot examples.
TPSeNCE: Towards Artifact-Free Realistic Rain Generation for Deraining and Object Detection in Rain
Venue: arXiv cs.CV 2023
Year: 2023

Rain generation algorithms have the potential to improve the generalization of deraining methods and scene understanding in rainy conditions. However, in practice, they produce artifacts and distortions and struggle to control the amount of rain generated due to a lack of proper constraints. In this paper, we propose an unpaired image-to-image translation framework for generating realistic rainy images. We first introduce a Triangular Probability Similarity (TPS) constraint to guide the generated images toward clear and rainy images in the discriminator manifold, thereby minimizing artifacts and distortions during rain generation. Unlike conventional contrastive learning approaches, which indiscriminately push negative samples away from the anchors, we propose a Semantic Noise Contrastive Estimation (SeNCE) strategy and reassess the pushing force of negative samples based on the semantic similarity between the clear and the rainy images and the feature similarity between the anchor and the negative samples. Experiments demonstrate realistic rain generation with minimal artifacts and distortions, which benefits image deraining and object detection in rain. Furthermore, the method can be used to generate realistic snowy and night images, underscoring its potential for broader applicability. Code is available at https://github.com/ShenZheng2000/TPSeNCE.
ProcSim: Proxy-based Confidence for Robust Similarity Learning
Venue: arXiv cs.CV 2023
Year: 2023

Deep Metric Learning (DML) methods aim at learning an embedding space in which distances are closely related to the inherent semantic similarity of the inputs. Previous studies have shown that popular benchmark datasets often contain numerous wrong labels, and DML methods are susceptible to them. Intending to study the effect of realistic noise, we create an ontology of the classes in a dataset and use it to simulate semantically coherent labeling mistakes. To train robust DML models, we propose ProcSim, a simple framework that assigns a confidence score to each sample using the normalized distance to its class representative. The experimental results show that the proposed method achieves state-of-the-art performance on the DML benchmark datasets injected with uniform and the proposed semantically coherent noise.
Is Human Culture Locked by Evolution?
Venue: arXiv cs.SI 2023
Year: 2023

Human culture has evolved for thousands of years and thrived in the era of Internet. Due to the availability of big data, we could do research on human culture by analyzing its representation such as user item rating values on websites like MovieLens and Douban. Industrial workers have applied recommender systems in big data to predict user behavior and promote web traffic. In this paper, we analyze the social impact of an algorithm named ZeroMat to show that human culture is locked into a state where individual's cultural taste is predictable at high precision without historic data. We also provide solutions to this problem and interpretation of current Chinese government's regulations and policies.
Towards Collective Superintelligence, a Pilot Study
Venue: arXiv cs.HC 2023
Year: 2023

Conversational Swarm Intelligence (CSI) is a new technology that enables human groups of potentially any size to hold real-time deliberative conversations online. Modeled on the dynamics of biological swarms, CSI aims to optimize group insights and amplify group intelligence. It uses Large Language Models (LLMs) in a novel framework to structure large-scale conversations, combining the benefits of small-group deliberative reasoning and large-group collective intelligence. In this study, a group of 241 real-time participants were asked to estimate the number of gumballs in a jar by looking at a photo. In one test case, individual participants entered their estimation in a standard survey. In another test case, participants converged on groupwise estimates collaboratively using a prototype CSI text-chat platform called Thinkscape. The results show that when using CSI, the group of 241 participants estimated within 12% of the correct answer, which was significantly more accurate (p<0.001) than the average individual (mean error of 55%) and the survey-based Wisdom of Crowd (error of 25%). The group using CSI was also more accurate than an estimate generated by GPT 4 (error of 42%). This suggests that CSI is a viable method for enabling large, networked groups to hold coherent real-time deliberative conversations that amplify collective intelligence. Because this technology is scalable, it could provide a possible pathway towards building a general-purpose Collective Superintelligence (CSi).
ZEETAD: Adapting Pretrained Vision-Language Model for Zero-Shot End-to-End Temporal Action Detection
Venue: arXiv cs.CV 2023
Year: 2023

Temporal action detection (TAD) involves the localization and classification of action instances within untrimmed videos. While standard TAD follows fully supervised learning with closed-set setting on large training data, recent zero-shot TAD methods showcase the promising open-set setting by leveraging large-scale contrastive visual-language (ViL) pretrained models. However, existing zero-shot TAD methods have limitations on how to properly construct the strong relationship between two interdependent tasks of localization and classification and adapt ViL model to video understanding. In this work, we present ZEETAD, featuring two modules: dual-localization and zero-shot proposal classification. The former is a Transformer-based module that detects action events while selectively collecting crucial semantic embeddings for later recognition. The latter one, CLIP-based module, generates semantic embeddings from text and frame inputs for each temporal unit. Additionally, we enhance discriminative capability on unseen classes by minimally updating the frozen CLIP encoder with lightweight adapters. Extensive experiments on THUMOS14 and ActivityNet-1.3 datasets demonstrate our approach's superior performance in zero-shot TAD and effective knowledge transfer from ViL models to unseen action categories.