OpenRaiser/Intern-Atlas huggingface dataset:
Approach to Learning Generalized Audio Representation Through Batch Embedding Covariance Regularization and Constant-Q Transforms
Venue: arXiv cs.SD 2023
Year: 2023

General-purpose embedding is highly desirable for few-shot even zero-shot learning in many application scenarios, including audio tasks. In order to understand representations better, we conducted a thorough error analysis and visualization of HEAR 2021 submission results. Inspired by the analysis, this work experiments with different front-end audio preprocessing methods, including Constant-Q Transform (CQT) and Short-time Fourier transform (STFT), and proposes a Batch Embedding Covariance Regularization (BECR) term to uncover a more holistic simulation of the frequency information received by the human auditory system. We tested the models on the suite of HEAR 2021 tasks, which encompass a broad category of tasks. Preliminary results show (1) the proposed BECR can incur a more dispersed embedding on the test set, (2) BECR improves the PaSST model without extra computation complexity, and (3) STFT preprocessing outperforms CQT in all tasks we tested. Github:https://github.com/ankitshah009/general_audio_embedding_hear_2021
Guided Image-to-Image Translation by Discriminator-Generator Communication
Venue: arXiv cs.CV 2023
Year: 2023

The goal of Image-to-image (I2I) translation is to transfer an image from a source domain to a target domain, which has recently drawn increasing attention. One major branch of this research is to formulate I2I translation based on Generative Adversarial Network (GAN). As a zero-sum game, GAN can be reformulated as a Partially-observed Markov Decision Process (POMDP) for generators, where generators cannot access full state information of their environments. This formulation illustrates the information insufficiency in the GAN training. To mitigate this problem, we propose to add a communication channel between discriminators and generators. We explore multiple architecture designs to integrate the communication mechanism into the I2I translation framework. To validate the performance of the proposed approach, we have conducted extensive experiments on various benchmark datasets. The experimental results confirm the superiority of our proposed method.
Data Games: A Game-Theoretic Approach to Swarm Robotic Data Collection
Venue: arXiv cs.AI 2023
Year: 2023

Fleets of networked autonomous vehicles (AVs) collect terabytes of sensory data, which is often transmitted to central servers (the ''cloud'') for training machine learning (ML) models. Ideally, these fleets should upload all their data, especially from rare operating contexts, in order to train robust ML models. However, this is infeasible due to prohibitive network bandwidth and data labeling costs. Instead, we propose a cooperative data sampling strategy where geo-distributed AVs collaborate to collect a diverse ML training dataset in the cloud. Since the AVs have a shared objective but minimal information about each other's local data distribution and perception model, we can naturally cast cooperative data collection as an $N$-player mathematical game. We show that our cooperative sampling strategy uses minimal information to converge to a centralized oracle policy with complete information about all AVs. Moreover, we theoretically characterize the performance benefits of our game-theoretic strategy compared to greedy sampling. Finally, we experimentally demonstrate that our method outperforms standard benchmarks by up to $21.9\%$ on 4 perception datasets, including for autonomous driving in adverse weather conditions. Crucially, our experimental results on real-world datasets closely align with our theoretical guarantees.
Best-of-Both-Worlds Fairness in Committee Voting
Venue: arXiv cs.GT 2023
Year: 2023

The best-of-both-worlds paradigm advocates an approach that achieves desirable properties both ex-ante and ex-post. We launch a best-of-both-worlds fairness perspective for the important social choice setting of approval-based committee voting. To this end, we initiate work on ex-ante proportional representation properties in this domain and formalize a hierarchy of notions including Individual Fair Share (IFS), Unanimous Fair Share (UFS), Group Fair Share (GFS), and their stronger variants. We establish their compatibility with well-studied ex-post concepts such as extended justified representation (EJR) and fully justified representation (FJR). Our first main result is a polynomial-time algorithm that simultaneously satisfies ex-post EJR, ex-ante GFS and ex-ante Strong UFS. Subsequently, we strengthen our ex-post guarantee to FJR and present an algorithm that outputs a lottery which is ex-post FJR and ex-ante Strong UFS, but does not run in polynomial time.
Bootstrap The Original Latent: Learning a Private Model from a Black-box Model
Venue: arXiv cs.CV 2023
Year: 2023

In this paper, considering the balance of data/model privacy of model owners and user needs, we propose a new setting called Back-Propagated Black-Box Adaptation (BPBA) for users to better train their private models via the guidance of the back-propagated results of a Black-box foundation/source model. Our setting can ease the usage of foundation/source models as well as prevent the leakage and misuse of foundation/source models. Moreover, we also propose a new training strategy called Bootstrap The Original Latent (BTOL) to fully utilize the foundation/source models. Our strategy consists of a domain adapter and a freeze-and-thaw strategy. We apply our BTOL under BPBA and Black-box UDA settings on three different datasets. Experiments show that our strategy is efficient and robust in various settings without manual augmentations.
Boosting the 3D thermal-aware floorplanning problem through a master-worker parallel MOEA
Venue: arXiv cs.AR 2023
Year: 2023

The increasing transistor scale integration poses, among others, the thermal-aware floorplanning problem; consisting of how to place the hardware components in order to reduce overheating by dissipation. Due to the huge amount of feasible floorplans, most of the solutions found in the literature include an evolutionary algorithm for, either partially or completely, carrying out the task of floorplanning. Evolutionary algorithms usually have a bottleneck in the fitness evaluation. In the problem of thermal-aware floorplanning, the layout evaluation by the thermal model takes 99.5\% of the computational time for the best floorplanning algorithm proposed so far.The contribution of this paper is to present a parallelization of this evaluation phase in a master$-$worker model to achieve a dramatic speed-up of the thermal-aware floorplanning process. Exhaustive experimentation was done over three dimensional integrated circuits, with 48 and 128 cores, outperforming previous published works.
Hidden Knowledge: Mathematical Methods for the Extraction of the Fingerprint of Medieval Paper from Digital Images
Venue: arXiv cs.CV 2023
Year: 2023

Medieval paper, a handmade product, is made with a mould which leaves an indelible imprint on the sheet of paper. This imprint includes chain lines, laid lines and watermarks which are often visible on the sheet. Extracting these features allows the identification of paper stock and gives information about chronology, localisation and movement of books and people. Most computational work for feature extraction of paper analysis has so far focused on radiography or transmitted light images. While these imaging methods provide clear visualisation for the features of interest, they are expensive and time consuming in their acquisition and not feasible for smaller institutions. However, reflected light images of medieval paper manuscripts are abundant and possibly cheaper in their acquisition. In this paper, we propose algorithms to detect and extract the laid and chain lines from reflected light images. We tackle the main drawback of reflected light images, that is, the low contrast attenuation of lines and intensity jumps due to noise and degradation, by employing the spectral total variation decomposition and develop methods for subsequent line extraction. Our results clearly demonstrate the feasibility of using reflected light images in paper analysis. This work enables the feature extraction for paper manuscripts that have otherwise not been analysed due to a lack of appropriate images. We also open the door for paper stock identification at scale.
AI Risk Skepticism, A Comprehensive Survey
Venue: arXiv cs.CY 2023
Year: 2023

In this thorough study, we took a closer look at the skepticism that has arisen with respect to potential dangers associated with artificial intelligence, denoted as AI Risk Skepticism. Our study takes into account different points of view on the topic and draws parallels with other forms of skepticism that have shown up in science. We categorize the various skepticisms regarding the dangers of AI by the type of mistaken thinking involved. We hope this will be of interest and value to AI researchers concerned about the future of AI and the risks that it may pose. The issues of skepticism and risk in AI are decidedly important and require serious consideration. By addressing these issues with the rigor and precision of scientific research, we hope to better understand the objections we face and to find satisfactory ways to resolve them.
How to Construct Energy for Images? Denoising Autoencoder Can Be Energy Based Model
Venue: arXiv cs.CV 2023
Year: 2023

Energy-based models parameterize the unnormalized log-probability of data samples, but there is a lack of guidance on how to construct the "energy". In this paper, we propose a Denoising-EBM which decomposes the image energy into "semantic energy" and "texture energy". We define the "semantic energy" in the latent space of DAE to model the high-level representations, and define the pixel-level reconstruction error for denoising as "texture energy". Inspired by score-based model, our model utilizes multi-scale noisy samples for maximum-likelihood training and it outputs a vector instead of a scalar for exploring a larger set of functions during optimization. After training, the semantics are first synthesized by fast MCMC through "semantic energy", and then the pixel-level refinement of semantic image will be performed to generate perfect samples based on "texture energy". Ultimately, our model can outperform most EBMs in image generation. And we also demonstrate that Denoising-EBM has top performance among EBMs for out-of-distribution detection.
A Juridicidade e a Regulamentação dos Dark Patterns
Venue: arXiv cs.CY 2023
Year: 2023

The evolution of audiovisual computer interfaces was an important milestone for the popularization of the internet without which it is impossible to conceive the use of this technology in modern society However the progress of these interfaces has not taken exclusively beneficial paths for humanity From the beginning of the 21st century onwards an increase in interface design patterns was observed that instead of facilitating navigation harmed users or restricted their decisionmaking capabilities earning them the name of Dark Patterns In view of this the present work aims to address whether Dark Patterns are legal or illegal in the face of Brazilian data protection and consumer law verifying in the absence of specific norms on Dark Patterns the best way to regulate them The research method employed is qualitative analyzing research court cases norms and national and foreign documents on Dark Patterns After addressing its effects its legal development and establishing a definition compatible with Brazilian law it was concluded that although some implementations are capable of producing damage and violating rights in some cases the mere declaration of the illegality of these techniques is an insufficient solution requiring further investigations regarding the hypotheses in which their negative impacts are less apparent or when they are used for beneficial purposes among other unsolved problems Therefore it is suggested that the regulation of Dark Patterns should occur through a system composed of formal laws and regulations of public administration bodies through a multidisciplinary approach that is adaptable to new findings and technologies
Document-level Relation Extraction with Cross-sentence Reasoning Graph
Venue: arXiv cs.CL 2023
Year: 2023

Relation extraction (RE) has recently moved from the sentence-level to document-level, which requires aggregating document information and using entities and mentions for reasoning. Existing works put entity nodes and mention nodes with similar representations in a document-level graph, whose complex edges may incur redundant information. Furthermore, existing studies only focus on entity-level reasoning paths without considering global interactions among entities cross-sentence. To these ends, we propose a novel document-level RE model with a GRaph information Aggregation and Cross-sentence Reasoning network (GRACR). Specifically, a simplified document-level graph is constructed to model the semantic information of all mentions and sentences in a document, and an entity-level graph is designed to explore relations of long-distance cross-sentence entity pairs. Experimental results show that GRACR achieves excellent performance on two public datasets of document-level RE. It is especially effective in extracting potential relations of cross-sentence entity pairs. Our code is available at https://github.com/UESTC-LHF/GRACR.
Error convergence and engineering-guided hyperparameter search of PINNs: towards optimized I-FENN performance
Venue: arXiv cs.LG 2023
Year: 2023

In our recently proposed Integrated Finite Element Neural Network (I-FENN) framework (Pantidis and Mobasher, 2023) we showcased how PINNs can be deployed on a finite element-level basis to swiftly approximate a state variable of interest, and we applied it in the context of non-local gradient-enhanced damage mechanics. In this paper, we enhance the rigour and performance of I-FENN by focusing on two crucial aspects of its PINN component: a) the error convergence analysis and b) the hyperparameter-performance relationship. Guided by the available theoretical formulations in the field, we introduce a systematic numerical approach based on a novel set of holistic performance metrics to answer both objectives. In the first objective, we explore in detail the convergence of the PINN training error and the global error against the network size and the training sample size. We demonstrate a consistent converging behavior of the two error types for any investigated combination of network complexity, dataset size and choice of hyperparameters, which empirically proves the conformance of the PINN setup and implementation to the available convergence theories. In the second objective, we establish an a-priori knowledge of the hyperparameters which favor higher predictive accuracy, lower computational effort, and the least chances of arriving at trivial solutions. The analysis leads to several outcomes that contribute to the better performance of I-FENN, and fills a long-standing gap in the PINN literature with regards to the numerical convergence of the network errors while accounting for commonly used optimizers (Adam and L-BFGS). The proposed analysis method can be directly extended to other ML applications in science and engineering. The code and data utilized in the analysis are posted publicly to aid the reproduction and extension of this research.
FFT-based Dynamic Token Mixer for Vision
Venue: arXiv cs.CV 2023
Year: 2023

Multi-head-self-attention (MHSA)-equipped models have achieved notable performance in computer vision. Their computational complexity is proportional to quadratic numbers of pixels in input feature maps, resulting in slow processing, especially when dealing with high-resolution images. New types of token-mixer are proposed as an alternative to MHSA to circumvent this problem: an FFT-based token-mixer involves global operations similar to MHSA but with lower computational complexity. However, despite its attractive properties, the FFT-based token-mixer has not been carefully examined in terms of its compatibility with the rapidly evolving MetaFormer architecture. Here, we propose a novel token-mixer called Dynamic Filter and novel image recognition models, DFFormer and CDFFormer, to close the gaps above. The results of image classification and downstream tasks, analysis, and visualization show that our models are helpful. Notably, their throughput and memory efficiency when dealing with high-resolution image recognition is remarkable. Our results indicate that Dynamic Filter is one of the token-mixer options that should be seriously considered. The code is available at https://github.com/okojoalg/dfformer
DEDGAT: Dual Embedding of Directed Graph Attention Networks for Detecting Financial Risk
Venue: arXiv cs.LG 2023
Year: 2023

Graph representation plays an important role in the field of financial risk control, where the relationship among users can be constructed in a graph manner. In practical scenarios, the relationships between nodes in risk control tasks are bidirectional, e.g., merchants having both revenue and expense behaviors. Graph neural networks designed for undirected graphs usually aggregate discriminative node or edge representations with an attention strategy, but cannot fully exploit the out-degree information when used for the tasks built on directed graph, which leads to the problem of a directional bias. To tackle this problem, we propose a Directed Graph ATtention network called DGAT, which explicitly takes out-degree into attention calculation. In addition to having directional requirements, the same node might have different representations of its input and output, and thus we further propose a dual embedding of DGAT, referred to as DEDGAT. Specifically, DEDGAT assigns in-degree and out-degree representations to each node and uses these two embeddings to calculate the attention weights of in-degree and out-degree nodes, respectively. Experiments performed on the benchmark datasets show that DGAT and DEDGAT obtain better classification performance compared to undirected GAT. Also,the visualization results demonstrate that our methods can fully use both in-degree and out-degree information.
Efficient Computation of Redundancy Matrices for Moderately Redundant Truss and Frame Structures
Venue: arXiv cs.CE 2023
Year: 2023

Large statically indeterminate truss and frame structures exhibit complex load-bearing behavior, and redundancy matrices are helpful for their analysis and design. Depending on the task, the full redundancy matrix or only its diagonal entries are required. The standard computation procedure has a high computational effort. Many structures fall in the category of moderately redundant, i.e., the ratio of the statical indeterminacy to the number of all load-carrying modes of all elements is less one half. This paper proposes a closed-form expression for redundancy contributions that is computationally efficient for moderately redundant systems. The expression is derived via a factorization of the redundancy matrix that is based on singular value decomposition. Several examples illustrate the behavior of the method for increasing size of systems and, where applicable, for increasing degree of statical indeterminacy.
Video traffic identification with novel feature extraction and selection method
Venue: arXiv cs.NI 2023
Year: 2023

In recent years, the rapid rise of video applications has led to an explosion of Internet video traffic, thereby posing severe challenges to network management. Therefore, effectively identifying and managing video traffic has become an urgent problem to be solved. However, the existing video traffic feature extraction methods mainly target at the traditional packet and flow level features, and the video traffic identification accuracy is low. Additionally, the issue of high data dimension often exists in video traffic identification, requiring an effective approach to select the most relevant features to complete the identification task. Although numerous studies have used feature selection to achieve improved identification performance, no feature selection research has focused on measuring feature distributions that do not overlap or have a small overlap. First, this study proposes to extract video-related features to construct a large-scale feature set to identify video traffic. Second, to reduce the cost of video traffic identification and select an effective feature subset, the current research proposes an adaptive distribution distance-based feature selection (ADDFS) method, which uses Wasserstein distance to measure the distance between feature distributions. To test the effectiveness of the proposal, we collected a set of video traffic from different platforms in a campus network environment and conducted a set of experiments using these data sets. Experimental results suggest that the proposed method can achieve high identification performance for video scene traffic and cloud game video traffic identification. Lastly, a comparison of ADDFS with other feature selection methods shows that ADDFS is a practical feature selection technique not only for video traffic identification, but also for general classification tasks.
Probing Graph Representations
Venue: arXiv cs.LG 2023
Year: 2023

Today we have a good theoretical understanding of the representational power of Graph Neural Networks (GNNs). For example, their limitations have been characterized in relation to a hierarchy of Weisfeiler-Lehman (WL) isomorphism tests. However, we do not know what is encoded in the learned representations. This is our main question. We answer it using a probing framework to quantify the amount of meaningful information captured in graph representations. Our findings on molecular datasets show the potential of probing for understanding the inductive biases of graph-based models. We compare different families of models and show that transformer-based models capture more chemically relevant information compared to models based on message passing. We also study the effect of different design choices such as skip connections and virtual nodes. We advocate for probing as a useful diagnostic tool for evaluating graph-based models.
Teaching AI and Robotics to Children in a Mexican town
Venue: arXiv cs.CY 2023
Year: 2023

In this paper, we present a pilot study aiming to investigate the challenges of teaching AI and Robotics to children in low- and middle-income countries. Challenges such as the little to none experts and the limited resources in a Mexican town to teach AI and Robotics were addressed with the creation of inclusive learning activities with Montessori method and open-source educational robots. For the pilot study, we invited 14 participants of which 10 were able to attend, 6 male and 4 female of (age in years: mean=8 and std=$\pm$1.61) and four instructors of different teaching experience levels to young audiences. We reported results of a four-lesson curriculum that is both inclusive and engaging. We showed the impact on the increase of general agreement of participants on the understanding of what engineers and scientists do in their jobs, with engineering attitudes surveys and Likert scale charts from the first and the last lesson. We concluded that this pilot study helped children coming from low- to mid-income families to learn fundamental concepts of AI and Robotics and aware them of the potential of AI and Robotics applications which might rule their adult lives. Future work might lead (a) to have better understanding on the financial and logistical challenges to organise a workshop with a major number of participants for reliable and representative data and (b) to improve pretest-posttest survey design and its statistical analysis. The resources to reproduce this work are available at \url{https://github.com/air4children/dei-hri2023}.