Reviewerly/exHarmony huggingface dataset:
Reevaluating Main Concepts of Intellectual Property in the Light of AI-Challenges

Abstract This paper examines the impact of Artificial Intelligence (AI) systems on some of the main principles of Intellectual Property, specifically those inherent to copyright and patent law. Particular attention is dedicated to the concept of authorship in relation to AI-assisted and AI-generated creations in the field of art. The chapter also examines the potential need, or a lack thereof, to re-evaluate the anthropocentric foundations of copyright embedded in the acquis communautaire , copyright laws of the Member States and the case law of CJEU. Taking into consideration the AI’s inability to act as an autonomous agent, attention is drawn to possible legislative interventions to introduce legal certainty, whilst honoring and maintaining core values of copyright law.
Inheritance Law in the Twenty-First Century: New Circumstances and Challenges

Abstract This chapter explores the influence of technology on traditional components of inheritance. An increasing amount of social interaction takes place online. Consequently, people accumulate large quantities of digital assets. Some of these assets have a considerable material value, while others might have emotional value for the family of a decedent. They might also have historical value for society in general. However, from the standpoint of inheritance law, in many countries it is unclear what happens to digital assets after their owner/user dies. These assets represent a new type of “property” and few legislators around the world have regulated them specifically. The need to implement these assets into inheritance law is why it is important for law students to be familiar with the concept of digital assets, understand their categories, as well as the differences between them. The chapter also considers possible solutions on what will happen to them, post mortem . To map the landscape for the readers, the chapter divides digital assets into four comprehensive groups and explains what happens to each of these digital assets post mortem . This chapter also lists the reasons why some of these assets cannot be inherited and possible problems with inheritance of those digital assets that are inheritable.
Measuring Sentiment Bias in Machine Translation

Biases induced to text by generative models have become an increasingly large topic in recent years. In this paper we explore how machine translation might introduce a bias in sentiments as classified by sentiment analysis models. For this, we compare three open access machine translation models for five different languages on two parallel corpora to test if the translation process causes a shift in sentiment classes recognized in the texts. Though our statistic test indicate shifts in the label probability distributions, we find none that appears consistent enough to assume a bias induced by the translation process.
VITS: Quality Vs. Speed Analysis

In this paper, we analyze the performance of a modern end-to-end speech synthesis model called Variational Inference with adversarial learning for end-to-end Text-to-Speech (VITS). We build on the original VITS model and examine how different modifications to its architecture affect synthetic speech quality and computational complexity. Experiments with two Czech voices, a male and a female, were carried out. To assess the quality of speech synthesized by the different modified models, MUSHRA listening tests were performed. The computational complexity was measured in terms of synthesis speed over real time. While the original VITS model is still preferred regarding speech quality, we present a modification of the original structure with a significantly better response yet providing acceptable output quality. Such a configuration can be used when system response latency is critical.
When Whisper Meets TTS: Domain Adaptation Using only Synthetic Speech Data

Automatic Speech Recognition is among the most important areas of Artificial Intelligence research today. One of the most notable advances in this area is the development of end-to-end models, which have shown state-of-the-art performance in many benchmark scenarios. In spite of the recent improvements, these architectures still require large amounts of transcribed speech data to be trained, which can be challenging in low resource languages, or in specific domains due to privacy concerns. This study proposes a methodology to fine-tune Whisper-based models using only synthetic speech. The aim is to enable training robust systems for specific domains and low resource languages, where large labeled corpora are difficult to collect. Our approach is based on a language model adaptation by fine-tuning only the decoder of the model, thus the network is able to learn specific vocabulary that is not initially available. The proposed methodology is evaluated with data from different languages and domains. In addition, Parameter Efficient Fine-Tuning strategies were used to efficiently adapt the large pre-trained Whisper models. This is one of the first studies that considers the effect of using only synthetic speech for domain adaption of speech recognition systems in non-English data, providing word error rate reductions in low resource languages between 2 and 30 points, depending on the Whisper version.
Automatic Pronunciation Assessment of Non-native English Based on Phonological Analysis

The rapid development of speech recognition systems has motivated the community to work on accent classification, considerably improving the performance of these systems. However, only a few works or tools have focused on evaluating and analyzing in depth not only the accent but also the pronunciation level of a person when learning a non-native language. Our study aims to evaluate the pronunciation skills of non-native English speakers whose first language is Arabic, Chinese, Spanish, or French. We considered training a system to compute posterior probabilities of phonological classes from English native speakers and then evaluating whether it is possible to discriminate between native English speakers vs. non-native English speakers. Posteriors of each phonological class separately and also their combination are considered. Phonemes with low posterior results are used to give feedback to the speaker regarding which phonemes should be improved. The results suggest that it is possible to distinguish between each of the non-native languages and native English with accuracies between 67.6% and 80.6%. According to our observations, the most discriminant phonological classes are alveolar, lateral, velar, and front. Finally, the paper introduces a graphical way to interpret the results phoneme-by-phoneme, such that the speaker receives feedback about his/her pronunciation performance.
Learning nonlocal weights for second-order nonlocal super-resolution

This research introduces an enhanced approach for multiframe super-resolution (SR) that incorporates a bilevel optimization technique for learning the space variable weights parameter in the second-order nonlocal term.This novel approach effectively preserves image edges and ensures accurate reconstruction of image textures.The main contributions of this work include the establishment of the existence and uniqueness of the solution within a well-posed framework and a bilevel optimization procedure to compute the weights α.The experimental results demonstrate the efficiency and accuracy of the proposed method compared to existing super-resolution approaches.These findings highlight the practical effectiveness and potential of the proposed approach in real-world applications.
Privacy-preserving outsourcing decision tree evaluation from homomorphic encryption

A decision tree is a common algorithm in machine learning, which performs classification prediction based on a tree structure. In real-world scenarios, input attribute values may be sensitive and tree models are usually private, thus the computation potentially incurs security and privacy risks. In this paper, we focus on how to retain privacy when outsourcing decision tree evaluation. Leveraging homomorphic encryption, our construction achieves data confidentiality against semi-honest adversaries while enabling both the client (who provides the attributes) and the model holder to be offline during evaluation. Based on an unbalanced-computational-ability two-server model, we achieve single-branch evaluation, which is exponentially less computation than previous schemes that evaluated the entire decision tree. A proof-of-concept implementation demonstrates this. Additionally, with multi-key encryption and joint decryption, our protocol easily supports multi-client scenarios.
Regularized Vector Quantization for Tokenized Image Synthesis

Quantizing images into discrete representations has been a fundamental problem in unified generative modeling. Predominant approaches learn the discrete representation either in a deterministic manner by selecting the best-matching token or in a stochastic manner by sampling from a predicted distribution. However, deterministic quantization suffers from severe codebook collapse and misalignment with inference stage while stochastic quantization suffers from low codebook utilization and perturbed reconstruction objective. This paper presents a regularized vector quantization framework that allows to mitigate above issues effectively by applying regularization from two perspectives. The first is a prior distribution regularization which measures the discrepancy between a prior token distribution and the predicted token distribution to avoid code-book collapse and low codebook utilization. The second is a stochastic mask regularization that introduces stochasticity during quantization to strike a good balance between inference stage misalignment and unperturbed reconstruction objective. In addition, we design a probabilistic contrastive loss which serves as a calibrated metric to further mitigate the perturbed reconstruction objective. Extensive experiments show that the proposed quantization framework outperforms prevailing vector quantization methods consistently across different generative models including auto-regressive models and diffusion models.
Defining and Quantifying the Emergence of Sparse Concepts in DNNs

This paper aims to illustrate the concept-emerging phenomenon in a trained DNN. Specifically, we find that the inference score of a DNN can be disentangled into the effects of a few interactive concepts. These concepts can be understood as causal patterns in a sparse, symbolic causal graph, which explains the DNN. The faithfulness of using such a causal graph to explain the DNN is theoretically guaranteed, because we prove that the causal graph can well mimic the DNN's outputs on an exponential number of different masked samples. Besides, such a causal graph can be further simplified and re-written as an And-Or graph (AOG), without losing much explanation accuracy. The code is released at https://github.com/sjtu-xai-lab/aog.
How to Prevent the Continuous Damage of Noises to Model Training?

Deep learning with noisy labels is challenging and inevitable in many circumstances. Existing methods reduce the impact of mislabeled samples by reducing loss weights or screening, which highly rely on the model's superior discriminative power for identifying mislabeled samples. However, in the training stage, the trainee model is imperfect and will wrongly predict some mislabeled samples, which cause continuous damage to the model training. Consequently, there is a large performance gap between existing anti-noise models trained with noisy samples and models trained with clean samples. In this paper, we put forward a Gradient Switching Strategy (GSS) to prevent the continuous damage of mislabeled samples to the classifier. Theoretical analysis shows that the damage comes from the misleading gradient direction computed from the mislabeled samples. The trainee model will deviate from the correct optimization direction under the influence of the accumulated misleading gradient of mislabeled samples. To address this problem, the proposed GSS alleviates the damage by switching the gradient direction of each sample based on the gradient direction pool, which contains all-class gradient directions with different probabilities. During training, each gradient direction pool is updated iteratively, which assigns higher probabilities to potential principal directions for high-confidence samples. Conversely, uncertain samples are forced to explore in different directions rather than mislead model in a fixed direction. Extensive experiments show that GSS can achieve comparable performance with a model trained with clean data. Moreover, the proposed GSS is pluggable for existing frameworks. This idea of switching gradient directions provides a new perspective for future noisy-label learning.
Learning 3D Scene Priors with 2D Supervision

Holistic 3D scene understanding entails estimation of both layout configuration and object geometry in a 3D environment. Recent works have shown advances in 3D scene estimation from various input modalities (e.g., images, 3D scans), by leveraging 3D supervision (e.g., 3D bounding boxes or CAD models), for which collection at scale is ex-pensive and often intractable. To address this shortcoming, we propose a new method to learn 3D scene priors of layout and shape without requiring any 3D ground truth. Instead, we rely on 2D supervision from multi-view RGB images. Our method represents a 3D scene as a latent vector, from which we can progressively decode to a sequence of objects characterized by their class categories, 3D bounding boxes, and meshes. With our trained autoregressive decoder representing the scene prior, our method facilitates many downstream applications, including scene synthesis, interpolation, and single-view reconstruction. Experiments on 3D-FRONT and ScanNet show that our method outperforms state of the art in single-view reconstruction, and achieves state-of-the-art results in scene synthesis against baselines which require for 3D supervision. Project page: https://yinyunie.github.io/sceneprior-page/
Backdoor Cleansing with Unlabeled Data

Due to the increasing computational demand of Deep Neural Networks (DNNs), companies and organizations have begun to outsource the training process. However, the externally trained DNNs can potentially be backdoor attacked. It is crucial to defend against such attacks, i.e., to postprocess a suspicious model so that its backdoor behavior is mitigated while its normal prediction power on clean inputs remain uncompromised. To remove the abnormal backdoor behavior, existing methods mostly rely on additional labeled clean samples. However, such requirement may be unrealistic as the training data are often unavailable to end users. In this paper, we investigate the possibility of circumventing such barrier. We propose a novel defense method that does not require training labels. Through a carefully designed layer-wise weight reinitialization and knowledge distillation, our method can effectively cleanse backdoor behaviors of a suspicious network with negligible compromise in its normal behavior. In experiments, we show that our method, trained without labels, is on-par with state-of-the-art defense methods trained using labels. We also observe promising defense results even on out-of-distribution data. This makes our method very practical. Code is available at: https://github.com/luluppang/BCU.
Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction

We study the problem of learning goal-conditioned policies in Minecraft, a popular, widely accessible yet challenging open-ended environment for developing human-level multi-task agents. We first identify two main challenges of learning such policies: 1) the indistinguishability of tasks from the state distribution, due to the vast scene diversity, and 2) the non-stationary nature of environment dynamics caused by partial observability. To tackle the first challenge, we propose Goal-Sensitive Backbone (GSB) for the policy to encourage the emergence of goal-relevant visual state representations. To tackle the second challenge, the policy is further fueled by an adaptive horizon prediction module that helps alleviate the learning uncertainty brought by the non-stationary dynamics. Experiments on 20 Minecraft tasks show that our method significantly outperforms the best baseline so far; in many of them, we double the performance. Our ablation and exploratory studies then explain how our approach beat the counterparts and also unveil the surprising bonus of zero-shot generalization to new scenes (biomes). We hope our agent could help shed some light on learning goal-conditioned, multi-task agents in challenging, open-ended environments like Minecraft. The code is released at https://github.com/CraftJarvis/MC-Controller.
BKinD-3D: Self-Supervised 3D Keypoint Discovery from Multi-View Videos

Quantifying motion in 3D is important for studying the behavior of humans and other animals, but manual pose annotations are expensive and time-consuming to obtain. Self-supervised keypoint discovery is a promising strategy for estimating 3D poses without annotations. However, current keypoint discovery approaches commonly process single 2D views and do not operate in the 3D space. We propose a new method to perform self-supervised keypoint discovery in 3D from multi-view videos of behaving agents, without any keypoint or bounding box supervision in 2D or 3D. Our method, BKinD-3D, uses an encoder-decoder architecture with a 3D volumetric heatmap, trained to reconstruct spatiotemporal differences across multiple views, in addition to joint length constraints on a learned 3D skeleton of the subject. In this way, we discover keypoints without requiring manual supervision in videos of humans and rats, demonstrating the potential of 3D keypoint discovery for studying behavior.
Learning to Retain while Acquiring: Combating Distribution-Shift in Adversarial Data-Free Knowledge Distillation

Data-free Knowledge Distillation (DFKD) has gained popularity recently, with the fundamental idea of carrying out knowledge transfer from a Teacher neural network to a Student neural network in the absence of training data. However, in the Adversarial DFKD framework, the student network's accuracy, suffers due to the non-stationary distribution of the pseudo-samples under multiple generator updates. To this end, at every generator update, we aim to maintain the student's performance on previously encountered examples while acquiring knowledge from samples of the current distribution. Thus, we propose a meta-learning inspired framework by treating the task of Knowledge-Acquisition (learning from newly generated samples) and Knowledge-Retention (retaining knowledge on previously met samples) as meta-train and meta-test, respectively. Hence, we dub our method as Learning to Retain while Acquiring. Moreover, we identify an implicit aligning factor between the Knowledge-Retention and Knowledge-Acquisition tasks indicating that the proposed student update strategy enforces a common gradient direction for both tasks, alleviating interference between the two objectives. Finally, we support our hypothesis by exhibiting extensive evaluation and comparison of our method with prior arts on multiple datasets.
Defending Against Patch-based Backdoor Attacks on Self-Supervised Learning

A.1. Additional AblationsWe consider the effect of number of images scored by iterative search in Table A1 andFigure A1.Number of scored images can be changed by varying hyperparameters, l (number of clusters), s (number of samples scored per cluster) and r (percentage of least poisonous clusters removed in each iteration).The results show generally good performance when processing more than 6% of samples.Next, we consider the effect of varying the size of flip test set X f in Table A2.We find that we can get reasonable results with |X f | as small as 316.Next, we evaluate intermediate checkpoints of one of our models to understand the relationship between overall performance of the model (measured with clean data Acc) and the attack effectiveness (measured with patched data FP) in Table A3.We can see that attack effectiveness improves as the overall model performance improves in the early epochs.In later epochs, the attack effectiveness fluctuates but is still high relative to earlier epochs.This indicates that it is possible to reduce the compute requirements for the model used during defense.We only need to train this model until the attack effectiveness becomes broadly comparable to a fully trained model.Next, we show that our defense is effective even when changing datasets in Table A4.Next, we consider the effect of varying top-k in Table A5, and find that any top-k <= 20 is a good choice.Next, we consider changing the trigger size w during the attack in Table A6.We find that our defense works despite changing w from 50 to 75 and 100.Next, we consider attacking images from a downstream dataset instead of pre-training dataset.We use Food101 [6] classification dataset for this purpose.It consists 101 fine-grained food categories and 750 images per category.For each category, we randomly select 700 images for training and 50 images for validation.We backdoor BYOL, ResNet-18 models by picking random category from Food101, poisoning all of its 700 training images,
OSRT: Omnidirectional Image Super-Resolution with Distortion-aware Transformer

Omnidirectional images (ODIs) have obtained lots of research interest for immersive experiences. Although ODIs require extremely high resolution to capture details of the entire scene, the resolutions of most ODIs are insufficient. Previous methods attempt to solve this issue by image super-resolution (SR) on equirectangular projection (ERP) images. However, they omit geometric properties of ERP in the degradation process, and their models can hardly generalize to real ERP images. In this paper, we propose Fisheye downsampling, which mimics the real-world imaging process and synthesizes more realistic low-resolution samples. Then we design a distortion-aware Transformer (OSRT) to modulate ERP distortions continuously and self-adaptively. Without a cumbersome process, OSRT outperforms previous methods by about 0.2dB on PSNR. Moreover, we propose a convenient data augmentation strategy, which synthesizes pseudo ERP images from plain images. This simple strategy can alleviate the over-fitting problem of large networks and significantly boost the performance of ODISR. Extensive experiments have demonstrated the state-of-the-art performance of our OSRT.
PointCMP: Contrastive Mask Prediction for Self-supervised Learning on Point Cloud Videos

Self-supervised learning can extract representations of good quality from solely unlabeled data, which is ap-pealing for point cloud videos due to their high labelling cost. In this paper, we propose a contrastive mask prediction (PointCMP) framework for self-supervised learning on point cloud videos. Specifically, our PointCMP employs a two-branch structure to achieve simultaneous learning of both local and global spatiotemporal information. On top of this two-branch structure, a mutual similarity based augmentation module is developed to synthesize hard samples at the feature level. By masking dominant tokens and erasing principal channels, we generate hard samples to facilitate learning representations with better discrimi-nation and generalization performance. Extensive experiments show that our PointCMP achieves the state-of-the-art performance on benchmark datasets and outperforms existing full-supervised counterparts. Transfer learning results demonstrate the superiority of the learned representations across different datasets and tasks.