dataesr/scientific-paragraphs-categorization huggingface dataset:
(i) For English speaker anonymization  [11] ,  [12] ,  [13] , the performance of speaker verifiability has gradually decreased against more powerful attackers. Additionally, voice distinctiveness is significantly degraded by anonymization. (ii) For unseen-language (e. g. , Mandarin) speaker anonymization, pseudo-speaker representations are generated from an external English speaker vector pool, and the resulting language mismatch increases the character error rate (CER)  [33] ,  [34]. Following this pipeline of disentanglement-based anonymization, with special consideration of the selectionbased approach's problems, we propose a novel speaker anonymization system (SAS) based on an orthogonal Householder neural network (OHNN). As shown in the lower part of Fig. 1 , the OHNN-based anonymizer generates distinctive anonymized speaker vectors that can protect privacy under all attack scenarios and can successfully be adapted to unseen-language speaker anonymization without severe language mismatch.
The main contributions of this work are as follows: • We propose an OHNN-based anonymizer that transforms original speaker vectors into anonymized ones with carefully designed training constraints. We show empirically that these anonymized speaker vectors are diverse and language-neutral. • We visualize the cosine similarities between pairs of speaker vectors extracted from the generated speech of users and different attackers. These generated speech are obtained using the commonly used selection-based anonymizer and our OHNN-based anonymizer. The results show that our proposed method effectively reduces the privacy leakage against different attackers and improves the diversity of anonymized speakers. We conducted experiments on VPC English datasets and the AISHELL-3 Mandarin datasets. Our findings show that the proposed model can be successfully adapted to both a matched language condition (i.
e. , English) and a mismatched language condition where the target language (Mandarin) is not included in the training database. The proposed anonymizer achieved a competitive performance under all attack scenarios in terms of privacy and utility metrics. Under the Semi-informed condition, our proposed methods achieved better results for English speaker anonymization than all the submissions to VPC2022  [35] ,  [36] ,  [24] ,  [37] ,  [38] ,  [25].
A. The VoicePrivacy Challenges The VPC formulates the speaker anonymization task as a game between users and attackers, as shown in Fig. 2. A user publishes anonymized data, called test trials, after applying an SAS to his or her original private speech. According to the VPC evaluation plan  [13] , an SAS should: • output an anonymized speech waveform; • conceal the speaker's identity from different attackers; • keep content and other paralinguistic attributes unchanged to maintain intelligibility and naturalness; • ensure all test trials from the same speaker are attributed to the same pseudo-speaker, while test trials from different speakers have different pseudo-speakers  1 1) Attack Models and Objective Evaluation Metrics: a) Privacy metric: To assess the ability to protect a speaker's identity in different scenarios, the ASV performance in terms of the equal error rate (EER) is computed as the primary privacy metric by using language-matched ASV evaluation models.
This metric is calculated under the four attack models shown in the lower left of Fig. 2. The attackers are assumed to have access to a few original or anonymized utterances for each speaker, called enrollment utterances, and to have different levels of knowledge about the SAS: • Unprotected: No anonymization is applied, and attackers verify the original test trials against the original enrollment data by using an ASV system trained on the original dataset, denoted ASV eval. • Ignorant: Attackers are unaware of the anonymization strategy used for the test trial utterances; instead, they use the original enrollment data and ASV eval to infer a speaker's identity. • Lazy-informed: Attackers use a similar SAS without accurate parameters to anonymize their enrollment data, and they use ASV eval to detect a speaker's identity.
First, they use an ASR-based content extractor that requires large amounts of transcribed English training data. Such an ASR-based content extractor is ineffective for speaker anonymization in unseen languages. Our previous work alleviates this issue by using an SSL-based content extractor  [33]. As shown in Fig. 1 , this SSL-based SAS consists of a HuBERT-based soft content encoder  [49] , an ECAPA-TDNN speaker encoder  [29] , an F0 extractor, and a HiFi-GAN decoder  [32]. It does not require text transcriptions or any other language-specific resources, and it has demonstrated the ability to anonymize speech data with reasonable performance even if the data is in a language not included in the training data. However, it suffers from a remaining limitation of selectionbased anonymizers according to previous results  [11] ,  [12] ,  [13] ,  [33] ,  [34] : the distribution of the external speaker pool significantly affects anonymized speakers, and the averaging of vectors from the speaker pool reduces voice distinctiveness.
The outputs may be defined for a multi-speaker classification task in which the original and corresponding anonymized speaker vectors are intentionally treated as different target speaker classes. This means that all speaker vectors after anonymization are treated as different classes, as well as different classes from the original speakers to maintain speaker diversity. Finally, D p x o , p fΘ(x o ) is the divergence between distributions of x included in a training database before and after anonymization. This term ensures similarity between the distributions of the anonymized and original speaker vectors, with some tolerance ϵ. The Kullback-Leibler divergence (KLD) or other types of divergence are applicable.
IV. EVALUATION To evaluate the effectiveness of the SSL-based SAS using the proposed OHNN-based anonymizer under all the attack scenarios for English speaker anonymization, we followed the VPC evaluation plan  [11] ,  [12] ,  [13]  described in Section II. Then, we conducted anonymization experiments under a language-mismatched condition, using Mandarin data as the non-included language in the training database. The purpose of these experiments was to determine whether that the proposed OHNN-based anonymizer, which eliminates the need for an English speaker pool, can effectively reduce the language mismatch present in anonymized speaker representations. As a result, better speech content preservation is achieved for Mandarin speaker anonymization. A. Speaker Anonymization Dataset and Experimental Setup 1) Dataset: The SSL-based SAS was built using the following VPC standard datasets  [11] : an ECAPA-TDNN speaker encoder trained on the VoxCeleb-2  [56] ; a HuBERT-based soft content encoder finetuned from a pretrained HuBERT Base model 5 on LibriTTS-train-clean-100  [57]  to capture content representations; and a HiFi-GAN model trained on LibriTTStrain-clean-100  [57].
Unlike the selection-based anonymizer, which relies on an additional multi-speaker English dataset (LibriTTS-trainother-500) containing data from 1,160 speakers as the external pool, the OHNN-based anonymizers reuse a multi-speaker multi-language dataset (VoxCeleb-2), that is used to train the ECAPA-TDNN of the SSL-based SAS  [33]. This large-scale dataset contains over 1 million utterances by 5,994 speakers of 145 different nationalities. English speaker anonymization was evaluated on the official VPC development and test sets  [11] ,  [12] ,  [13]. These two sets contain English utterances by several female and male speakers from the LibriSpeech and VCTK  [58]  corpora. For the Ignorant and Lazy-informed conditions, we used the languagematched ASV eval system provided by the VPC  [11] ,  [12] ,  [13].
It was trained on the original LibriSpeech-train-clean-360 English dataset. For the Semi-informed condition, we trained ASV anon eval system in the same way as ASV eval , but with anonymized speech data. Likewise, ASR eval and ASR anon eval were trained with the same original and anonymized speech data, respectively. The same anonymization systems used for English speakers were directly adopted for Mandarin speaker anonymization without training or fine-tuning on Mandarin data. The evaluation for Mandarin was conducted on a test set sampled from    [60] ,  [61]. The ASV evaluation model under the Semi-informed condition called ASV anon mand eval was finetuned from ASV mand eval using anonymized utterances from 285 speakers in the interview, speech and live broadcasting genres of CN-Celeb-1 & 2.
The ASR evaluation model ASR mand was a publicly available ASR Transformer  [62]  trained on a 150hour Mandarin ASR dataset, AISHELL-1  [63]. 2) Experimental Setup: Table  I  lists notations for the different speaker anonymization approaches that we examined. B1. a, B1. b, and B2 are the baseline systems from VPC 2022  [13]. S-Select denotes the SSL-based SAS using a selectionbased anonymizer. S-ROH denotes a system obtained by replacing the selection-based anonymizer of S-Select with a random OH (ROH) anonymizer and keeping other components unchanged. Likewise, S-LOH indicates the use of a learnable OH (LOH) anonymizer. Noted that, hereafter, S-ROH* and S-LOH* refer to models trained with the w-AAM and cosine similarity losses. For S-Select, the YAAPT algorithm  [64]  is used to extract the F0.
The OHNN-based anonymizer accepts 192-dimensional speaker representations extracted from a pretrained ECAPA-TDNN, which was the same here as the ECAPA-TDNN of the SSL-based SAS. We followed the VPC evaluation plan, in which attackers in the Lazy-informed and Semi-informed scenarios have partial knowledge of the speaker anonymizer. They are assumed to know the training dataset, structure, loss functions, and other training parameters of the user's OHNNbased anonymizer, except for the training seed to initialize the training weights. Specifically, the training seeds were 50 and 1986 for users and attackers, respectively  7. Using knowledge of the OHNN-based anonymizer, an attacker trains a new anonymizer to anonymize speech. All the OHNN-based anonymizers were trained with a cyclical learning rate  [67] , which varied between 1e-8 and 1e-3, and the Adam optimizer  [68]  by using the SpeechBrain  [62]  toolkit based on PyTorch  [69].
Cosine similarity distribution on speech pairs: Fig. 6  plots the cosine similarities between pairs of speaker vectors extracted from generated speech for all the test sets of LibriSpeech and VCTK on speech pairs provided by  [12]. Depending on the attack condition, the speech can be original or anonymized generated by S-Select or S-LOH*. For the Unprotected condition, shown on the left side of Fig. 6 , the positive cosine similarity distributions (green) are close to 1, and the negative distributions (yellow) are close to 0, which indicates that the speaker vectors of the original speech were highly discriminative. To protect speaker privacy, an ideal SAS should push the positive score distributions toward the negative ones regardless of the attacker type. On the right side of Fig. 6 , the top part shows the score distributions for three attacker conditions with S-Select.
C. Speaker Anonymization Experiments in Mandarin Table IV lists the EERs and CERs for the Mandarin test dataset. The first observation is that baselines B1. a and B2 obtained EERs higher than 30% under the three conditions, but the CERs were higher than 60%. These results indicate that both systems achieved a high level of speaker identity protection by heavily distorting the speech contents. In particular, the results of B1. a suggest that it was inappropriate to use the ASR AM trained on the English data to extract speech content from the Mandarin data. The second observation is that the trends for the S-Select and OHNN-based anonymizers with different losses were remarkably similar to those observed on the English test sets. The proposed OHNN-based anonymizers obtained ASV EERs higher than 30% under all evaluation conditions, and the CERs were lower than those of other systems.
Compared to the baselines, the proposed systems adequately protected the speaker information without heavily sacrificing the speech contents. Compared to the selection-based system, the proposed system[s?] achieved a lower CER while obtaining much higher ASV EERs, particularly in the most challenging Lazyinformed and Semi-informed scenarios. In particular, the CER on the anonymized speech decreased to less than 18% with the OHNN-based anonymizers, suggesting improved utility. One possible reason for the decreased CER when using OHNNbased anonymizers is that this mismatch was mitigated by the OHNN-based anonymizers trained using VoxCeleb 2, which contains large-scale, multi-speaker, and multi-language data.
V. CONCLUSIONS This paper has proposed a novel OHNN-based speaker anonymization approach that rotates original speaker vectors into anonymized ones with a distribution following the original speaker vector space. Towards good privacy protection and voice distinctiveness, AAM/w-AAM and cosine similarity loss functions were introduced to encourage the generation of distinctive anonymized speaker vectors. Experiments on English VPC datasets demonstrated that the proposed model protects speaker privacy while maintaining speech content: it achieved competitive performance under all attack scenarios in terms of privacy and utility metrics. Comparison of the cosine similarities between pairs of speaker vectors extracted from the generated speech with a commonly used selectionbased anonymizer and the OHNN-based anonymizer further verified that our proposed method effectively reduce privacy leakage when facing different attackers, while improving the diversity of anonymized speakers.
Experiments on the Mandarin AISHELL-3 datasets demonstrated that our OHNNbased anonymizer is more robust to the language mismatch scenario than the selection-based methods and can be adopted for this unseen-language anonymization task directly. To further improve the privacy protection performance under various attack scenarios, our future work will investigate the training loss. One potential direction is to optimize the distance between the original and anonymized speaker vectors by integrating a proxy ASV evaluation model into the training process i. e. , using an ASV to measure L s in Eq. (  5 ) on original and anonymized speech waveforms. Such a training scheme is closer to how attackers infringe on the speaker's identity. Additionally, we are considering extending the OHNN-based anonymizer to protect other personal attributes such as age, gender, emotion, and dialect.
This study is supported by  JST CREST Grants  ( JPMJCR18A6  and  JP-MJCR20D3 ),  MEXT KAKENHI Grants  ( 21K17775 ,  21H04906 ,  21K11951 ,  22K21319 ), and the  VoicePersonal  project ( ANR-18-JSTS-0001 ).
Audio samples are available at https://github. com/nii-yamagishilab/ SSL-SAS This article has been accepted for publication in IEEE/ACM Transactions on Audio, Speech and Language Processing. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10. 1109/TASLP. 2023. 3313429 This work is licensed under a Creative Commons Attribution 4. 0 License. For more information, see https://creativecommons. org/licenses/by/4. 0/.
Study Design The period from spring to summer is a crucial time for museum studies involving actual visitors, as tourism is highly concentrated during this period. The before-restoration phase of our study was conducted in July and August 2018 during European summer holidays. The after-restoration phase was conducted during the same period in 2022. The four-year interval between the two phases of the study was due to the time required for the Isenheim Altarpiece to be fully restored. Intermediary data collection was also conducted during the restoration in November 2019, during which the unvarnished panels were analyzed. However, this paper only presents the results of the before-and after-restoration phases to allow for a more relevant comparison between the two main restoration phases. The exhibition chapel at the Unterlinden Museum has an artificial electric lighting system that faces the altarpiece directly and is used throughout the year to ensure constant illumination.
Throughout the study, visitors remained alone, and we monitored each visit from a distance using a tablet equipped with Tobii Glass Controller software  (version 1. 114. 20033 ). After completing their painting observations, the participants were asked to return to the museum's chapel entrance without removing their eye-tracking glasses. In the third step of the experiment, we invited the participants to one-to-one qualitative interviews, in which they were encouraged to discuss their visual experiences while looking at the paintings freely. Interviews were recorded using a microphone attached to the eye-tracking glasses. The participants were instructed to describe what elicited their greatest attention while observing the paintings. In the fourth step, we asked the participants to exit the museum's chapel, remove the eye-tracking glasses, and complete a brief questionnaire.
Data Analysis The evolution of head-mounted eye-tracking technology has enabled the proper and practical use of mobile eye-tracking in museums  (Santini et al. , 2018). To ensure data quality, we used the open-access software GlassesViewer  (Niehorster et al. , 2020)  to analyze the Tobii Pro Glasses 2 eye-tracker data. The mean precision was 1. 74° (SD = 1. 24°), and the data loss was 18%. According to the manufacturer's instructions, the accuracy was 3. 05°. We used Tobii's fixation filter [Tobii I-VT (Fixation)] with the gaze velocity threshold set at 100°/s to determine whether the data should be classified as fixation. Data with a velocity above the threshold were classified as saccades, and data with velocities below the threshold were considered fixations. Heat Maps were created based on the absolute Fixation Count.
We processed the eye-tracking data using Tobii Pro Lab version 1. 152. 30002 (x64). Gaze-positioning data were automatically mapped onto pictures of the paintings using Tobii's algorithm for real-world mapping, which compares static snapshots of the paintings with picture frames in the recording from the eye-tracking device. Our data cleaning process included manual frame-byframe correction of the automatically mapped data through visual assessment within the same Tobii Pro Lab software. The data output (. xlsx files) was processed using RStudio. Similarly, the questionnaires were converted to. xlsx files and analyzed using Microsoft Excel. We used VoyantTools -a web-based, open-source reading and analysis environment for digital texts -to analyze the interview texts (http://voyant -tools.
Although specialized literature suggests that the size of AOIs should cover the entire shape of the figures  (Holmqvist et al. , 2011, pp. 218-220) , we decided to frame only the segments judged to be the most important for each target object. We positioned the different AOIs such that they did not overlap. To ensure that fixations directed to one area were not detected in an adjacent area, we set a minimum space corresponding to the accuracy rate between the AOIs (M = 1. 74°, SD = 1. 24°; M = 14 pixels², SD = 6. 75 pixels²). We added an additional margin of 3° (i. e. , double the average precision rate) to the space between the different AOIs to ensure that fixations from different areas did not mix. We designed the AOIs using the Tobii Pro Lab software. Figure  A1  in Appendix A2 presents the AOIs and describes the elements of interest for each.
Participants Of the 126 participants who completed the protocol, 40 were excluded from the analysis because of insufficient data quality (precision less than 1. 74°, SD = 1. 24°), as described in Section 3. 2. Data Analysis. Thus, we retained the recordings of 86 volunteers whose data sets were complete and of good quality. Half of the participants (n = 43) participated in the study before restoration, and the other half participated afterward (n = 43). In both the before-and after-restoration phases of the study, the participants' profiles were comparable in terms of age (M Before = 38 years, SD = 19; M After = 40 years, SD = 20) and country of origin (before: 70% European, 26% from the Americas, 2% Asian, and 2% from Oceania; after: 79% European, 16% from the Americas, 5% Asian, and 0% from Oceania).
Following restoration, the data showed a significant increase in the Relative Fixation Count, indicating that the panels received more fixations in general during visits, possibly owing to the enhanced brightness resulting from the restoration procedure. Moreover, the expanded salient zones on Fixation Count Heat Maps indicated that people adopted a more exploratory approach to the images after restoration. These findings aligned with those reported by  Locher et al. (2020)  and, more specifically, the investigation by Quian Quiroga et al. (2011) , which revealed a broader visual exploration of images in museum observations, particularly in regions of interest surrounding characters. Nonetheless, in our study, visual exploration also increased in areas pertaining to objects and minutiae in the backgrounds of the paintings.
The significant decrease found in the mean values for Time to First Fixation within AOIs after restoration suggested that visitors could quickly identify and focus on critical areas of paintings where major restoration effects were expected to be perceived. Furthermore, the larger observed areas identified in the Heat Maps reinforced the assumption that restoration can reveal a significant amount of previously concealed information. We hypothesized that people's visual behavior in a museum would change after a painting's restoration. Although some of our eye-tracking results supported this hypothesis, not all parameters aligned with those observed by  Locher et al. (2020). The implications of these changes for the viewer's conscious perception cannot be assessed with eye tracking alone. Consciousness is a complex phenomenological process that can be preliminarily estimated through qualitative results and by associating verbal reports  (Ericsson and Simon, 1980)  with eye-movement data  (Bauer and Schwan, 2018; Locher et al.
Acknowledgements We would like to express our gratitude to  Paris Sciences et Lettres -PSL Research University  and its Interdisciplinary and  Strategic Research Initiative (IRIS) -Creation, Cognition, Society (CCS) , the  SublimAE  project  -Le sublime et les expériences esthétiques  ( ANR-18-CE27-0023 ) and the  Patrimoniochromies program of LabEX Les Passés dans le Présent  ( 11-LABX-0026 ) for funding this research. We also express our sincere gratitude to the  Unterlinden Museum  and its visitors, who generously participated in our research project. Their cooperation was essential to the successful completion of this study. Finally, we would like to convey our profound appreciation to  Thérèse Collins ,  Pantxika De Paepe ,  Nathalie Delbard ,  Jérôme Dokic ,  Michel Paysant ,  Raphael Rosenberg ,  Carole Talon-Hugon ,  Romain Thomas , our colleagues at the  C2RMF , the CRAL, the  CReA Lab , and all others who have made direct or indirect contributions to this project.