# Authenticity at Risk: Key Factors in the Generation and Detection of Audio Deepfakes +
## 2 Related Work  
Methods for generating audio include Text-To-Speech (TTS) synthesis and voice conversion (impersonation), as well as non-ML-generated audio (_replay attacks_), which involve replaying a victim's voice recordings. While TTS converts text to speech using linguistic rules [7], voice conversion modifies the voice signal to sound like another person [8]. Current voice cloning methods can learn speech features from very few samples, producing speech that is almost indistinguishable from the original.  
In 2016 _The Voice Conversion Challenge 2016_[9] was released to compare different voice conversion techniques using a common dataset. The task to be solved was to transform the voice identity of a source speaker into that of a target speaker while preserving the linguistic content. Both speech conversion systems and TTS have advanced to the present day with approaches such as _Deep Voice 3_[10], _Fastspech 2_[11] or _Cyclegan-vc2_[12]. Among them, _StarGANvc-v2_[13], a non-parallel and many-to-many speech conversion method that uses a generative adversarial network (GAN), is trained with only 20 English speakers, and outperforms previous models by being able to perform cross-language conversions, not requiring text labels and using Parallel WaveGAN [14] as a vocoder. These advances have led to the creation of numerous deepfake datasets such as _H-voice_[15], _FakeAVCeleb_[16], or _Half-truth Speech_[17], a dataset that used a text-to-speech model to generate and replace various words in audio clips.  
The risk arises when, as the technology for generating audio deepfakes becomes more complex and accessible, non-experts can easily create fake audio. While this technology can be used for legitimate purposes, it can also be misused for disinformation and fraud [18; 19; 20]. To mitigate this impact, recent research has explored techniques for detecting audio deepfakes, a task also addressed in challenges such as _ADD challenge 2022_[21] which sought to differentiate between various types of audio deepfakes and in _ADD challenge 2023_[22] that reached further than the previous binary classification of real/fake, to locate manipulated intervals in partially fake speech, as well as to identify the source that generated the fake audio.  
For the detection of these deepfakes, different techniques have been tested such as models based on logistic regression [23], quadratic SVMs [24], Res2 Net [25], convolutional recurrent neural networks [26], EfficientCNN [27] or Deep4 SNet, a model designed by [28] that seeks to turn the problem of fake audio detection into a computer vision task, by transforming audio data into the corresponding histogram images.  
Some recent studies highlighting the scarcity of research focused on general audio, which encompasses all types of auditory content, including ambient sounds or speech, among others. In the absence of specific approaches in this field, the _FakeSound_ dataset was developed [29]. For its creation, an audio-text model [30] was initially used to identify key audio segments whose modification could have a considerable impact. These segmentswere then masked and regenerated using generation models such as _AudioSR_[31], integrating them back into the original audio. To generate this dataset, they used _AudioCaps_[32] where they generated subtitles for audios in the wild.  
However, to our knowledge, the generation and detection of such audios in real uncontrolled environments that were weakly labeled, as in our dataset, had not been previously investigated further than our preliminary results in [1] that we further develop in this paper.