# Partial Fake Speech Attacks in the Real World Using Deepfake Audio
## 7 Partial Fake Speech and Human Speech
### User Study B: Can Users Identify Partial Fake Speech (Within Entirely Real Video)  
The objective of the second user study was to determine how humans respond to a new type of deepfake media by embedding partial fake audio within entirely real video. To the best of our knowledge, this was the first time a partial fake audio was placed within an entirely real video and used in a user study, as previous works focused on fake video [70; 71; 72].  
**Participants.** The same participants involved in Section 7.2 answered the questions presented in this section.  
**Procedure.** We generated six videos sourced from the YouTube-8m dataset. We created and embedded partially faked audio segments into entirely real videos. To create the audio, we used the Tortoise zero-shot model for TTS segments and FreeVC for VC speech. We started by extracting the audio from the videos. Then, we generated a cloning voice of the actual speaker's voice that we had extracted from the video. Finally, we replaced a word or a few words of the speech with the cloning voice speech to process the audio.  
As part of our experiment, we embedded partially fake audio into two types of real videos. The first type involved the speaker being in front of the camera, making it possible for the viewers to see the speaker's face and lips while they were speaking. The second type involved the speaker being farther away from the camera, making it impossible for the viewers to see the speaker's lips while the fake audio segments were playing. We believed that it would be easier to identify partial fake speech when the camera was in front of the face, which was why we only created one partial fake segment within a video that was recorded with a camera at a distance. The videos feature three male and three female speakers.  
**Result.** We aimed to provide answers to the questions listed below:  
_(1) How accurately can participants distinguish the partial fake speech within an entirely real video?_ Similar to the experiment presented in Section 7.2, participants found it  
Figure 7: Average response accuracy to 4 PF speech questions.  
challenging to identify when audio was partially faked in a real video. Participants were only able to accurately identify partially fake audio between 11% and 31% of the time. Table 10 presents the detailed results of the experiment. In contrast, participants were able to correctly identify entirely real audio and video 69% of the time. However, they were deceived when entirely faked audio was used in an entirely real video. Correct responses to entirely fake speech were as low as 13%. We used FreeVC voice conversion to generate the fake speech, resulting in a real video with a converted voice from a different speaker, while the content of the speech remained the same.  
**(2) _Does seeing the face and lips during a fake segment make a difference in partial fake identification?_** Table 10 presents the results of the study, which found no significant difference in identifying PF speech within a real video whether the camera was positioned in front of the speaker's face or not. The study showed that when the camera was far from the speaker's face, the accuracy average of identifying PF was only 17%, while the accuracy average improved to 24% when the camera was placed in front of the speaker's face.  
**(3) _Does participant gender affect responses?_** We found that female participants outperformed male by 19% when the camera was in front of the speaker's face, while male outperformed female by 14% when the camera was far from the speaker's face.