## 1 Introduction  
Since 2022, artificial intelligence (AI) and the metaverse have been included as part of South Korea's top 10 key issues identified by the Institute for Information & Communication Technology Planning and Evaluation. By 2025, artificial intelligence, the metaverse, and humanoids are expected to emerge as major topics of focus. The metaverse requires communication within virtual environments, while humanoids necessitate interactions with humans. These interactions, whether between users or between robots and users, emphasize the importance of emotional expression as a crucial element for enhancing a sense of realism. To develop these techniques, researchers must identify and implement methods for recognizing users' emotions. Emotion refers to an organism's response to a specific stimulus (e.g., a person, a situation, or an event) or a complex psychological experience that arises in humans from sensations and perceptions caused by external stimuli [1]. To recognize emotions, various vital signs, such as facial expressions, speech, brain waves, heart rates, and breathing rates, can be analyzed. Particularly, speech, which is a key element of interaction for expressing emotions in communication, plays a crucial role in recognizing intrinsic emotions.  
Speech serves as a channel for communication and emotional expression, reflecting the speaker's psychological state [2]. There are various methods employed to recognize emotions based on speech, including interpreting word meanings, analyzing voice pitch and loudness, considering prosodic information, and more [2]. Among these, emotion analysis based on word meanings is significant as it facilitates the understanding of emotions through semantic interpretation. Additionally, frequency features of speech, such as pitch, intensity, and spectral energy, are valuable for capturing the vibrancy and diversity of emotions, providing a complementary perspective. In other words, the same sentence can have different meanings depending on the tone and intensity of speech [3]. For instance, \"You are very good at it\" has a complimentary meaning when pronounced in a positive tone (reflecting high valence emotions, such as happiness or enthusiasm [4]) and high voice intensity. On the other hand, it can be used as an ironic expression when pronounced in a negative tone (reflecting low valence emotions, such as sadness or displeasure [4]) and low voice intensity. Therefore, emotion recognition techniques should comprehensively consider voice pitch, loudness, and prosodic information. Despite ongoing research and advancements in the field, SER techniques exhibit a low emotion recognition rate compared to other emotion recognition techniques, such as those based on facial expressions [2]. This issue can be attributed to the lack of speech-emotion recognition databases (DBs) reflecting social and cultural elements (e.g., dialects, regional intonations) [5].  
Insufficient DBs may lead to overfitting, where the model memorizes patterns specific to the limited training data, thereby reducing its performance on unseen data and hindering its ability to generalize effectively [6]. To address these problems, data augmentation--a technique that artificially expands a dataset by adjusting the position, orientation, color, saturation, brightness, etc. of an image--can be employed. By applying data augmentation techniques, SER models can overcome the limitations posed by insufficient DBs, ensuring diversity in data.  
This study investigates SER methods based on voice tone, loudness, and prosodic cues. Additionally, it introduces a method that enhances SER accuracy through data augmentation, focusing on identifying the most effective data augmentation technique for identifying speech characteristics.