Imagine you’ve been handed a box of chocolates, but some of them are artificial replicas instead of the real thing. In a similar way, this study explores how our brains respond to synthetic voices that try to imitate human emotions. The researchers focused on determining what makes a voice sound natural and then created altered versions with reduced naturalness. They used special brain scans called Event-Related Potentials (ERPs) to track how people’s brains reacted to these different voices. Additionally, participants rated the voices on various aspects like emotional intensity and how natural they sounded. The results showed that as the naturalness of the voices decreased, both behavioral and brain responses were affected. People had more difficulty recognizing specific emotions, and their perception of naturalness and intelligibility decreased. However, overall emotional valence and arousal remained fairly consistent, regardless of naturalness. Interestingly, certain patterns in brain activity were also related to these emotional perceptions. Ultimately, this study sheds light on the intricate relationship between our brains and synthetic emotional voices, providing valuable insights into the future development of more authentic artificial voices. Check out the link below for more details!

Artificial voices are nowadays embedded into our daily lives with latest neural voices approaching human voice consistency (naturalness). Nevertheless, behavioral, and neuronal correlates of the perception of less naturalistic emotional prosodies are still misunderstood. In this study, we explored the acoustic tendencies that define naturalness from human to synthesized voices. Then, we created naturalness-reduced emotional utterances by acoustic editions of human voices. Finally, we used Event-Related Potentials (ERP) to assess the time dynamics of emotional integration when listening to both human and synthesized voices in a healthy adult sample. Additionally, listeners rated their perceptions for valence, arousal, discrete emotions, naturalness, and intelligibility. Synthesized voices were characterized by less lexical stress (i.e., reduced difference between stressed and unstressed syllables within words) as regards duration and median pitch modulations. Besides, spectral content was attenuated toward lower F2 and F3 frequencies and lower intensities for harmonics 1 and 4. Both psychometric and neuronal correlates were sensitive to naturalness reduction. (1) Naturalness and intelligibility ratings dropped with emotional utterances synthetization, (2) Discrete emotion recognition was impaired as naturalness declined, consistent with P200 and Late Positive Potentials (LPP) being less sensitive to emotional differentiation at lower naturalness, and (3) Relative P200 and LPP amplitudes between prosodies were modulated by synthetization. Nevertheless, (4) Valence and arousal perceptions were preserved at lower naturalness, (5) Valence (arousal) ratings correlated negatively (positively) with Higuchi’s fractal dimension extracted on neuronal data under all naturalness perturbations, (6) Inter-Trial Phase Coherence (ITPC) and standard deviation measurements revealed high inter-individual heterogeneity for emotion perception that is still preserved as naturalness reduces. Notably, partial between-participant synchrony (low ITPC), along with high amplitude dispersion on ERPs at both early and late stages emphasized miscellaneous emotional responses among subjects. In this study, we highlighted for the first time both behavioral and neuronal basis of emotional perception under acoustic naturalness alterations. Partial dependencies between ecological relevance and emotion understanding outlined the modulation but not the annihilation of emotional integration by synthetization.

Read Full Article (External Site)