Article Dans Une Revue Speech Communication Année : 2023

Dynamic Mask based Iterative Adapted Child Speech Separation under Multilingual Real Conditions

Résumé

We develop two improvements over our previously-proposed joint enhancement and separation (JES) framework for child speech extraction in real-world multilingual scenarios. First, we introduce an iterative adaptation based separation (IAS) technique to iteratively fine-tune our pre-trained separation model in JES using data from real scenes to adapt the model. Second, to purify the training data, we propose a dynamic mask separation (DMS) technique with variable lengths in movable windows to locate meaningful speech segments using a scale-invariant signal-to-noise ratio (SI-SNR) objective. With DMS on top of IAS, called DMS+IAS, the combined technique can remove a large number of noise backgrounds and correctly locate speech regions in utterances recorded under real-world scenarios. Evaluated on the BabyTrain corpus, our proposed IAS system achieves consistent extraction performance improvements when compared to our previously-proposed JES framework. Moreover, experimental results also show that the proposed DMS+IAS technique can further improve the quality of separated child speech in real-world scenarios and obtain a relatively good extraction performance in difficult situations where adult speech is mixed with child speech.
Fichier principal
Vignette du fichier
Dynamic_Mask_based_Iterative_Adapted_Child_Speech_Separation_under_Multilingual_Real_Conditions.pdf (14.93 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04353571 , version 1 (18-01-2024)

Identifiants

Citer

Shi Cheng, Jun Du, Shutong Niu, Chin-Hui Lee, Alejandrina Cristia, et al.. Dynamic Mask based Iterative Adapted Child Speech Separation under Multilingual Real Conditions. Speech Communication, 2023, 152, pp.102956. ⟨10.1016/j.specom.2023.102956⟩. ⟨hal-04353571⟩
25 Consultations
114 Téléchargements

Altmetric

Partager

More