Phonetic and Prosody-aware Self-supervised Learning Approach for Non-native Fluency Scoring

Clicks: 115
ID: 282566
2023
Free PDF
Article Quality & Performance Metrics
Overall Quality
Not rated
Combines reader engagement with the AI quality analysis. This article has not been analysed, so there is no overall score — reader engagement is measured and shown alongside.
AI Quality Assessment
Not analyzed
Readership in this journal
Steady

Ranked #54 of 803 articles by views in arXiv

Most read Least read

Bar heights use a square-root scale. Only the 120 most-read articles are drawn; the journal has 803 in total.

Mint this article as an NFT
Not yet minted

Create a permanent, verifiable on-chain record of this article on the Scimatic Network. The NFT is held in your Journament account, and you can withdraw it to your own wallet at any time.

5 SUSD one-off · no wallet required
Sign in to mint
Abstract
Speech fluency/disfluency can be evaluated by analyzing a range of phonetic and prosodic features. Deep neural networks are commonly trained to map fluency-related features into the human scores. However, the effectiveness of deep learning-based models is constrained by the limited amount of labeled training samples. To address this, we introduce a self-supervised learning (SSL) approach that takes into account phonetic and prosody awareness for fluency scoring. Specifically, we first pre-train the model using a reconstruction loss function, by masking phones and their durations jointly on a large amount of unlabeled speech and text prompts. We then fine-tune the pre-trained model using human-annotated scoring data. Our experimental results, conducted on datasets such as Speechocean762 and our non-native datasets, show that our proposed method outperforms the baseline systems in terms of Pearson correlation coefficients (PCC). Moreover, we also conduct an ablation study to better understand the contribution of phonetic and prosody factors during the pre-training stage.
Reference Key
ma2023phonetic Use this key to autocite in the manuscript while using SciMatic Manuscript Manager or Thesis Manager
Authors Kaiqi Fu; Shaojun Gao; Shuju Shi; Xiaohai Tian; Wei Li; Zejun Ma
Journal arXiv
Year 2023
DOI
DOI not found
URL
http://arxiv.org/abs/2305.11438v1
Keywords
cs.cl eess.as

Citations

No citations found. To add a citation, contact the admin at info@scimatic.org

Comments

No comments yet. Be the first to comment on this article.