Acoustic-to-articulatory inversion: speech quality assessment and smoothness constraint

Abstract

The ability of humans to speak effortlessly, require coordinated movements of various articulators, muscles, etc. This effortless movement contributes towards naturalness, intelligibility and speaker identity in human speech, which is only partially present in speech, obtained from most of voice conversion (VC) systems. Hence, during voice conversion, the information related to speech production is lost. For quantification of the loss in information two quantities, i.e., mutual information (I) and estimation error were calculated. In this thesis, the differences in the estimated articulator trajectories are exploited to propose articulatory features based objective measure for assessing the quality of voice conversion. Moreover, a new smoothness criterion, i.e., jerk minimization is explored to deal non-uniqueness of speech inversion mapping. Speech is the result of coordinated movements of the articulators such as lips, tongue, jaw, velum, etc. Therefore, measured trajectories obtained are smooth and slowly varying. However, the trajectories estimated from acoustic-to-articulatory inversion are found to be jagged. Thus, energy minimization is used as smoothness constraint for improving performance of the acoustic-to-articulatory inversion. Moreover, jerk (i.e., rate of change of acceleration) is known for quantification of smoothness in case of human motor movements. This motivates us to propose jerk minimization as the smoothness criteria for frame-based acoustic-to-articulatory inversion.