BTS-TTS:
Blockwise Temporal Specializaiton for Flow Matching-Based Text-to-Speech
Gwangju Institute of Science and Technology
Abstract
Flow matching-based text-to-speech systems typically employ a single vector field estimator over the entire flow trajectory. However, as intermediate states evolve from noise-dominated samples to structured acoustic representations, a single shared estimator must model the vector field across substantially changing state distributions. To address this limitation, we propose the blockwise temporal specialization for flow matching-based text-to-speech (BTS-TTS), which partitions the flow trajectory into temporal blocks and assigns an independent Diffusion Transformer estimator to each block. During inference, only the estimator associated with the current block is evaluated, preserving both the number of function evaluations and the per-step computational complexity of the corresponding single-estimator model. Experiments on the LJSpeech and VCTK corpora showed that the BTS-TTS improved the quality of the synthesized speech compared with single-estimator counterparts while maintaining the same inference complexity. Furthermore, adversarial fine-tuning significantly enhanced perceptual speech quality, achieving superior subjective preference compared to the RapFlow-TTS.
Model Architecture
Evaluation
Sample Comparison
† indicates the results with the "improved techniques" in RapFlow-TTS.
‡ indicates the results with the adversarial fine-tuning in BTS-TTS.