BTS-TTS:
Blockwise Temporal Specializaiton for Flow Matching-Based Text-to-Speech

Jongwook Chae* Jongmoon Ryu* Seonggyu Lee Jong Won Shin

Gwangju Institute of Science and Technology

* Equal contribution.

Abstract

Flow matching-based text-to-speech systems typically employ a single vector field estimator over the entire flow trajectory. However, as intermediate states evolve from noise-dominated samples to structured acoustic representations, a single shared estimator must model the vector field across substantially changing state distributions. To address this limitation, we propose the blockwise temporal specialization for flow matching-based text-to-speech (BTS-TTS), which partitions the flow trajectory into temporal blocks and assigns an independent Diffusion Transformer estimator to each block. During inference, only the estimator associated with the current block is evaluated, preserving both the number of function evaluations and the per-step computational complexity of the corresponding single-estimator model. Experiments on the LJSpeech and VCTK corpora showed that the BTS-TTS improved the quality of the synthesized speech compared with single-estimator counterparts while maintaining the same inference complexity. Furthermore, adversarial fine-tuning significantly enhanced perceptual speech quality, achieving superior subjective preference compared to the RapFlow-TTS.

Model Architecture

Architecture diagram of BTS-TTS
Figure 1. Overview of the BTS-TTS architecture. The text frontend produces an acoustic prior and duration estimates. Conditioned on the acoustic prior, the flow prediction network uses Diffusion Transformer blocks to predict a vector field. ODE integration using this vector field progressively transforms Gaussian noise into an acoustic representation.

Evaluation

Sample Comparison

† indicates the results with the "improved techniques" in RapFlow-TTS.

‡ indicates the results with the adversarial fine-tuning in BTS-TTS.

Show spectrogram