* Add stream_return switch to get wavs from yield
* Add more_segment_before arg for more segmenting.
more_segment_before is a int, for token_index < more_segment_before, more segmenting will be applied.
0: no effect; 80 is recommended for better first-wav-latency
* Uncomment silence insertion
* fix: rename quick streaming tokens argument
* fix: rename quick streaming tokens argument
* fix: Add a wrapper for the yield function. It will not return a generator in normal condition.
- The WebUI was secretly squashing all emotion vectors and re-scaling them. It's a good idea for user friendliness, but it makes it harder to learn what values will work in Python when using the WebUI for testing.
- Instead, let's move the normalization code into IndexTTS2 as a helper function which is used by Gradio and can be used from other people's code too.
- The emotion bias (which reduces the influence of certain emotions) has also been converted into an optional feature, which can be turned off if such biasing isn't wanted. And all biasing values have been re-scaled to use 1.0 as the reference, to avoid scaling relative to 0.8 (which previously meant that it applied double scaling).
- Added support for `emo_alpha` scaling of emotion vectors and emotion text inputs.
- This is a major new feature, which now allows for much more natural speech generation by lowering the influence of the emotion vector/text control modes.
- It is particularly useful for the "emotion text description" control mode, where a strength of 0.6 or lower is useful to get much more natural speech. Before this feature, it was not possible to make natural speech with that mode, because QwenEmotion assigns emotion scores to the text from 0.0-1.0, and that score was used directly as an emotion vector. This meant that the text mode always used very high strengths. Now, the user can adjust the strength of the emotions to get very natural results.
- Refactored `IndexTTS2.infer()` variable initialization logic to avoid repetition and ensure cleaner code paths.
- A recent change made DeepSpeed optional (off by default), but the code was still trying to load DeepSpeed even when `use_deepspeed = False`. This means users would still have a big startup slowdown and a lot of error messages if their DeepSpeed module isn't working (usually because it's not able to compile itself on their machines).
- We now only load DeepSpeed if the user requested it.
- Translated the DeepSpeed error message to English, since all other errors in the same function were already English.
* fix: Configure "uv" build system to use CUDA on supported platforms
- Linux builds of PyTorch always have CUDA acceleration built-in, but Windows only has it if we request a CUDA build.
- The built-in CUDA on Linux uses old libraries and can be slow.
- We now request PyTorch built for the most modern CUDA Toolkit on Linux + Windows, to solve both problems.
- Mac uses PyTorch without CUDA support, since it doesn't exist on that platform.
- Other dependencies have received new releases and are included in this fix too:
* click was downgraded because the author revoked 8.2.2 due to a bug.
* wetext received a new release now.
* fix: Use PyPI as the hashing reference in "uv" lockfile
- PyPI is the most trustworthy source for package hashes. We need to remove the custom mirror from the config, otherwise that mirror always becomes the default lockfile/package source, which leads to user trust issues and package impersonation risks.
- Regional mirrors should be added by users during installation instead, via the `uv sync --default-index` flag. Documented with example for Chinese mirror.
- When users add `--default-index`, "uv" will try to discover the exact same packages via the mirror to improve download speeds, but automatically uses PyPI if the mirror didn't have the files or if the mirror's file hashes were incorrect. Thus ensuring that users always have the correct package files.
* docs: Improve README for IndexTTS2 release!
- "Abstract" separated into paragraphs for easier readability.
- Clearer document structure and many grammatical improvements.
- More emojis, to make it easier to find sections when scrolling through the page!
- Added missing instructions:
* Needing `git-lfs` to clone the code.
* Needing CUDA Toolkit to install the dependencies.
* How to install the `hf` or `modelscope` CLI tools to download the models.
- Made our web demo the first section within "quickstart", to give users a quick, fun demo to start experimenting with.
- Fixed a bug in the "PYTHONPATH" recommendation. It must be enclosed in quotes `""`, otherwise the new path would break on systems that had spaces in their original path.
- Improved all Python code-example descriptions to make them much easier to understand.
- Clearly marked the IndexTTS1 legacy section as "legacy" to avoid confusion.
- Removed outdated Windows "conda/pip" instruction which is no longer relevant since we use "uv" now.
* refactor(webui): Remove unused imports
The old IndexTTS1 module and ModelScope were being loaded even though we don't need them. They also have a lot of dependencies, which slowed down loading and could even cause some conflicts.
* feat!: Remove obsolete build system (setup.py)
BREAKING CHANGE: The `setup.py` file has been removed.
Users should now use the new `pyproject.toml` based "uv" build system for installing and developing the project.
* feat: Add support for installing IndexTTS as a CLI tool
- We now support installing as a CLI tool via "uv".
- Uses the modern "hatchling" as the package / CLI build system.
- The `cli.py` code is currently outdated (doesn't support IndexTTS2). Marking as a TODO.
* chore: Add authors and classifiers metadata to pyproject.toml
* feat: Faster installs by making WebUI dependencies optional
* refactor!: Rename "sentences" to "segments" for clarity
- When we are splitting text into generation chunks, we are *not* creating "sentences". We are creating "segments". Because a *sentence* must always end with punctuation (".!?" etc). A *segment* can be a small fragment of a sentence, without any punctuation, so it's not accurate (and was very misleading) to use the word "sentences".
- All variables, function calls and strings have been carefully analyzed and renamed.
- This change will be part of user-facing code via a new feature, which is why the change was applied to the entire codebase.
- This change also helps future code contributors understand the code.
- All affected features are fully tested and work correctly after this refactoring.
- The `is_fp16` parameter has also been renamed to `use_fp16` since the previous name could confuse people ("is" implies an automatic check, "use" implies a user decision to enable/disable FP16).
- `cli.py`'s "--fp16" default value has been set to False, exactly like the web UI.
- `webui.py`'s "--is_fp16" flag has been changed to "--fp16" for easier usage and consistency with the CLI program, and the help-description has been improved.
* feat(webui): Set "max tokens per generation segment" via CLI flag
- The "Max tokens per generation segment" is a critical setting, as it directly impacts VRAM usage. Since the optimal value varies significantly based on a user's GPU, it is a frequent point of adjustment to prevent out-of-memory issues.
- This change allows the default value to be set via a CLI flag. Users can now conveniently start the web UI with the correct setting for their system, eliminating the need to manually reconfigure the value on every restart.
- The `webui.py -h` help text has also been enhanced to automatically display the default values for all CLI settings.
* refactor(i18n): Improve clarity of all web UI translation strings
* feat(webui): Use main text as emotion guidance when description is empty
If the user selects "text-to-emotion" control, but leaves the emotion description empty, we now automatically use the main text prompt instead.
This ensures that web users can enjoy every feature of IndexTTS2, including the ability to automatically guess the emotion from the main text prompt.
* feat: Add PyTorch GPU acceleration diagnostic tool
* chore: Use NVIDIA CUDA Toolkit v12.8
Downgrade from CUDA 12.9 to 12.8 to simplify user installation, since version 12.8 is very popular.
* docs: Simplify "uv run" command examples
The "uv run" command can take a `.py` file as direct argument and automatically understands that it should run via python.
- The "低落" (melancholic) emotion will always be mapped to "悲伤" (sad) by QwenEmotion's text analysis. It doesn't know the difference between those emotions even if the user writes the exact words.
- Since the words and their meanings are so similar, it might not be possible to train QwenEmotion to learn the difference.
- As a workaround, we perform input text analysis and look for words that mean "melancholic", and swap the "sad" detection result, to make the melancholic/low-energy speech emotion work correctly for users via text-to-emotion.
- The new algorithm is now very fast and uses less memory, since it doesn't chain multiple `.replace()` calls or create a bunch of temporary strings and temporary dictionaries and lists anymore.
- Parses the JSON output from the QwenEmotion model directly instead of trying to manually parse it. If JSON parsing fails, it falls back to a fast and highly-accurate RegEx search which finds all key-value pairs.
- The desired emotion vector order is now stored as a static class attribute instead of being created from scratch on every call.
- The emotion dictionary creation has been completely rewritten to use a clear algorithm which takes the QwenEmotion answers, builds a new dictionary using `self.desired_vector_order`, maps each key's name to their English translations, fetches the values from QwenEmotion's answers or 0.0 if no value was given by QE, and clamps the values to the min/max ranges.
- The `backup_dict` is now removed, since it was error-prone and fragile. It could grow out of sync with the code if not carefully maintained to keep the correct order and labels.
- To handle the "fallback" dictionary creation, we now automatically scan the final emotion vectors, and if none of them are above 0.0 (meaning we didn't detect any emotions in the input text), we give the final vectors a "calm: 1.0" value. This means that we never have to worry about the fallback dictionary's correctness.
- The previous algorithm had multiple bugs. This rewrite fixes a serious vector order bug: The old algorithm built the dictionary via the found keys, and only checked if there's 8 keys in QwenEmotion's response, but it didn't check that the keys were valid. When building the final emotion dict, it skipped any values if they were not found in QE's response. Meaning that if the QE response only contained 4 of the 8 expected emotion vector labels, those would all be added at the start of the new dictionary as the "first 4 dict slots". After that, it looped through the "backup_dict" and appended any missing values at the end. This resulted in a final emotion dictionary with the wrong order for the emotion vectors. The new code always produces the correct emotion vector order.
- Discovered another bug in the text-to-emotion handling for the "melancholic" emotion, which has never worked for Chinese or English at all. It will be fixed in an upcoming patch.
- The order of the `convert_dict` now matches the desired order of the emotion vectors, for clarity.
- Internal text labels now match the updated English translations.
- This (and the previous commit) also fixes a bug: The previous, inaccurate Emotion translations meant that QwenEmotion could not understand words such as "low" at all (no emotion mapping), and it always mapped "hate" to "angry". With the fixed translations, QwenEmotion now correctly maps text-to-emotions from English inputs when users input the words that they've been taught by the user interface.
- Introduced `de_tokenized_by_CJK_char` for restoring original text from tokenized format.
- Added `TextTokenizer` class for improved tokenization, including sentence splitting and handling of special tokens.
- Enhanced `TextNormalizer` to handle names and pinyin tones with placeholder mechanisms.
- Added regression tests for new features in `regression_test.py`.
