12 Commits

Author SHA1 Message Date
yrom
d3bd7eb8b2 Fix split_sentences_by_token 2025-04-24 23:58:16 +08:00
Yrom
475fb12574
Fix pinyin correction 2025-04-24 20:38:52 +08:00
Yrom
35b6514ee5
Enhance text normalization and tokenization
- Introduced `de_tokenized_by_CJK_char` for restoring original text from tokenized format.
- Added `TextTokenizer` class for improved tokenization, including sentence splitting and handling of special tokens.
- Enhanced `TextNormalizer` to handle names and pinyin tones with placeholder mechanisms.
- Added regression tests for new features in `regression_test.py`.
2025-04-24 20:28:44 +08:00
Yrom Wang
18c32c06b1
修复拼音问题和分句问题,支持轻音声调(如yi1 shang5) (#83)
* Update Pinyin tone handling in TextNormalizer

* Enhance sentence splitting and improve tokenizer integration in inference

* Update character replacement mappings

test: "在电影《肖申克的救赎》中,安迪·杜佛兰被错误地判处终身监禁..."

* Refactor TextNormalizer and enhance testing with additional cases
2025-04-14 19:50:36 +08:00
root
18e20ccbb4 enable front-end caching to speed up startup. 2025-04-09 10:35:47 +08:00
kemuriririn
1734698f3d
一些输入文本处理的fix (#18)
* +简单前端

* 前端兼容arm机器

* fix

* fix

* WeTextProcessing: overwrite_cache=True 刷新前端缓存

* 恢复输入中的拼音

---------
Co-authored-by: kemuriririn <10inspiral@gmail.com>
2025-03-27 14:12:12 +08:00
kemuriririn
c73344ecc9
集成简单前端 (#15)
* +简单前端

* 前端兼容arm机器

* fix

* fix

---------

Co-authored-by: wangyining02 <wangyining02@bilibili.com>
2025-03-26 19:39:08 +08:00
wangyining02
de60f6829b Merge branch 'main' of github.com:eschmidbauer/index-tts into eschmidbauer-main 2025-03-26 12:46:19 +08:00
wangyining02
8031b5d654 fix import error in feature_extractors.py 2025-03-26 12:19:57 +08:00
wangyining02
b591e84bf9 rename utils.utils to utils.common 2025-03-26 12:15:48 +08:00
Emmanuel Schmidbauer
2fe6a73ada fix packages 2025-03-25 14:03:29 -04:00
wangyining02
8db92eda8c init infer code 2025-03-25 12:52:52 +08:00