From d777b8a0290cced0064eb5f503a25b65b148c1bc Mon Sep 17 00:00:00 2001 From: Arcitec <38923130+Arcitec@users.noreply.github.com> Date: Thu, 11 Sep 2025 07:06:25 +0200 Subject: [PATCH] docs: Add FP16 usage advice for faster inference --- README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index b77a31c..c5076ca 100644 --- a/README.md +++ b/README.md @@ -264,7 +264,10 @@ uv run webui.py -h Have fun! > [!IMPORTANT] -> **DeepSpeed** *may* speed up inference on some systems, but it could also +> It can be very helpful to use **FP16** (half-precision) inference. It is faster +> and uses less VRAM, with a very small quality loss. +> +> **DeepSpeed** *may* also speed up inference on some systems, but it could also > make it slower. The performance impact is highly dependent on your specific > hardware, drivers and operating system. Please try with and without it, > to discover what works best on your personal system.