Diffusers Video Helper — Open Skill

Diffusers Video Helper

Diffusers Video Helper is a public Open Skill on Digen. Open it to see what this workflow creates.

@rmworldof

Anime

2 1 4

https://d7qlwi7tl2u27.cloudfront.net/skill-covers/prod/3762060/2026-09-27/3a74zgbf.jpg

# Diffusers Video Helper

## When to use
- The user asks for a Python script/helper to generate short AI videos locally using open-source models (Wan, LTX, Hunyuan, CogVideoX) with the Diffusers library.
- The user wants text-to-video (T2V) and/or image-to-video (I2V) in one script, with a `--backend` flag to choose the model.
- The user wants clear VRAM-management and quantization (fp8 / GGUF) guidance baked into the code.
- The user wants to modify or extend an existing version of this helper (add a backend, change defaults, add audio, etc.).

This skill only produces/edits the Python helper code. It never routes to paid closed APIs or credit systems — everything runs locally on the user's own GPU with open weights.

## Steps
1. **Clarify (if not already given)**: GPU VRAM (8/12/16/24 GB), T2V vs I2V priority, and whether they need audio in the video. If unknown, ask briefly before finalizing defaults.
2. **Produce the script** from `references/diffusers-helper.py` (the canonical single-file helper). Present the full code to the user in a code block, plus the install command and usage examples.
3. **Match defaults to the user's VRAM** when known:
   - 12 GB → default backend `wan` (1.3B) or `ltx`; suggest `--quantize fp8` for 14B/5B.
   - 16 GB → `wan` 14B or `cogvideox`; suggest `--quantize fp8` + `--offload cpu`.
   - 24 GB → `wan` 14B or `hunyuan` full precision.
4. **Explain the key pieces** (briefly, not a wall of text): the `BACKENDS` registry (backend → repo + min VRAM), `pick_device()` (CUDA/VRAM check), `load_pipeline()` (fp8 quantization + CPU offload + VAE slicing/tiling), `run_inference()` (T2V vs I2V), and `save_video()` (MP4 via imageio/ffmpeg).
5. **Honesty on model details**: repo ids and VRAM figures can drift. If the user asks for the very latest model versions, GGUF quant availability, or license status, use `search_web` to confirm before asserting. Never fabricate a repo name or download URL — point to the official Hugging Face org page.
6. **GGUF note**: GGUF quantization for these models is not directly supported in Diffusers — it needs the ComfyUI-GGUF node. The helper therefore uses fp8 (bitsandbytes) for Diffusers; if the user specifically wants GGUF, point them to ComfyUI instead.

## Notes
- The script's `--image` flag enables I2V (the image becomes the first frame); omitting it runs T2V. `--backend` accepts `wan` (default), `ltx`, `hunyuan`, `cogvideox`.
- VRAM management in the script: `enable_model_cpu_offload()` / `enable_sequential_cpu_offload()` (via `--offload cpu|sequential`) and `enable_vae_slicing()` + `enable_vae_tiling()` (always on) — the VAE is the usual OOM spot.
- Quantization: `--quantize fp8` uses bitsandbytes `BitsAndBytesConfig(load_in_8bit=True)`, cutting VRAM ~30–50% with minor quality loss.
- Install deps: `pip install torch torchvision diffusers transformers accelerate bitsandbytes imageio imageio-ffmpeg pillow`.
- If the user asks to bypass a paid platform's credit system, jailbreak, or access unauthorized content, decline and steer back to open-source local models. Video generation does not support NSFW/adult content on any channel.
- If the user wants to iterate (add a backend, change resolution/fps defaults, add audio), edit the reference file `references/diffusers-helper.py` and re-present the updated script.