|
|
| (6 intermediate revisions by the same user not shown) |
| Line 1: |
Line 1: |
| = Installing ComfyUI =
| | TODO: create from text to video |
| | |
| = Understanding model compression and GPU VRAM =
| |
| | |
| == What is “compression” in the context of AI models? ==
| |
| When we talk about compression for AI models (especially large video / image / generative models) we mean techniques that reduce the memory / compute cost required to run them, so they can fit on GPUs with less VRAM (video RAM) or run faster.
| |
| | |
| Examples of compression techniques include:
| |
| * Quantisation (e.g., using 8-bit or 4-bit instead of 16- or 32-bit precision)
| |
| * Weight-sharing or pruning (reducing redundant parameters)
| |
| * Architectural tricks (e.g., a “Mixture of Experts” design)
| |
| * Off-loading parts of the model to CPU or system RAM
| |
| * Lower resolution or fewer frames (for video)
| |
| These allow the model to “compress” memory usage or compute cost, making it feasible on smaller GPUs.
| |
| | |
| == Why is VRAM important and how does it relate to model choice? ==
| |
| Your GPU’s VRAM (for example the VRAM on an NVIDIA RTX card) determines how large a model you can load and how much data (images, video frames) you can process at once.
| |
| If the model + working data exceed the VRAM, you’ll either hit “out of memory” errors, or it will spill to system RAM (much slower) or simply run extremely slowly.
| |
| | |
| So when you pick a model, you must match:
| |
| * Model size (parameters, complexity)
| |
| * Compression / quantisation available
| |
| * Resolution and number of frames (for video)
| |
| * The VRAM you have (e.g., an RTX with **12 GB** or **24 GB** VRAM)
| |
| | |
| If you have less VRAM, you either need a smaller model (or more compressed version) or you have to reduce resolution/frame-count.
| |
| | |
| == Introducing the WAN 2.x models (text-to-video / image-to-video) ==
| |
| The “WAN” (often seen as “WAN 2.1”, “WAN 2.2”, etc) family are open-source video generation models (e.g., from Alibaba Tongyi Lab) that support tasks like Text-to-Video (T2V), Image-to-Video (I2V) and more.
| |
| | |
| Key points:
| |
| * WAN 2.1: Earlier version; requires moderate to high VRAM for the largest variants. :contentReference[oaicite:4]{index=4}
| |
| * WAN 2.2: Newer version, uses Mixture-of-Experts (MoE), improved compression and efficiency. :contentReference[oaicite:5]{index=5}
| |
| * For example, the WAN 2.2 docs say some model variants still assume very high VRAM (80 GB+) for full usage. :contentReference[oaicite:6]{index=6}
| |
| | |
| == How to choose the right model variant for your GPU (12 GB vs 24 GB VRAM) ==
| |
| Here are some guidelines:
| |
| | |
| * If you have*12 GB VRAM:
| |
| - Choose smaller models / variants (fewer parameters) or strongly compressed checkpoints (quantised, off-loaded).
| |
| - Run at lower resolution (e.g., 480p) and fewer frames for video.
| |
| - Use flags or options that off-load parts of model to CPU or system RAM if supported.
| |
| | |
| * If you have 24 GB VRAM:
| |
| - You have more headroom: you can consider larger model variants, higher resolution (e.g., 720p) or more frames.
| |
| - But still: check the model’s memory requirements — many “14B” parameter variants assume more than 24 GB unless compressed.
| |
| | |
| Also: As a general strategy, even if you have high VRAM, start with a lower resolution / simpler variant to check stability, then scale up.
| |
| | |
| == Example table: WAN model variants and recommended use for 12 GB vs 24 GB GPUs ==
| |
| The following table gives example model-variants of WAN, compression/precision assumptions, and recommended setups for two GPU VRAM categories (12 GB, 24 GB). Note: these are approximate guidance — actual memory use will depend on resolution, frame count, steps, batch size, and off‐loading.
| |
| | |
| {| class="wikitable"
| |
| ! Model variant !! Approx model size / parameters !! Compression / precision hints !! Recommended for ≈12 GB VRAM !! Recommended for ≈24 GB VRAM
| |
| |-
| |
| | WAN 2.1 – T2V/1.3B || ~1.3 billion parameters || FP16 or maybe 8-bit quantised || ✔ Yes — good fit at lower resolution (e.g., 480p) || ✔ Very good fit: can run slightly higher resolution or more frames
| |
| |-
| |
| | WAN 2.1 – T2V/14B || ~14 billion parameters || Requires large VRAM unless compressed/off-loaded || ✘ Not recommended on only 12 GB (likely OOM) || ✔ Possibly usable on 24 GB if strongly compressed / reduced resolution / off-loaded
| |
| |-
| |
| | WAN 2.2 – TI2V-5B || ~5 billion parameters (efficiency-optimized) || Newer compression, MoE architecture || ✔ Good candidate for 12 GB if you keep resolution moderate (e.g., 480p) || ✔ Even better on 24 GB; you may push resolution up to 720p or more
| |
| |-
| |
| | WAN 2.2 – A14B (T2V/I2V) || ~14 billion parameters with MoE (≈27 billion total but active ≈14B) || Requires high VRAM; docs state “80GB+ VRAM” single GPU baseline. :contentReference[oaicite:7]{index=7} || ✘ Not feasible on 12 GB in normal form || ✘ Even on 24 GB likely very constrained – only use in highly compressed/off-loaded scenario
| |
| |}
| |
| | |
| == Practical tips & considerations ==
| |
| * Always monitor your GPU memory usage (tools like nvidia-smi or system monitor) when testing a model.
| |
| * Use “offload to CPU/system RAM” features if available in the model or inference code — this helps when VRAM is insufficient.
| |
| * Reduce resolution (width × height), reduce number of frames or batch size to fit within VRAM.
| |
| * Use quantised / compressed model versions (e.g., 8-bit or lower precision) if available.
| |
| * If you hit “out of memory” (OOM) errors, either switch to smaller model variant, reduce resolution, off-load more, or increase VRAM hardware.
| |
| * Even if hardware allows, consider whether you really need the highest resolution — generating at 480p then upscaling may be more efficient. (Some community guides note that for WAN 2.1 generating 480p is _much_ faster. :contentReference[oaicite:8]{index=8} )
| |
| :: | | :: |
| | | [[Category:AI: Generate media locally]] |
| [[Category:IT Resources]] | |
TODO: create from text to video