|
|
| Line 1: |
Line 1: |
| = Installing ComfyUI = | | = Basic concepts = |
|
| |
|
| = Understanding model compression and GPU VRAM = | | == GGUF == |
| | It is a compressed version of the model, with less parameters and quality loss. |
|
| |
|
| == What is “compression” in the context of AI models? == | | == checkpoint == |
| When we talk about compression for AI models (especially large video / image / generative models) we mean techniques that reduce the memory / compute cost required to run them, so they can fit on GPUs with less VRAM (video RAM) or run faster.
| | A checkpoint is the file of the model, it includes both the high and the low models. |
|
| |
|
| Examples of compression techniques include:
| | == safetensor == |
| * Quantisation (e.g., using 8-bit or 4-bit instead of 16- or 32-bit precision)
| | It is an extension where checkpoints work without compression. |
| * Weight-sharing or pruning (reducing redundant parameters)
| |
| * Architectural tricks (e.g., a “Mixture of Experts” design)
| |
| * Off-loading parts of the model to CPU or system RAM
| |
| * Lower resolution or fewer frames (for video)
| |
| These allow the model to “compress” memory usage or compute cost, making it feasible on smaller GPUs.
| |
|
| |
|
| == Why is VRAM important and how does it relate to model choice? == | | == Quantification (Q) == |
| Your GPU’s VRAM (for example the VRAM on an NVIDIA RTX card) determines how large a model you can load and how much data (images, video frames) you can process at once.
| | If a model is quantified it is a '''compressed''' version of the original model where the numbers are stored with fewer bits (less precision). It saves memory and runs faster, at the cost of some quality. |
| If the model + working data exceed the VRAM, you’ll either hit “out of memory” errors, or it will spill to system RAM (much slower) or simply run extremely slowly.
| |
|
| |
|
| So when you pick a model, you must match:
| | The '''higher''' '''number''' of the quantification is, '''lower quality''' the result will have and '''lower memory''' requirements will have. |
| * Model size (parameters, complexity)
| |
| * Compression / quantisation available
| |
| * Resolution and number of frames (for video)
| |
| * The VRAM you have (e.g., an RTX with **12 GB** or **24 GB** VRAM)
| |
|
| |
|
| If you have less VRAM, you either need a smaller model (or more compressed version) or you have to reduce resolution/frame-count.
| | == Floating point (fp) == |
| | It is a way of storing real numbers (like 3.14159) with '''high precision'''. Used for model weights and calculations. |
|
| |
|
| == Introducing the WAN 2.x models (text-to-video / image-to-video) ==
| | The '''higher number''' of fp is, '''higher quality''' will have the result and '''higher memory''' requirements will have. |
| The “WAN” (often seen as “WAN 2.1”, “WAN 2.2”, etc) family are open-source video generation models (e.g., from Alibaba Tongyi Lab) that support tasks like Text-to-Video (T2V), Image-to-Video (I2V) and more. | |
| | |
| Key points:
| |
| * WAN 2.1: Earlier version; requires moderate to high VRAM for the largest variants. :contentReference[oaicite:4]{index=4}
| |
| * WAN 2.2: Newer version, uses Mixture-of-Experts (MoE), improved compression and efficiency. :contentReference[oaicite:5]{index=5}
| |
| * For example, the WAN 2.2 docs say some model variants still assume very high VRAM (80 GB+) for full usage. :contentReference[oaicite:6]{index=6}
| |
| | |
| == How to choose the right model variant for your GPU (12 GB vs 24 GB VRAM) ==
| |
| Here are some guidelines:
| |
| | |
| * If you have*12 GB VRAM:
| |
| - Choose smaller models / variants (fewer parameters) or strongly compressed checkpoints (quantised, off-loaded).
| |
| - Run at lower resolution (e.g., 480p) and fewer frames for video.
| |
| - Use flags or options that off-load parts of model to CPU or system RAM if supported.
| |
| | |
| * If you have 24 GB VRAM:
| |
| - You have more headroom: you can consider larger model variants, higher resolution (e.g., 720p) or more frames.
| |
| - But still: check the model’s memory requirements — many “14B” parameter variants assume more than 24 GB unless compressed.
| |
| | |
| Also: As a general strategy, even if you have high VRAM, start with a lower resolution / simpler variant to check stability, then scale up.
| |
| | |
| == Example table: WAN model variants and recommended use for 12 GB vs 24 GB GPUs ==
| |
| The following table gives example model-variants of WAN, compression/precision assumptions, and recommended setups for two GPU VRAM categories (12 GB, 24 GB). Note: these are approximate guidance — actual memory use will depend on resolution, frame count, steps, batch size, and off‐loading.
| |
| | |
| {| class="wikitable"
| |
| ! Model variant !! Approx model size / parameters !! Compression / precision hints !! Recommended for ≈12 GB VRAM !! Recommended for ≈24 GB VRAM
| |
| |-
| |
| | WAN 2.1 – T2V/1.3B || ~1.3 billion parameters || FP16 or maybe 8-bit quantised || ✔ Yes — good fit at lower resolution (e.g., 480p) || ✔ Very good fit: can run slightly higher resolution or more frames
| |
| |-
| |
| | WAN 2.1 – T2V/14B || ~14 billion parameters || Requires large VRAM unless compressed/off-loaded || ✘ Not recommended on only 12 GB (likely OOM) || ✔ Possibly usable on 24 GB if strongly compressed / reduced resolution / off-loaded
| |
| |-
| |
| | WAN 2.2 – TI2V-5B || ~5 billion parameters (efficiency-optimized) || Newer compression, MoE architecture || ✔ Good candidate for 12 GB if you keep resolution moderate (e.g., 480p) || ✔ Even better on 24 GB; you may push resolution up to 720p or more
| |
| |-
| |
| | WAN 2.2 – A14B (T2V/I2V) || ~14 billion parameters with MoE (≈27 billion total but active ≈14B) || Requires high VRAM; docs state “80GB+ VRAM” single GPU baseline. :contentReference[oaicite:7]{index=7} || ✘ Not feasible on 12 GB in normal form || ✘ Even on 24 GB likely very constrained – only use in highly compressed/off-loaded scenario
| |
| |}
| |
| | |
| == Practical tips & considerations ==
| |
| * Always monitor your GPU memory usage (tools like nvidia-smi or system monitor) when testing a model.
| |
| * Use “offload to CPU/system RAM” features if available in the model or inference code — this helps when VRAM is insufficient.
| |
| * Reduce resolution (width × height), reduce number of frames or batch size to fit within VRAM.
| |
| * Use quantised / compressed model versions (e.g., 8-bit or lower precision) if available.
| |
| * If you hit “out of memory” (OOM) errors, either switch to smaller model variant, reduce resolution, off-load more, or increase VRAM hardware.
| |
| * Even if hardware allows, consider whether you really need the highest resolution — generating at 480p then upscaling may be more efficient. (Some community guides note that for WAN 2.1 generating 480p is _much_ faster. :contentReference[oaicite:8]{index=8} )
| |
| :: | | :: |
|
| |
|
| [[Category:IT Resources]] | | [[Category:IT Resources]] |
Basic concepts
GGUF
It is a compressed version of the model, with less parameters and quality loss.
checkpoint
A checkpoint is the file of the model, it includes both the high and the low models.
safetensor
It is an extension where checkpoints work without compression.
Quantification (Q)
If a model is quantified it is a compressed version of the original model where the numbers are stored with fewer bits (less precision). It saves memory and runs faster, at the cost of some quality.
The higher number of the quantification is, lower quality the result will have and lower memory requirements will have.
Floating point (fp)
It is a way of storing real numbers (like 3.14159) with high precision. Used for model weights and calculations.
The higher number of fp is, higher quality will have the result and higher memory requirements will have.