Install ComfyUI: Difference between revisions

From A Technological Wiki
Jump to navigation Jump to search
VisualWikitext
Created page with "= Installation = == Installing on linux == Make sure you have '''Python 3.10 or higher''' and '''git''' installed: <syntaxhighlight lang="bash"> sudo apt install python3 python3-venv python3-pip git </syntaxhighlight> Clone the ComfyUI repository: <syntaxhighlight lang="bash"> git clone https://github.com/comfyanonymous/ComfyUI.git cd ComfyUI </syntaxhighlight> Create a virtual environment and install dependencies: <syntaxhighlight lang="bash"> python3 -m venv ven..."
 
 
(15 intermediate revisions by the same user not shown)
Line 1: Line 1:
= What is ComfyUI =
'''ComfyUI''' is a powerful and modular graphical interface for running and creating '''Stable Diffusion''' workflows for generating images and videos from text or images.
It allows you to build complex diffusion pipelines visually by connecting nodes, without needing to write code. It is designed for flexibility, performance, and transparency, letting you see exactly how each part of a [[Diffussion models|diffusion model]] (VAE, CLIP, sampler, etc.) is connected and processed.
= Installation =
= Installation =


Line 5: Line 10:
Make sure you have '''Python 3.10 or higher''' and '''git''' installed:
Make sure you have '''Python 3.10 or higher''' and '''git''' installed:


<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">sudo apt install python3 python3-venv python3-pip git</syntaxhighlight>
sudo apt install python3 python3-venv python3-pip git
</syntaxhighlight>


Clone the ComfyUI repository:
Clone the ComfyUI repository:
Line 24: Line 27:
</syntaxhighlight>
</syntaxhighlight>


Run ComfyUI:
'''Install NVIDIA dependencies'''<syntaxhighlight lang="bash">
pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu130
pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu130
 
</syntaxhighlight>Run ComfyUI:


<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
Line 31: Line 38:


The interface will open in your browser at
The interface will open in your browser at
'''http://127.0.0.1:8188'''
'''http://127.0.0.1:8188'''


== Installing on windows ==
== Installing and running on windows ==


Install '''Python 3.10+''' from https://www.python.org/downloads/
# [https://github.com/comfyanonymous/ComfyUI/releases/latest/download/ComfyUI_windows_portable_nvidia.7z Download comfyUI]
During installation, enable '''Add Python to PATH'''.
# Unzip the downloaded file
# Double click either run nvidia_gpu.bat or run cpu.bat depending on your computer’s configuration to launch ComfyUI.
# You can access to http://127.0.0.1:8188 when you see it in the opened window.


Install '''Git for Windows''' from https://git-scm.com/download/win
= Understanding workflows and nodes =
Open a terminal (PowerShell) and run:


<syntaxhighlight lang="bash">
== What workflows are ==
git clone https://github.com/comfyanonymous/ComfyUI.git
A '''workflow''' in ComfyUI is a graph that defines how data (text prompts, models, images, etc.) flows between nodes to generate an output media.
cd ComfyUI
pip install -r requirements.txt
python main.py
</syntaxhighlight>


After installation, ComfyUI will start locally and open in your web browser.
Example workflow of image to video:
[[File:ComfUI example i2v.png|none|thumb|484x484px]]This is the file with the previous workflow in json format: [[File:ComfyUI image to video example workflow.json|none|thumb]]


= Executing ComfyUI =
Each workflow is saved as a .json file that can be shared or loaded into ComfyUI. They are stored in your '''ComfyUI/user/default/workflows''' folder.


You can start ComfyUI in different ways:
== What nodes are ==
Nodes are the building blocks of a workflow. Each node performs a specific task, such as:


=== Standard mode ===
* loading a model
<syntaxhighlight lang="bash">
* encoding a text prompt
python main.py
* generating noise
</syntaxhighlight>
* decoding the latent image


=== With a specific model or workflow ===
Nodes are connected through inputs and outputs, creating a chain of operations that define the diffusion process.
<syntaxhighlight lang="bash">
python main.py --extra-model-paths=/path/to/models
python main.py --disable-auto-launch
</syntaxhighlight>


Once running, open your browser and go to:
Common node types include:
<syntaxhighlight lang="other">
http://localhost:8188
</syntaxhighlight>


= Understanding workflows and nodes =
'''Checkpoint Loader''' - loads the main diffusion model


== What are workflows ==
'''VAE Loader''' - loads a Variational Autoencoder
A '''workflow''' in ComfyUI is a graph that defines how data (text prompts, models, images, etc.) flows between nodes to generate an image.
Each workflow is saved as a '''.json''' file that can be shared or loaded into ComfyUI.


You can import a workflow by dragging the JSON file directly into the interface.
'''CLIP Text Encoder''' - processes text prompts


== What are nodes ==
'''Sampler''' - controls the diffusion steps
'''Nodes''' are the building blocks of a workflow. Each node performs a specific task, such as:


loading a model
'''Save Image''' - exports the final result


encoding a text prompt
You can double-click on the canvas to add new nodes or search for them by name.


generating noise
= Workflow templates and dependencies =
You can import a workflows by:


decoding the latent image
* dragging directly into the interface a JSON file or media (image or video) generated with comfyUI
* On the right sidebar, in the "Templates" menu, you can find many different workflows for all type of media.
When loading workflows, they might have nodes that are not installed yet on your system. To install them, click on "Manager" at the top right of the interface, then on "Install missing custom nodes".


Nodes are connected through inputs and outputs, creating a chain of operations that define the diffusion process.
= ComfyUI folder structure =


Common node types include:
ComfyUI organizes its files in:


'''Checkpoint Loader''' - loads the main diffusion model (e.g. SDXL)
'''/ComfyUI/models/unet''' - diffussion models


'''VAE Loader''' - loads a Variational Autoencoder
'''/ComfyUI/models/vae''' - Variational Autoencoders (VAEs)


'''CLIP Text Encoder''' - processes text prompts
'''/ComfyUI/models/clip''' - CLIP encoders for text and image embeddings


'''Sampler''' - controls the diffusion steps
'''/ComfyUI/models/checkpoints''' - checkpoints that contain with model, vae and clip loader.


'''Save Image''' - exports the final result
'''/ComfyUI/custom_nodes''' - external or user-made node scripts


You can right-click the canvas to add new nodes or search for them by name.
'''/ComfyUI/output''' - generated media


= ComfyUI folder structure =
'''ComfyUI/user/default/workflows''' - saved workflow files in JSON format


ComfyUI organizes its files in a clear directory structure:
= Loading assets on your workflow =
Depending on how much VRAM does your graphics card has and how much RAM you have, you will be able to use resources up to the total amount of memory you have.


'''/ComfyUI/models/checkpoints''' - main diffusion models (e.g. SDXL, SD1.5, SD2.1)
'''Example'''


'''/ComfyUI/models/vae''' - Variational Autoencoders (VAEs)
* Computer available memory 56GB, distributed in:
** graphics card VRAM: 24GB
** computer RAM: 32GB


'''/ComfyUI/models/clip''' - CLIP encoders for text and image embeddings
* Resources on nodes 38,5GB:
** Model wan2.2_i2v_high_noise_14B_Q8_0.gguf : 15GB
** Model wan2.2_i2v_low_noise_14B_Q8_0.gguf : 15GB
** LoRA wan2.2_i2v_A14b_high_noise_lora_rank64_lightx2v_4step_1022.safetensors: 0,6GB
** LoRA wan2.2_i2v_A14b_low_noise_lora_rank64_lightx2v_4step_1022.safetensors: 0,7GB
** VAE wan_2.1_vae.safetensors: 0,3GB
** Text Encoder umt5_xxl_fp8_e4m3fn_scaled.safetensors: 6,7GB
* ComfyUI processing and temporary data: 10-20% of assets


'''/ComfyUI/models/upscale''' - upscaling models
With this configuration, we would fit the requirements, as we have 56GB of memory and the process requires less than 50GB.


'''/ComfyUI/models/controlnet''' - ControlNet models for guiding generation
If comfyUI stops abruptly, it's probably due to an excessive usage of memory. You will have to reduce the size of the assets you are using.


'''/ComfyUI/custom_nodes''' - external or user-made node scripts
== Installing custom nodes ==
Users can create their very own custom nodes. To install them:


'''/ComfyUI/input''' - user input images and resources
# get the url of the repository, for example <nowiki>https://github.com/SeanScripts/ComfyUI-Unload-Model</nowiki>
# go to the custom_nodes folder, open a terminal and execute git clone <nowiki>https://github.com/SeanScripts/ComfyUI-Unload-Model.git</nowiki>
# restart comfyUI server
#


'''/ComfyUI/output''' - generated images
= Troubleshooting =


'''/ComfyUI/workflows''' - saved workflow files in JSON format
* Do not try to generate videos longer than 100 frames, models start hallucinating and generate random content.


'''/ComfyUI/web''' - interface and static assets
= Assets resources =


= Resources for models and components =
== Previous examples resources ==


You can expand ComfyUI by downloading compatible models and components from community repositories:
'''Wan GGUFs (Models). Destiny folder: models/unet'''


== Diffusion models ==
https://huggingface.co/bullerwins/Wan2.2-I2V-A14B-GGUF/tree/main


civitai.com - community hub for diffusion models (SD1.5, SDXL, LoRA, etc.)
'''Wan VAE. Destiny folder: models/vae'''


huggingface.co/models - official and community models
https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/tree/main/split_files/vae


modelscope.cn/models - AI models for various domains
'''Wan sampling LoRAs. Destiny folder: models/loras'''


== GGUF models ==
https://huggingface.co/lightx2v/Wan2.2-Distill-Loras/tree/main
GGUF format is used for quantized models optimized for inference.
You can find them at:


huggingface.co - TheBloke profile (repository of GGUF models)
'''Wan ext encoder. Destiny folder: models/text_encoders'''


github.com/ggerganov/llama.cpp - technical details on the GGUF format
https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/tree/main/split_files/text_encoders


== VAE (Variational Autoencoders) ==
== LoRAs resources ==
VAEs are used to improve image color, clarity, and detail.
You can use additional LoRAs in your workflow to get better results for specific prompts. Your can get them from:
Common sources: civitai.com and huggingface.co (search for "vae-fp16" or "vae-ft-mse").


== CLIP models ==
https://civitai.com/
CLIP encoders transform text or images into embeddings.
They are usually included with diffusion models, but standalone versions can be found on:
huggingface.co/openai/clip-vit-large-patch14


== LoRAs and ControlNets ==
'''IMPORTANT:''' LoRAs are trained using specific Models (Wan, Flux, SD, etc). Make sure that the LoRAs you download use your base model. Otherwise, change your base model to use LoRA's.
These extend model behavior for styles, poses, or compositions.
[[index.php?title=Category:AI: Generate media locally]]
Available on:
[[Category:AI: Generate media locally]]
https://civitai.com/models (LoRA and ControlNet section)
https://huggingface.co/models?q=controlnet

Latest revision as of 15:42, 3 November 2025

What is ComfyUI

ComfyUI is a powerful and modular graphical interface for running and creating Stable Diffusion workflows for generating images and videos from text or images.

It allows you to build complex diffusion pipelines visually by connecting nodes, without needing to write code. It is designed for flexibility, performance, and transparency, letting you see exactly how each part of a diffusion model (VAE, CLIP, sampler, etc.) is connected and processed.

Installation

Installing on linux

Make sure you have Python 3.10 or higher and git installed: 

sudo apt install python3 python3-venv python3-pip git

Clone the ComfyUI repository: 

git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI

Create a virtual environment and install dependencies: 

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Install NVIDIA dependencies

pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu130
pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu130

Run ComfyUI: 

python main.py

The interface will open in your browser at

http://127.0.0.1:8188

Installing and running on windows

  1. Download comfyUI
  2. Unzip the downloaded file
  3. Double click either run nvidia_gpu.bat or run cpu.bat depending on your computer’s configuration to launch ComfyUI.
  4. You can access to http://127.0.0.1:8188 when you see it in the opened window.

Understanding workflows and nodes

What workflows are

A workflow in ComfyUI is a graph that defines how data (text prompts, models, images, etc.) flows between nodes to generate an output media.

Example workflow of image to video:

This is the file with the previous workflow in json format: File:ComfyUI image to video example workflow.json

Each workflow is saved as a .json file that can be shared or loaded into ComfyUI. They are stored in your ComfyUI/user/default/workflows folder.

What nodes are

Nodes are the building blocks of a workflow. Each node performs a specific task, such as:

  • loading a model
  • encoding a text prompt
  • generating noise
  • decoding the latent image

Nodes are connected through inputs and outputs, creating a chain of operations that define the diffusion process.

Common node types include:

Checkpoint Loader - loads the main diffusion model

VAE Loader - loads a Variational Autoencoder

CLIP Text Encoder - processes text prompts

Sampler - controls the diffusion steps

Save Image - exports the final result

You can double-click on the canvas to add new nodes or search for them by name.

Workflow templates and dependencies

You can import a workflows by:

  • dragging directly into the interface a JSON file or media (image or video) generated with comfyUI
  • On the right sidebar, in the "Templates" menu, you can find many different workflows for all type of media.

When loading workflows, they might have nodes that are not installed yet on your system. To install them, click on "Manager" at the top right of the interface, then on "Install missing custom nodes".

ComfyUI folder structure

ComfyUI organizes its files in:

/ComfyUI/models/unet - diffussion models

/ComfyUI/models/vae - Variational Autoencoders (VAEs)

/ComfyUI/models/clip - CLIP encoders for text and image embeddings

/ComfyUI/models/checkpoints - checkpoints that contain with model, vae and clip loader.

/ComfyUI/custom_nodes - external or user-made node scripts

/ComfyUI/output - generated media

ComfyUI/user/default/workflows - saved workflow files in JSON format

Loading assets on your workflow

Depending on how much VRAM does your graphics card has and how much RAM you have, you will be able to use resources up to the total amount of memory you have.

Example

  • Computer available memory 56GB, distributed in:
    • graphics card VRAM: 24GB
    • computer RAM: 32GB
  • Resources on nodes 38,5GB:
    • Model wan2.2_i2v_high_noise_14B_Q8_0.gguf : 15GB
    • Model wan2.2_i2v_low_noise_14B_Q8_0.gguf : 15GB
    • LoRA wan2.2_i2v_A14b_high_noise_lora_rank64_lightx2v_4step_1022.safetensors: 0,6GB
    • LoRA wan2.2_i2v_A14b_low_noise_lora_rank64_lightx2v_4step_1022.safetensors: 0,7GB
    • VAE wan_2.1_vae.safetensors: 0,3GB
    • Text Encoder umt5_xxl_fp8_e4m3fn_scaled.safetensors: 6,7GB
  • ComfyUI processing and temporary data: 10-20% of assets

With this configuration, we would fit the requirements, as we have 56GB of memory and the process requires less than 50GB.

If comfyUI stops abruptly, it's probably due to an excessive usage of memory. You will have to reduce the size of the assets you are using.

Installing custom nodes

Users can create their very own custom nodes. To install them:

  1. get the url of the repository, for example https://github.com/SeanScripts/ComfyUI-Unload-Model
  2. go to the custom_nodes folder, open a terminal and execute git clone https://github.com/SeanScripts/ComfyUI-Unload-Model.git
  3. restart comfyUI server

Troubleshooting

  • Do not try to generate videos longer than 100 frames, models start hallucinating and generate random content.

Assets resources

Previous examples resources

Wan GGUFs (Models). Destiny folder: models/unet

https://huggingface.co/bullerwins/Wan2.2-I2V-A14B-GGUF/tree/main

Wan VAE. Destiny folder: models/vae

https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/tree/main/split_files/vae

Wan sampling LoRAs. Destiny folder: models/loras

https://huggingface.co/lightx2v/Wan2.2-Distill-Loras/tree/main

Wan ext encoder. Destiny folder: models/text_encoders

https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/tree/main/split_files/text_encoders

LoRAs resources

You can use additional LoRAs in your workflow to get better results for specific prompts. Your can get them from:

https://civitai.com/

IMPORTANT: LoRAs are trained using specific Models (Wan, Flux, SD, etc). Make sure that the LoRAs you download use your base model. Otherwise, change your base model to use LoRA's. index.php?title=Category:AI: Generate media locally

Retrieved from "http://wikitechia.top/index.php?title=Install_ComfyUI&oldid=1896"