AI: Text to image in local
In this case, where are going to use Flux (https://bfl.ai/)
Flux (styled often as FLUX or FLUX.1) is a family of text-to-image generation models developed by Black Forest Labs (BFL).
Key features & properties:
| Feature | Description |
|---|---|
| Architecture | A “rectified flow transformer” model, ~12B parameters, combining approaches from diffusion, transformer flow matching, etc. |
| Model Variants / Licencing | There are several variants: |
| – FLUX.1 [schnell] | The fast, distilled version. Open source, Apache-2.0 • high speed, fewer inference steps. |
| – FLUX.1 [dev] | More powerful, more resource-intensive, open weights under non-commercial licence. Better fidelity. |
| – FLUX.1 [pro] | Commercial / API access only. Higher quality, possibly features not in open models. |
| Additional modes | E.g. “Ultra”, “Raw” for very high resolution or photorealism. Also “Kontext” for editing / in-context image generation. |
| Usage | Can be used via APIs, via HuggingFace / other model hubs, or locally with GUIs / “WebUIs” (Stable Diffusion‐style UIs like Forge, ComfyUI), etc. |
So it’s kind of a next‐gen Stable Diffusion-like set of models, with some distilled/faster versions and some high-quality ones. Good prompt following, potentially strong image quality.
Use cases / What it is used for
- Generating images from text prompts (text→image).
- Image editing / in-context generation: e.g. “Kontext” mode which allows combining text + image input to guide edits.
- In creative workflows, concept art, renders, style transfer, etc.
- Can be used as a base for fine-tuning (LoRA, etc.), or with low-bit quantization etc. to run on lower-VRAM hardware.
How to install & run Flux locally (CLI + GUI)
Graphical User Interface
To do it, we need another software to manage the GUI, in this case we will use comfyUI. It works with a node-based workflow.
Installation
Follow the guide for your OS: https://comfyui-wiki.com/en/install/install-comfyui
# For Ubuntu: sudo apt install python3 python3-pip python3-venv python3 -m venv comfy-env source comfy-env/bin/activate pip install comfy-cli comfy --install-completion comfy install comfy --workspace=/path/to/your/workspace install #comfy --here install #GPU support for Nvidia pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124 #CPU support only pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
Download Flux.1 dev here https://huggingface.co/black-forest-labs/FLUX.1-dev/tree/main. You need to signup and download file into local folder ~/comfyUI/models/difussion_models/. This will be the base model to use.
Download VAE (used to learn and generate new data based on patterns and features from a given dataset) http://huggingface.co/stabilityai/sd-vae-ft-mse-original/blob/main/vae-ft-mse-840000-ema-pruned.safetensors and place it in models/vae
Download CLIP Loader https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/clip_l.safetensors and place it in models/text_encoders folder
Download T5 https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/t5xxl_fp16.safetensors and place it in models/text_encoders folder
If you want to use a custom Model (checkpoint) or LoRA, you can download them from civitai.com
Launch comfyUI
comfy launch
Load Template
In the left side, there are multiple you can use.
Configure workflow
- for adding new nodes, double click on an empty space