Tag Archives: CUDA

AI, GPU, Linux, LLMs, Tools

Installing ComfyUI with Python 3.12 on Debian 13 (Trixie) with CUDA

This guide provides instructions for installing and configuring ComfyUI on Debian 13 (Trixie) using Python 3.12. The process encompasses system preparation, Python version management, dependency installation, and configuration for optimal performance with NVIDIA GPU support.

The installation assumes that NVIDIA graphics hardware and CUDA are properly installed and configured on the system. For users who need guidance on setting up CUDA specifically for Debian 13 (Trixie), a related tutorial is available at: Building Llama.cpp with CUDA on Debian 13 (Trixie).

Prerequisites and System Preparation

Before initiating the ComfyUI installation process, it is crucial to ensure that the system has all necessary dependencies installed. This foundational step involves updating the package repository and installing development tools and libraries required for building and running the ComfyUI application effectively.

The initial system preparation begins with updating the package list to access the latest available packages:

sudo apt update

Following this update, a comprehensive set of build tools and libraries must be installed. These dependencies are fundamental for compiling software, managing Python environments, and supporting graphical operations that ComfyUI requires:

sudo apt install -y build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev git gcc bc

In addition to the core development dependencies, several system-level packages are essential for proper functionality. These include utilities for managing Python virtual environments, graphics libraries for rendering, and core system libraries:

sudo apt install wget git python3 python3-venv libgl1 libglib2.0-0

These packages establish the necessary foundation for Python version management, Git operations, and graphical interface support that ComfyUI requires for optimal performance.

Installing Python 3.12 Using pyenv

ComfyUI requires Python 3.12 for full compatibility with its latest features and performance optimizations. Since Debian 13 (Trixie) may not include this specific Python version in its default repositories, we utilize pyenv to manage the installation and execution of the required Python environment.

The installation process begins with downloading and executing the official pyenv installation script from the pyenv repository:

curl https://pyenv.run | bash

This command fetches and executes the installation script, setting up the pyenv environment in the user’s home directory. Following the installation, proper shell configuration is essential to initialize pyenv correctly for each terminal session.

The configuration involves appending specific environment variable exports to the .bashrc file. These settings ensure that pyenv is properly initialized and that the appropriate Python version paths are included in the system’s PATH:

echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc
echo '[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(pyenv init -)"' >> ~/.bashrc
source ~/.bashrc

With the environment properly configured, the specific Python version can be installed using pyenv. The command below installs Python 3.12.12, which is compatible with ComfyUI requirements:

pyenv install 3.12.12

Creating and Configuring the ComfyUI Environment

After establishing the Python environment, the next step involves creating a dedicated directory for ComfyUI and setting up the project structure. This organization ensures proper isolation of dependencies and facilitates easy management of the installation.

The creation of the ComfyUI directory and navigation into it follows these commands:

mkdir ComfyUI
cd ComfyUI

To ensure that the correct Python version is used for this specific project, set the local Python version to 3.12.12 using pyenv:

pyenv local 3.12.12

This command creates a .python-version file in the current directory, which pyenv will automatically use when entering this directory in future sessions.

With the environment properly configured, the next step involves installing the ComfyUI command-line interface tool. This utility simplifies the installation and management of ComfyUI components:

pip install comfy-cli

Following the installation of the CLI tool, it is recommended to install shell completion support for enhanced usability:

comfy --install-completion

The final step in the initial setup process involves installing all necessary ComfyUI dependencies and components:

comfy install

This command downloads and configures all required packages and models, which may take considerable time depending on network speed and system resources.

Configuring CUDA Support

For users with NVIDIA graphics hardware, configuring CUDA support is essential for optimal performance. The installation process checks for the presence of CUDA by verifying the nvcc compiler version.

To determine if CUDA is properly installed, execute the following command:

nvcc --version

If CUDA is correctly installed, the output will display information similar to:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Mar_28_02:18:24_PDT_2024
Cuda compilation tools, release 12.4, V12.4.131
Build cuda_12.4.r12.4/compiler.34097967_0

If CUDA is detected, install the appropriate PyTorch version with CUDA support using the following command:

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

The cu124 suffix corresponds to the CUDA compilation tools release 12.4, as shown in the example output. This ensures that PyTorch is compiled with support for the installed CUDA version, enabling GPU acceleration for ComfyUI operations.

Launching ComfyUI

With all dependencies properly installed and configured, ComfyUI can be launched using the command-line interface. The basic launch command starts the application with default settings:

comfy launch

For users who require remote access to the ComfyUI interface, the application can be configured to listen on all network interfaces and specific ports. This configuration enables access from other machines on the network:

comfy launch -- --listen 0.0.0.0 --port 8080

This command configures ComfyUI to accept connections from any IP address (0.0.0.0) on port 8080, making it accessible across the network while maintaining security through proper firewall configuration.

It is important to always ensure that you are working within the ComfyUI directory before launching the application. This practice guarantees that the correct Python version and dependencies are used, preventing potential conflicts or errors during execution.

With these steps completed, ComfyUI is successfully installed and configured to run with Python 3.12 on Debian 13 (Trixie). The system is now ready for use with NVIDIA graphics hardware and CUDA support, providing users with a powerful and flexible interface for creating complex image generation workflows.

AI, GPU, Linux, LLMs, Tools

Installing Stable Diffusion WebUI with Python 3.10 on Debian 13 (Trixie)

This guide provides detailed instructions for installing and configuring the Stable Diffusion WebUI on Debian 13 (Trixie), utilizing Python 3.10. The process involves several key steps including system preparation, Python version management, repository cloning, and configuration adjustments to enable network accessibility.

The installation assumes that NVIDIA graphics hardware and CUDA are already properly installed and configured on the system. For users who need guidance on setting up CUDA specifically for Debian 13 (Trixie), a related tutorial is available at: Building Llama.cpp with CUDA on Debian 13 (Trixie).

Prerequisites and System Preparation

Before beginning the installation process, it is essential to ensure that the system has all necessary dependencies installed. This includes development tools and libraries required for building and running the Stable Diffusion WebUI application.

The first step involves updating the package list to ensure access to the latest available packages. This is followed by installing a comprehensive set of build tools and libraries that are fundamental for compiling software and managing Python environments:

sudo apt update
sudo apt install -y build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev git gcc bc

In addition to the core development dependencies, several system-level packages are required for proper functionality. These include utilities for managing Python virtual environments, graphics libraries for rendering, and core system libraries:

sudo apt install wget git python3 python3-venv libgl1 libglib2.0-0

These packages provide the foundation necessary for Python version management, Git operations, and graphical interface support that the Stable Diffusion WebUI requires.

Installing Python 3.10 Using pyenv

The Stable Diffusion WebUI specifically requires Python 3.10, which may not be available in the default repositories for Debian 13 (Trixie). To address this requirement, we utilize pyenv, a powerful tool for managing multiple Python versions on a single system.

The installation of pyenv begins with downloading and executing the official installation script from the pyenv repository:

curl https://pyenv.run | bash

This command fetches the installation script and executes it, setting up the pyenv environment in the user’s home directory. Following the installation, it is necessary to configure the shell environment to properly initialize pyenv each time a new terminal session is started.

The configuration involves appending specific environment variable exports to the .bashrc file. These settings ensure that pyenv is correctly initialized and that the appropriate Python version paths are included in the system’s PATH:

echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc
echo '[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(pyenv init -)"' >> ~/.bashrc
source ~/.bashrc

Once the environment variables are properly configured, the specific Python version can be installed using pyenv. The command below installs Python 3.10.6, which is compatible with the Stable Diffusion WebUI requirements:

pyenv install 3.10.6

Cloning and Configuring the Stable Diffusion WebUI Repository

With the Python environment properly established, the next step involves obtaining the source code for the Stable Diffusion WebUI. This is accomplished by cloning the official repository from GitHub, which contains all necessary files and dependencies for running the web interface.

The cloning process retrieves the complete repository including all branches, commit history, and configuration files:

git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui

After successfully cloning the repository, navigate into the newly created directory. This is where all subsequent configuration and setup operations will take place:

cd stable-diffusion-webui

To ensure that the correct Python version is used for this specific project, set the local Python version to 3.10.6 using pyenv:

pyenv local 3.10.6

This command creates a .python-version file in the current directory, which pyenv will automatically use when entering this directory in future sessions.

Launching the WebUI Application

With all prerequisites met and the environment properly configured, the final step involves starting the Stable Diffusion WebUI application. This is accomplished by executing the webui.sh script, which handles the initialization process including dependency installation and server startup:

webui.sh

The execution of this script may take some time as it downloads required model files and dependencies, initializes the Python environment, and prepares the web server for operation. Users should allow sufficient time for this process to complete fully.

Configuring Network Accessibility

By default, the Stable Diffusion WebUI is configured to only accept connections from the local machine. For users who wish to access the interface from other devices on the network, a configuration change is necessary.

The configuration file webui-user.sh contains various settings that can be adjusted to modify the behavior of the web application. To enable network accessibility, this file must be edited:

nano webui-user.sh

Within this file, locate the line that begins with #export COMMANDLINE_ARGS="". This line is commented out by default and serves as a placeholder for additional command-line arguments. To modify the behavior to accept external connections, change this line to:

export COMMANDLINE_ARGS="--listen"

This configuration change instructs the web application to listen on all available network interfaces rather than restricting access to localhost only. This modification enables remote access to the Stable Diffusion WebUI from other machines within the same network. The interface can be reached at http://<server-ip>:7860

With these comprehensive steps completed, the Stable Diffusion WebUI is successfully installed and configured to run with Python 3.10 on Debian 13 (Trixie). The system is now ready for use with NVIDIA graphics hardware and CUDA support, providing users with a fully functional interface for generating images using stable diffusion models.

AI, GPU, How-tos, Linux, LLMs, Tools

Building llama.cpp with CUDA on Debian 13 “Trixie”

2 Comments

If you’ve recently upgraded to Debian 13 or are fresh on a Trixie system, you may be eager to tap the power of your NVIDIA GPU for machine‑learning workloads. This post walks you through every step required to set up the necessary drivers, libraries, and build environment.

Why Enable CUDA in llama.cpp?

The original binaries of llama.cpp run on the CPU, which is perfectly fine for small models but can become a bottleneck with larger weights. By enabling the -DGGML_CUDA=ON flag, the project compiles the CUDA kernels that allow your NVIDIA GPU to perform inference. The result is a dramatic reduction in latency and a higher throughput for text generation tasks.

Prerequisites

  • A Debian 13 machine with an NVIDIA GPU that supports CUDA 11 or later.
  • Sudo access (or root) to install packages and modify system configuration.
  • An active internet connection so the package manager can fetch the necessary files.

Step 1 – Update Kernel Headers

Your system needs the headers that match the running kernel so that the NVIDIA driver can compile its kernel modules.

apt install linux-headers-$(uname -r)

This command pulls the headers for the current kernel release and installs them into the standard package locations.

Step 2 – Add Non‑Free Firmware Repositories

The Debian base repositories do not expose the proprietary firmware and driver packages needed for NVIDIA GPUs. By creating an additional source list file, we allow apt to pull the required non‑free components.

Create the file /etc/apt/sources.list.d/non‑free.sources and paste the following content:

Types: deb deb-src
URIs: http://deb.debian.org/debian/
Suites: trixie
Components: non-free-firmware contrib
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg

Types: deb deb-src
URIs: http://security.debian.org/debian-security/
Suites: trixie-security
Components: non-free-firmware contrib
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg

Types: deb deb-src
URIs: http://deb.debian.org/debian/
Suites: trixie-updates
Components: non-free-firmware contrib
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg

After saving the file, refresh the package lists so the new entries become available:

apt update

Step 3 – Install the NVIDIA Driver and CUDA Toolkit

3.1 Bring in the NVIDIA Keyring

The NVIDIA distribution for Debian ships a keyring package that allows your system to verify the authenticity of the driver packages.

wget https://developer.download.nvidia.com/compute/cuda/repos/debian12/x86_64/cuda-keyring_1.1-1_all.deb
dpkg -i cuda-keyring_1.1-1_all.deb

3.2 Install Driver Packages

apt -V install nvidia-driver-cuda nvidia-kernel-dkms

The meta‑package nvidia-driver-cuda pulls the latest driver binaries and the CUDA toolkit for the current kernel. It also installs nvidia-kernel-dkms, which provides a Dynamic Kernel Module Support interface so the driver can be built against any future kernel version.

3.3 Regenerate Initramfs and Update GRUB

After installing the driver modules, you must ensure that the initramfs contains the new driver and that GRUB will boot into the updated kernel configuration.

update-initramfs -u -k all
update-grub

Reboot the machine to let the new driver take effect.

3.4 Install the CUDA Toolkit

With the driver in place, install the toolkit components that provide nvcc, libraries, and headers used by llama.cpp.

apt install nvidia-cuda-toolkit

Step 4 – Install Build Dependencies

The build process for llama.cpp requires several libraries and developer tools. Installing them up front keeps the compile step straightforward.

apt install libtcmalloc-minimal4 libcurl4-openssl-dev glslc cmake make git pkg-config

These packages provide memory allocation utilities, SSL support, the GLSL compiler, CMake, Make, Git, and generic build configuration tools.

Step 5 – Clone and Compile llama.cpp

With the environment prepared, fetch the source code and build it.

cd ~
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
mkdir build
cd build
cmake .. \
  -DGGML_AVX=ON \
  -DGGML_AVX_VNNI=ON \
  -DGGML_AVX2=ON \
  -DGGML_CUDA=ON \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLAMA_CURL=ON
make -j8
echo 'export PATH=$PATH:'$(realpath bin) >> ~/.bashrc

After the build finishes, log out and back in again so the newly added binaries become visible in your shell path.

Step 6 – Keep the Driver in Sync with Kernel Updates

Kernel upgrades are common, and the driver must be rebuilt against each new kernel. The following routine ensures the driver modules stay current.

apt install linux-headers-$(uname -r)
apt install --reinstall nvidia-driver-cuda nvidia-kernel-dkms
apt install nvidia-cuda-toolkit
update-initramfs -u -k all
update-grub

Running this sequence after any kernel upgrade guarantees that the driver continues to load correctly.

Step 7 – Updating the Source Tree

When the upstream llama.cpp project publishes a new release or a bug fix, refresh your local copy and rebuild:

cd ~
cd llama.cpp/

# Clean the working directory
git clean -xdf
mkdir build

# Pull the latest changes and submodules
git pull
git submodule update --recursive

# Rebuild
cd build/
cmake .. \
  -DGGML_AVX=ON \
  -DGGML_AVX_VNNI=ON \
  -DGGML_AVX2=ON \
  -DGGML_CUDA=ON \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLAMA_CURL=ON
make -j8
How-tos, Linux

Setting up a mining system with xmr-stak built from source and Ubuntu 16.04

If using an Nvidia GPU, install the Nvidia CUDA toolkit:

Download installer type “deb(network)” from:

https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604

To install issue the following commands:

$ sudo dpkg -i cuda-repo-ubuntu1604_9.1.85-1_amd64.deb
$ sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub
$ sudo apt-get update
$ sudo apt-get install cuda

Editing the enviroment to include the CUDA path:

$ sudo nano /etc/environment

Find the PATH variable and include the /usr/local/cuda-9.1/bin folder at the end of the string.

Save the file and reboot.

If using and AMG GPU, install AMD APP SDK 3.0:

Download and install the latest version from:

http://developer.amd.com/amd-accelerated-parallel-processing-app-sdk/

Untar the SDK to a location of your choice.

Decompress the file wit the following command:

$ tar -xvjf AMD-APP-SDKInstaller-v<3.0.x.y>-GA-linux64.tar.bz2

Run the installer:

$ sudo ./AMD-APP-SDKInstaller-v<3.0.x.y>-GA-linux64.sh

To fix libOpenCL issue:

$ cd $AMDAPPSDKROOT/lib/x86_64
$ sudo ln -sf sdk/libOpenCL.so.1 libOpenCL.so

then logout and login again.

Installing amdgpu-pro

Download the latest package from:
https://support.amd.com/en-us/kb-articles/Pages/AMDGPU-PRO-Driver-for-Linux-Release-Notes.aspx

Decompress the file wit the following command:

$ tar -xJvf amdgpu-pro-17.40-492261.tar.xz
$ cd amdgpu-pro-17.40-492261
$ sudo ./amd-pro-install -y

Reference:
https://linuxconfig.org/install-amdgpu-pro-16-50-on-ubuntu-16-04-xenial-xerus-linux

Building xmr-stak from source

Install all dependencies:

$ sudo apt install git libmicrohttpd-dev libssl-dev cmake build-essential libhwloc-dev

Create a directory for the source files and clone the source:

$ mkdir GIT-sources
$ cd GIT-sources
$ git clone https://github.com/fireice-uk/xmr-stak.git

Create a build directory:

$ mkdir xmr-stak/build
$ cd xmr-stak/build

Configuring and building xmr-stak

If building xmr-stak for CPU only mining and without http server support, use the following cmake flags:

$ cmake .. -DCUDA_ENABLE=FALSE -DOpenCL_ENABLE=FALSE -DMICROHTTPD_ENABLE=FALSE

If building xmr-stak for AMD GPU mining and CPU mining, use the following cmake flags:

$ cmake .. -DCUDA_ENABLE=FALSE

If building xmr-stak for Nvidia GPU and CPU mining, use the following cmake flags:

$ cmake .. -DOpenCL_ENABLE=FALSE

If building for all (AMD GPU, Nvidia GPU and CPU mining)

$ cmake ..

After cmake finishes, execute the following to build:

$ make -j4 install

Final system configurations

If using GPU mining (Nvidia GPU or AMD GPU), ensure the user you will use to mine is part of the video group in /etc/group

$ sudo usermod -a -G video $LOGNAME

Enabling Large Page Support for AMDGPU-PRO

Edit /etc/default/grub and add GRUB_CMDLINE_LINUX=”amdgpu.vm_fragment_size=9″

After editing ggrub, do:
$ sudo update-grub
$ sudo reboot

Configuring Large Page Support for Operating system (applies to all GPUs and CPUs)

Create a file named 98-HugePages-miner.conf in /etc/sysctl.d with the following content:

############################
vm.nr_hugepages=128
############################

Add the following lines to /etc/security/limits.conf (where “miner” is the name of your mining account):

############################
miner soft memlock 262144
miner hard memlock 262144
############################

Then reboot with:

$ reboot

You will find the xmr-stak binary in ~/GIT-sources/xmr-stak/build/bin. Run xmr-stak and follow the prompts to begin mining.