This is an automated archive made by the Lemmit Bot.

The original was posted on /r/homelab by /u/NovaAIHub on 2026-03-28 14:37:50+00:00.

After destroying two Proxmox installs trying to get GPU passthrough working, I figured I’d share what actually worked.

The mistakes I made:

1. Installing the full CUDA toolkit on the host (wrong ▒_@T you only need the driver)
2. Using Docker instead of LXC (Docker adds complexity for GPU workloads)
3. Not enabling persistence mode (GPU reinitializes on every container start)
4. Mixing host/container CUDA versions (they must match)

What actually works:

# Host: Install driver only, no toolkit
apt install -y pve-headers-$(uname -r)
apt install -y nvidia-driver-550

# Enable persistence mode (survives reboots)
nvidia-persistenced --persistence-mode

# LXC container config (Proxmox)
lxc.cgroup2.devices.allow: c 195:* rwm
lxc.cgroup2.devices.allow: c 509:* rwm
lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file

Inside the container:

# Container: Match host driver version, no toolkit needed
apt install -y nvidia-driver-550
nvidia-smi  # Should show your GPU

Container vs Docker for AI:

LXC is simpler because:

• No nested virtualization
• Direct device access
• Same isolation benefits
• Easier debugging

Hardware I’m using:

• RTX 3090 24GB (used from eBay, ~$700)
• 128GB RAM
• Ryzen 7900X

Runs Llama 3.2 11B at ~45 tokens/sec, Qwen 2.5 14B at ~30 tokens/sec.

Happy to answer questions in comments. I documented everything in more detail if anyone wants the full setup - see link in bio.