Tag: vLLM

All the articles with the tag "vLLM".

• Jul 2026

  Two Boxes, One Model - Playing With Multi-Node vLLM

  A small learning experiment splitting an LLM across two GPU boxes with Ray and vLLM - tensor parallel vs pipeline parallel, and why plain ethernet ruins both.

• Sep 2025

  Memory Optimization Deep Dive Running 8B Models on a Single 4090 using vLLM

  An exploration of quantization techniques and memory optimization strategies for running Llama 8B models efficiently on consumer hardware using vLLM