Tag: candle
All the articles with the tag "candle".
rvllm - A Small LLM Inference Engine in Rust
Published: at 10:00 AMA basic local LLM inference engine written from scratch in Rust, running SmolLM2 on CPU with continuous batching and paged KV cache, and where CUDA and Metal support fit in next.