Tag: rust
All the articles with the tag "rust".
rvllm - A Small LLM Inference Engine in Rust
A basic local LLM inference engine written from scratch in Rust, running SmolLM2 on CPU with continuous batching and paged KV cache, and where CUDA and Metal support fit in next.
Building a KV Cache Block Scheduler in Rust
A from-scratch PagedAttention-style KV cache block manager in Rust - reference counting, prefix caching via radix trie, LRU eviction, and copy-on-write for beam search.
Elephant VM - stack-based VM written in Rust
An implementation of a simple stack-based VM written in Rust