Tag: performance
All the articles with the tag "performance".
Optimization of the Rust LLM engine
Published: at 10:00 AMrvllm generated 0.9 tokens per second from a 360M model. Profiling it showed almost all the time went to memcpy, not math.
All the articles with the tag "performance".
rvllm generated 0.9 tokens per second from a 360M model. Profiling it showed almost all the time went to memcpy, not math.