Posts
All the posts I've published.
Memory Optimization Deep Dive Running 8B Models on a Single 4090 using vLLM
Published: at 03:01 PMAn exploration of quantization techniques and memory optimization strategies for running Llama 8B models efficiently on consumer hardware using vLLM
Data capture for ML endpoints
Published: at 04:01 PMOne of the approaches of how how you can add data capture to your ML endpoints
Python asyncio internals
Published: at 12:11 PMA brief intro into asyncio internals in Python