Resources for Local AI

A running index of links I keep coming back to for running, serving, and tuning models on my own hardware. Sections get added as I find things worth keeping.

Table of contents

• Inference / Serving
  • vLLM
    • Inside vLLM: Anatomy of a High-Throughput LLM Inference System
• Inference engineering
  • inference engineering by philip kiely