#kv-cache

dgx-spark-inference - Keeping good habits around local inference

Local inference has a habit of becoming folklore. You launch a model from a shell history fragment you can never quite find again. A context setting lives in a note somewhere. A faster quantization ge