Writing · cuda

1 post tagged cuda, newest first.

Running a 27B model on a 3090: a budget, not a flag

[2026-10-16] / infrastructure

Fitting a 27B model on one 24GB card is two arithmetic unknowns - weight memory at your quantisation and KV cache at your context - and a curve, not a number.

[ read ] llm cuda quantisation benchmarking

󰣨 ymrtech@ymrtech | 󰌠 NixOS | 󰍢 UTF-8