LLM memory
at , with tokens of context for 4 users, needs 25 GB: it fits on one RTX 5090.
The number of users can be changed: drag it, use the arrow keys, or tap it to type a value.
- Weights
- Context cache
- Total
1,024 tokens262,144 tokens
Move along the curve with the pointer or the arrow keys to read each context. A click or the Enter key selects it.
- Weights
- 14 GB
- Context cache
- 8.6 GB
- Margin
- 2.2 GB
An estimate of memory, not speed. Configurations read on Hugging Face on 4 November 2026.
Method
- 01
The weights
The number of parameters, times the bits of each parameter, divided by 8.
- 02
The cache
For each user, what the model keeps of each token, from its published configuration, at 16 bits.
- 03
The margin
An assumption of 10% for running computations and the software. Real memory depends on the server used.
Questions
Why does the context take so much room?
The model keeps a trace of every token it has read, for every user. That trace grows with the conversation. Some recent models cut it sharply: compare Llama 3.3 with DeepSeek V4.
Does 4-bit quantization hurt the model?
A little, depending on the model and the method. Some are published directly at 4 or 8 bits, such as gpt-oss. Test on your own tasks before deciding.
Does a mixture-of-experts model take less room?
No. All its experts stay in memory, even if only a few work on each token. It answers faster but takes the same space.
Can several cards be added up?
Yes, within one server, up to eight in this calculation. A Mac Studio or a DGX Spark count here as a single machine.
How exact are these numbers?
They are orders of magnitude. DeepSeek V4’s compressed cache is an estimate. The serving software and an 8-bit cache also change the result.
Updated 4 November 2026