How Models Actually Work (weights, Parameters, Active Parameters)

A model is a big pile of numbers. Specifically:

WEIGHTS are the numbers. They're learned during training and frozen afterward. When you "download a model," you're downloading the weights.

PARAMETERS are how we count weights. A "7B parameter model" has 7 billion weights. A "70B parameter model" has 70 billion weights.

The number of parameters roughly correlates with capability. More parameters = (usually) smarter model = (always) more memory required to run = (usually) slower inference. The trade-off curve is non-linear: going from 7B to 13B is a noticeable jump in quality; going from 70B to 405B is incremental at huge cost.

Why parameter count matters for hardware:

Each parameter takes up space in memory. The space depends on the "precision" of the model:

• FP16 (full precision): 2 bytes per parameter
• FP8 / Q8: ~1 byte per parameter
• Q4 (4-bit quantized): ~0.5 bytes per parameter
• Q2 (2-bit quantized): ~0.25 bytes per parameter

A 7B model at FP16 needs ~14GB of memory. The same model at Q4 needs ~3.5GB. Quantization (the process of squashing the model into smaller numbers) trades a small amount of quality for a large amount of memory savings.

The rule: take the parameter count in billions, multiply by the bytes per parameter, that's your minimum RAM/VRAM. Add 2-4GB for overhead.

ACTIVE PARAMETERS is a newer concept that matters for "mixture of experts" (MoE) models. An MoE model has many parameters total, but only some of them activate for any given token. A model labeled "M2.7" (or similar) might have 200B total parameters but only 20B active per token. The total parameter count determines memory footprint. The active parameter count determines speed and (roughly) capability.

What you need to know when buying:

When someone says "this model is 70B parameters," they mean total. You need 35-140GB of memory to run it depending on quantization.

When someone says "this model has 30B active parameters of 200B total," they mean: capability roughly like a 30B dense model, memory footprint like a 200B model. The MoE pattern lets you trade memory (which is cheap if you have enough) for compute (which gets expensive).

For most people self-hosting in 2026: a 13B-30B model at Q4 or Q5 quantization is the sweet spot. Smart enough to be useful, small enough to run on a Mac mini or a single consumer GPU.