Compute, Models, and Cost
Pick a GPU from VRAM requirements, not vibes, and know what your work costs in month thirteen.
Before this:01 · The New Foundation for Researchers
Free compute teaches worse habits than expensive compute, because nothing corrects you. Spend the free year learning what your work actually costs and month thirteen is an invoice you predicted; skip it and month thirteen is a surprise you get to explain to whoever funds you. The meter is off right now, which makes this the safest possible time to learn how fast it spins.
Learning outcomes
- Select a GPU tier from a model's VRAM requirements rather than by trial and error.
- Explain parameters, active parameters, and what quantization trades away.
- Choose between Colab, Vertex AI, and hardware you own for a given workload.
- Produce a monthly cost figure for your own research workflow after the student offer ends.
What you already have
Colab AI. The 2025/26 AI-first redesign turns the notebook runner into a development partner:
- Data Science Agent: describe an analysis goal in plain English and it generates a complete working notebook, runs it, and iterates on results. Raw CSV to trained model and visualisations without writing a line.
- Learn Mode: step-by-step guided explanations instead of just code, for when you want the underlying logic.
- Intelligent Error Fixing: click any error for an explanation and a suggested fix.
- Drive integration: mounts directly to Drive, which is also the
@Drivegrounding source.
All four are available on every tier, including free.
GPU access by tier:
| GPU | VRAM | Best for |
|---|---|---|
| T4 (free tier) | 15 GB | Prototyping, fine-tuning small models, most coursework |
| L4 (Pro) | 22.5 GB | Transformer fine-tuning, medium training runs |
| A100 40 GB (Pro+) | 40 GB | Large model training, LLM work, serious research |
| A100 80 GB (Pro+) | 80 GB | Very large models, multi-GPU experiments |
Pro+ also unlocks background execution for up to 24 hours. Colab's disk resets between sessions. Checkpoint to Drive or lose the run.
Your Google Cloud credits unlock:
- Vertex AI: managed ML training, model deployment, AutoML. A production-ready Colab: persistent environments, guaranteed GPU access, real API endpoints.
- Model Garden: one-click access to 200+ models including Gemini, Llama, Mistral, and Stable Diffusion. Useful for comparing models or deploying a fine-tuned model for a thesis demo.
- Document AI: structured data out of PDFs, scanned documents, and forms. Basic OCR covers roughly 200,000 pages on a $300 credit.
- GPU VMs: dedicated T4, L4, or A100 instances with no session timeouts. Spot VMs run 60–90% cheaper; pair them with checkpoint saves for long training runs.
- BigQuery: 1 TB of free SQL queries per month plus 200+ public datasets, pullable into Colab with one line of Python.
Month 13
The offer ends after twelve months and auto-renews at $19.99/month unless you cancel. Before that date, know which of these you actually depend on:
- Costs money after: Gemini Advanced, Colab Pro GPU hours, Vertex AI endpoints, anything past BigQuery's free tier.
- Still free: the gemini-cli free tier, Colab's T4 when one is available, BigQuery's 1 TB/month, and every open-weights model you can run on hardware you already own.
Part 4 of the curriculum is the answer to the second column. §4.1 (Frontier vs. local), §4.4 (Self-hosting on Mac), and §4.5 (Self-hosting on NVIDIA) are where a free year becomes a durable setup instead of a cliff.
Concept
Parameter counts are marketing; VRAM is physics: the model either fits in memory or it does not, and every other spec is negotiable after that one. §4.6's $200 question is about what your setup costs when somebody is actually paying, and for you somebody will be, thirteen months from now.
Read §4.3 (RAM, VRAM, and unified memory) before you pick a tier, and §4.2 (How models actually work) before you assume a bigger parameter count means a better result. §4.6 (Cost literacy) is the unit's spine.
/stacks/local-agents-nvidia is the shape of a setup that does not expire.
Paired instantiation
| Portable idea | Google (what you have) | Internet Menace (the method) |
|---|---|---|
| VRAM decides what fits | T4 15 GB → A100 80 GB | §4.3: RAM, VRAM, unified memory |
| Managed vs. owned compute | Colab, Vertex AI | §4.4, §4.5: self-hosting; /stacks/local-agents-nvidia |
| Bigger is not automatically better | Model Garden's 200+ models | §4.2: weights, parameters, active parameters |
| Know the per-unit cost | Compute units, credits | §4.6: cost literacy |
Lab
- Pick a model you would actually fine-tune. Find its parameter count and estimate the VRAM it needs at full precision, then at 8-bit and 4-bit quantization.
- Map that estimate onto the tier table. Which is the smallest GPU that fits?
- Run the fine-tune on Colab at that tier. Checkpoint to Drive, not to the local disk. Confirm by restarting the runtime and reloading from the checkpoint.
- Price the same run three ways: Colab Pro compute units, a Vertex AI GPU VM (then again as a Spot VM), and hardware you would buy once.
- Pull a public BigQuery dataset into the notebook with one line of Python and confirm it lands against your free 1 TB.
- Write your month-13 number: what your current workflow costs per month once the offer ends, and which line items disappear if you move them local.
Deliverable
A compute and cost plan for your real thesis workload: chosen GPU tier with the VRAM reasoning, the three-way price comparison, and the month-13 monthly figure with the local-equivalent alternatives named.
Self-check
- Your fine-tune OOMs on an L4. Name three things you can change, and what each one costs you.
- What does quantization trade away, and when is that trade unacceptable for research you intend to publish?
- Your Colab session ended and the model is gone. What did you fail to do?
- What is the cheapest configuration that would still let you finish your thesis in month 14?