Self-hosting On Nvidia (spark, Orin, Full Gpus)

NVIDIA's strategy is to build a continuum from "tiny edge device" to "full data center" so you can scale up without changing your software stack. The relevant tiers for individuals and small businesses:

JETSON ORIN is the small/edge tier. A Jetson Orin Nano or Orin NX is a small computer with an integrated GPU optimized for AI workloads, priced from a few hundred dollars to a couple thousand. Power-efficient (can run on a wall outlet 24/7 at low cost). Runs smaller models (think 3B-13B) comfortably. Good for: a robotics project, a kiosk-like agent, a "smart" embedded device, learning the NVIDIA software stack without spending big.

DGX SPARK is the workstation tier. NVIDIA's compact AI workstation designed for individual researchers and small teams. Can run larger models (70B+) and handle real workloads. Pricier than a Mac Studio for roughly equivalent capability, but with NVIDIA's full software stack (CUDA, TensorRT, NIM containers) which matters if you're going to do serious ML work beyond just running pre-trained models.

CONSUMER GPUS (RTX 4090, 5090) in a regular PC. The traditional self-hoster path. Build a PC with one or two consumer GPUs, install Ubuntu or Windows, run Ollama or vLLM. 24-32GB of VRAM per card. With two cards, you can fit a 70B model. Pricing: $2,000-$4,000 per card.

WORKSTATION GPUS (RTX 6000, A6000) when you need more VRAM in a single card. Pricier than consumer cards, but 48GB of VRAM in one slot. Used to run 70B models without splitting across cards.

DATA CENTER GPUS (H100, B200) when none of the above are enough. You are not buying these for personal use. You are renting them by the hour on AWS, Azure, GCP, or RunPod.

The software stack on NVIDIA:

OLLAMA works on Linux with NVIDIA GPUs. Same as on Mac.

VLLM runs natively on CUDA. The standard for production inference on NVIDIA hardware.

NVIDIA NIM is NVIDIA's containerized inference service. Drop-in deployment, optimized for their hardware.

The honest comparison with Mac:

For a SOLO USER doing local coding assistance, Mac mini or Mac Studio is simpler. Plug in, install Ollama, done.

For a SMALL TEAM sharing inference, NVIDIA DGX Spark or a custom box with two consumer GPUs gives more flexibility and better multi-user support.

For PRODUCTION DEPLOYMENT (real customers hitting your API), NVIDIA is the path. The Mac path tops out before production scale.

For LEARNING the broader ML ecosystem, NVIDIA. CUDA is the lingua franca. Most ML papers, tutorials, and frameworks assume NVIDIA first.