I wanted private AI. What I had to decide was Kubernetes.

The other day I wrote about getting an open model running on my own laptop — private, offline, no bill. The part that stuck with me afterward was what I never needed: a GPU. It answered faster than I read, on hardware I already owned. So when I sat down to do the serious version — private AI I could actually put in front of other people — I assumed the first question would be which accelerator to buy. It turned out to be the last question. I never bought one.

Here’s the reframe that took me too long to see. If the model runs on CPUs you already have, then “private AI” isn’t a hardware project. A model other people can use is a container: something that has to be scheduled onto a machine, given storage, put on a network, health-checked, updated without downtime, and locked down. None of that is an AI problem. All of it is a Kubernetes problem. The GPU question is a distraction from the only question that actually costs you anything — what runs the container.

And that question has a specific shape if you’re one of the many shops that care about “private” because you’re regulated, or sovereign, or just allergic to shipping your data into someone else’s tenancy — because a lot of those shops already run vSphere. On vSphere you now have two doors to Kubernetes. Door one is the one almost everyone walked through for the last five years: bring your own — stand up Rancher, run RKE2 clusters on top of your VMs. Door two is newer, and most people haven’t noticed it’s open: use the Kubernetes that’s now built into the platform itself. VMware calls it vSphere Kubernetes Service. If your team is about to pick a Kubernetes for its private-AI work, it’s worth knowing the second door exists before you queue up the first.

The case for door two is mostly subtraction. vSphere’s Supervisor turns clusters you already operate into a Kubernetes control plane — you’re not standing up a second platform to patch, secure, monitor, and staff next to the one you already have; the Kubernetes is the vSphere. And the single biggest reason teams used to skip it is gone. It used to require NSX, VMware’s software-defined networking — a heavy prerequisite that, for a lot of teams, was reason enough to just run Rancher instead. It doesn’t anymore. I brought a Supervisor all the way up on an ordinary distributed switch with the load balancer that’s now built in — no NSX anywhere in the picture. That one change quietly removes the tax that pushed people toward bring-your-own in the first place.

Now the part the brochure skips, because I actually did this and not on a slide. Bringing the control plane up is real work, and almost all of it is networking. The load-balancer wiring and a floating-IP-versus-strict-reverse-path-filtering class of problem cost me an afternoon — and they cost it quietly, because they fail silent: the endpoint answers perfectly from the box itself and is stone dead from everywhere else, with nothing in any log to tell you why. That’s the genre of bug to budget for. Anyone who tells you it’s a wizard and a cup of coffee hasn’t stood one up.

There’s one piece I’d prove before I moved anything real onto it: the supply chain for the clusters themselves — how the worker-node images, and the container images underneath them, actually get delivered to your clusters. It’s the least forgiving part of the whole system and the first thing that bites in a tightly controlled environment, which is exactly the kind of environment that wants private AI in the first place. Stand up one cluster and watch that path end to end before you believe the demo.

None of this is a case for ripping Rancher out. Rancher has the more mature day-two story today — the fleet management, the upgrade muscle memory your team already has — and that fluency is worth real money. So don’t think of it as a swap; think of it as a migration you take at your own pace. Run both. Put new workloads on the built-in path as you prove that supply chain out, and let “one fewer control plane to operate” compound over a couple of quarters instead of over a weekend.

Circle back to where I started. On either of those platforms, a CPU-sized open model just… runs. The model was never the hard part and the GPU was never the gate. What you’re actually choosing when you choose “private AI” is who operates the platform underneath the model: a second product you bolt on and babysit, or the one you already pay for, already patch, and already govern with the controls you already trust.

I went looking for a GPU and came home with a platform decision. I think that’s the real shape of private AI for most teams — not a line item on a hardware quote, but a placement choice you either make on purpose or make by default. If you already run vSphere, make it on purpose. At the very least, walk over and check whether that second door is already open before you go build another one next to it.