Learning by Doing
I wanted to help companies who can’t move to the cloud, but I also wanted to understand how local AI really works. So, I built a simple local test environment from gear I already had at home.
This wasn’t some high-end datacenter setup. It was just a gaming laptop. But it helped me test different tools, run models, and figure out what’s possible with today’s local AI software.
My First Setup: ASUS Gaming Laptop
Specs:
-
GPU: NVIDIA GeForce 2060 Ti
-
CPU: Intel i7 (8th Gen)
-
RAM: 16 GB
-
Storage: 512 GB SSD
This setup allowed me to run lighter models (4GB and 8GB) without too much trouble. But the biggest lesson came quickly: model size doesn’t mean file size.
If a model says 16GB, it means you need that much free VRAM or RAM. And if your OS is already using some, you better go smaller. That’s why 4GB models worked much better for me in the beginning — until I realized that a smaller model also hallucinates more easily or cannot follow more complex tasks.
Setup Time and Struggles
Getting my first model running took around a day and a half. I used LM Studio, and while it eventually worked, there were lots of small issues: driver updates, memory errors, weird performance drops.
But once I got it working, the result was worth it. I had a private, local AI running on my machine.
What Surprised Me
-
Smaller models often hallucinate: You ask a question and they answer, but forget the context quickly.
-
Lightweight = Limited: With only 16GB RAM, I couldn’t run anything big. But I still learned a lot.
-
Open-source tools are amazing: You can get real results with Ollama, OpenWebUI, or LM Studio – even on a modest machine.
My Next Setup: Bigger, Better, Still Local
To push further, I’m now building a desktop with:
-
Ryzen CPU
-
NVIDIA 3060 Ti GPU
-
32 GB RAM to start, expandable to 128 GB
Once this setup is ready, I’ll test larger models, see how much context I can manage, and try real business workflows like document classification, CRM support, and internal chat.
Final Thoughts
Building your own local AI lab isn’t for everyone. But if you want to learn, test, or prepare for AI use cases in secure environments, there is no better teacher than your own setup.
The next article will dive into the tools I tried – which ones I liked, and which ones I won’t use again.
Markku Arvekari
Digital Transformation Expert
