|
AI Collab Score: 7 / 3
I am currently pursuing my degree in Artificial Intelligence, and one of the things I enjoy most about the coursework is taking what we are learning academically and testing it in my own AI lab.
This week, one of my classes focused on cloud-based AI platforms. We experimented with Amazon Rekognition and Google AI Studio to compare how cloud services can analyze images, what kinds of results they return, and how their approaches differ. That naturally led me to another question: What happens if I move that same general image-understanding capability out of the cloud and run it locally? I already have a home AI lab built for exactly this kind of experimentation, so I decided to find out. Taking the Classroom Experiment On-Premises
Figure 1. AI Lab Live Vision running locally on Ubuntu. Webcam frames are analyzed by a locally hosted Qwen vision-language model through Ollama and ROCm.
For the lab, I built a browser-based vision application that captures an image from a webcam and sends it to an AI model running entirely on my local workstation.
My current lab includes:
The basic architecture looks like this:
Webcam
↓ OpenCV ↓ Local Flask Application ↓ Ollama ↓ Qwen Vision-Language Model ↓ ROCm ↓ AMD Radeon AI PRO R9700 ↓ Local Result
The important difference is that the image does not need to be sent to Amazon, Google, or another external service.
The model, GPU processing, inference, and response all remain local. Rekognition and Qwen Are Not the Same Kind of AI
There is an important distinction here.
Amazon Rekognition is primarily a specialized computer-vision service. It is designed to return structured outputs such as:
A response might conceptually look like:
Person: 99.8%
Chair: 98.2% Computer: 96.4% Furniture: 94.7%
Qwen3-VL, however, is a vision-language model, or VLM.
Instead of only labeling what is present, it can interpret the overall scene and explain what it sees in natural language. For example, my local system might return something like: One person is visible seated at a desk in what appears to be an office or study area. Multiple monitors, framed artwork, furniture, and a resting dog are visible in the scene. That is a very different interaction model. My local lab is not simply recreating Amazon Rekognition on-premises. It is using a different class of AI model to solve a similar image-understanding problem. Running Qwen3-VL 4B Locally
The first model I tested was Qwen3-VL 4B.
In one recent run, the application reported:
The model-load time is important.
Of the 15.85 seconds of total latency, more than 11 seconds came from loading the model into GPU memory. Once loaded, the actual vision processing and response generation took only several seconds. That is a strong result for a local multimodal model running on a single workstation GPU. Then I Increased the Model Size to 27 Billion Parameters
The next test was much more interesting.
I moved to Qwen3.5 27B, a substantially larger multimodal model. I wanted to know whether the Radeon AI PRO R9700 could keep the entire model on the GPU rather than offloading part of it into system memory. It could. A recent run produced: 
Figure 2. Qwen3.5 27B running locally on the same workstation. The larger multimodal model remained fully GPU-resident while using about 18 GB of VRAM.
Again, the cold model load accounted for most of the total latency.
Once resident in VRAM, the actual inference portion was much closer to roughly ten seconds. More importantly, the model remained entirely on the GPU. That is a meaningful jump from the RTX 3060 12 GB configuration I previously used in this lab. The RTX 3060 was excellent for smaller models, but 12 GB of VRAM quickly became the limiting factor. The R9700’s 32 GB of VRAM opens the door to a very different class of local AI workload. What the Larger Model Also Taught Me
The larger model produced richer descriptions, but it also demonstrated something else we have been studying throughout the AI program.
It sometimes inferred more than the image actually proved. For example, the model described visible objects and then began making assumptions about:
Those conclusions may sound reasonable, but they are not necessarily supported by the image itself. That creates an important contrast between the two approaches. A specialized computer-vision service may remain relatively constrained:
A vision-language model may go further:
That additional reasoning can be useful, but it also increases the risk of hallucination, over-inference, and unsupported conclusions. This was one of the most useful connections between the classroom material and the hands-on lab. On-Premises Does Not Automatically Mean Better
Running the model locally has obvious advantages.
The webcam image can remain inside the local environment, and the organization has greater control over:
But that does not mean on-premises AI is automatically the right choice. Cloud platforms provide major benefits:
Running AI locally introduces responsibilities around:
The right answer depends on the workload. The Bigger Lesson
This experiment started as an extension of something I was learning in my AI coursework, but it reinforced a much broader architectural point.
The question is not only: Which AI model or service performs best? Organizations also need to decide: Where should the AI run? That might be:
Amazon Rekognition and Google AI Studio gave me the cloud-service perspective. My home lab gave me the opportunity to take the same general problem and explore what it looks like when the AI runs locally and under my own control. And perhaps the most important lesson is this:
That is the part I find most interesting about pursuing this degree: being able to take the concepts we discuss in class and immediately test what they look like on real hardware.
0 Comments
Your comment will be posted after it is approved.
Leave a Reply. |
Categories
All
Recognition
|
