1Why a Pi
I built the first version because I wanted a voice assistant that didn't send my questions to a server farm in Virginia. That's the principled explanation. The honest one is that I'd been playing with Ollama on my laptop and started wondering what would happen if I ran it on the smallest computer I owned, a Raspberry Pi 5 that was sitting on my desk looking available.
Most voice assistants feel useful the way hotels feel luxurious: the experience works because a lot of infrastructure you never see is standing behind the curtain. I wanted to find out how much of that I could reproduce on one device with no cloud calls. The Pi 5 was the right target, powerful enough to tempt you and limited enough to punish you.
2How it grew
It started as one persona and kept splitting. The first was the Matrix-flavored oracle, running llama3.2:1b with a tiny Whisper model and an aggressively unserious personality prompt. Once the hardware worked, I started wondering what else it could be. Pocket Doctor came next: a nursing reference that switches to smollm2:1.7b and a clinical voice I built while working on OslerAI. A third version uses llama3.2:3b for fuller answers. Same box, same touchscreen, different social ambitions.
That turned one novelty into a small design study. What changes when the same limited hardware has to feel mystical in one mode and clinically useful in another? Quite a lot, and almost none of it is adding buttons.
Table 1. The three versions.
| Variant | Model + personality |
|---|---|
| Oracle | Matrix-style, Gen Z-coded assistant using llama3.2:1b |
| Pocket Doctor | Clinical reference tool using smollm2:1.7b and base Whisper |
| Voice Assistant | Heavier clinical build using llama3.2:3b |
3Talking to it
The Oracle's interface is deliberately minimal. A tap starts listening. Your speech is transcribed on the device, run through a local language model and printed back as text on a full-screen fake terminal. A long press exits. A double tap clears the history. That's more or less the device's whole vocabulary.
The minimalism was partly taste and partly tactics. There wasn't room for a heavy UI layer, and I didn't want the interface explaining itself too much. It needed to be clear without getting chatty. A blinking cursor, a recording state and a visible stream of words were enough.
> ORACLE_OS loaded
> Listening...
> What is the meaning of life?
Processing...
The void suggests you already know. But also: connection, curiosity, and not taking yourself too seriously.
4Fitting it in 8GB
The main limit was memory. An 8GB Raspberry Pi sounds generous until the operating system, the audio pipeline, the screen, the transcription model and the language model all want their share at once. So picking a model stopped being purely about quality and became a systems decision.
The Matrix Oracle goes for faster, lighter inference with llama3.2:1b and tiny Whisper. Pocket Doctor spends a bit more of the budget on transcription quality and uses smollm2:1.7b. That kind of call sounds philosophical until you're standing over a small hot computer at 11 p.m., wondering whether the better model is worth three more seconds of latency.
silence detection
def callback(indata, frames, time_info, status):
state.recording_data.append(indata.copy())
level = np.max(np.abs(indata))
if level > SILENCE_THRESHOLD:
has_speech = True
silence_start = None
elif has_speech and time.time() - silence_start > SILENCE_DURATION:
state.stop_recording = Truestreamed generation
resp = requests.post('http://localhost:11434/api/generate',
json={'model': 'llama3.2:1b', 'prompt': full_prompt, 'stream': True},
stream=True)
for line in resp.iter_lines():
if line:
token = json.loads(line)['response']
oracle_msg.text += token5What got in the way
The first problem was feedback. A normal app can use loaders, progress bars and microcopy to tell you what's happening. A fake terminal has fewer options. I ended up with a small visual vocabulary: red or active states while it's listening, a processing indicator during inference and a blinking cursor when it's idle. It's a tiny UX system, but it does real work.
The second was heat. Speech recognition plus local inference makes the Pi work very hard for a very small box, so active cooling became part of the architecture. A mystical oracle that needs a fan is charming, but it does make the illusion a little more mechanical.
The third was development. A full-screen Pygame UI on a Pi touchscreen is not friendly when it crashes. Remote access over VNC became essential, because it kept a terminal in view while I worked on the device's interface. So the magic mirror got debugged like any other Linux box, which is probably healthy.
6Why the limits help
Pi Oracle convinced me that local AI gets more interesting when the hardware is tight. Scarcity forces taste. You have to decide what the device is actually for, which feedback matters and which model is good enough, instead of whichever is best on paper. And the constraints come free with an $80 computer.
It also confirmed that I like building objects that are a bit theatrical, as long as the theatrics serve the interaction. This one is part assistant, part prop, part clinical reference and part thermal management exercise. I showed it to my mom and she asked it a medical question. It gave a reasonable answer. Then the fan got loud and she asked if it was okay. I'm fond of every part of that story.

