lately

I put a brain in a potato

Brian Stever

2025 · Raspberry Pi 5, Python, Pygame, Ollama, faster-whisper

The short version. The potato is a Raspberry Pi 5. The brain is a voice assistant that runs entirely on it. It started as one app and turned into a small family of local assistants that share the same hardware limits. oracle.py is the theatrical Matrix one, pocket_doctor.py is a more clinically useful nursing reference, and voice_assistant.py tries the same idea with a heavier model. The question never changed: how much voice assistant can you fit on a tiny Linux box before the box starts radiating concern?

1Why a Pi

I built the first version because I wanted a voice assistant that didn't send my questions to a server farm in Virginia. That's the principled explanation. The honest one is that I'd been playing with Ollama on my laptop and started wondering what would happen if I ran it on the smallest computer I owned, a Raspberry Pi 5 that was sitting on my desk looking available.

Most voice assistants feel useful the way hotels feel luxurious: the experience works because a lot of infrastructure you never see is standing behind the curtain. I wanted to find out how much of that I could reproduce on one device with no cloud calls. The Pi 5 was the right target, powerful enough to tempt you and limited enough to punish you.

2How it grew

It started as one persona and kept splitting. The first was the Matrix-flavored oracle, running llama3.2:1b with a tiny Whisper model and an aggressively unserious personality prompt. Once the hardware worked, I started wondering what else it could be. Pocket Doctor came next: a nursing reference that switches to smollm2:1.7b and a clinical voice I built while working on OslerAI. A third version uses llama3.2:3b for fuller answers. Same box, same touchscreen, different social ambitions.

That turned one novelty into a small design study. What changes when the same limited hardware has to feel mystical in one mode and clinically useful in another? Quite a lot, and almost none of it is adding buttons.

Table 1. The three versions.

VariantModel + personality
OracleMatrix-style, Gen Z-coded assistant using llama3.2:1b
Pocket DoctorClinical reference tool using smollm2:1.7b and base Whisper
Voice AssistantHeavier clinical build using llama3.2:3b

3Talking to it

The Oracle's interface is deliberately minimal. A tap starts listening. Your speech is transcribed on the device, run through a local language model and printed back as text on a full-screen fake terminal. A long press exits. A double tap clears the history. That's more or less the device's whole vocabulary.

The minimalism was partly taste and partly tactics. There wasn't room for a heavy UI layer, and I didn't want the interface explaining itself too much. It needed to be clear without getting chatty. A blinking cursor, a recording state and a visible stream of words were enough.

TERM_01ONLINE

> ORACLE_OS loaded

> Listening...

> What is the meaning of life?

Processing...

The void suggests you already know. But also: connection, curiosity, and not taking yourself too seriously.

Figure 1. A typical exchange on the Pi's display, terminal style. There's no decorative chrome: changes of state show whether it's listening, running the model or idle.

4Fitting it in 8GB

The main limit was memory. An 8GB Raspberry Pi sounds generous until the operating system, the audio pipeline, the screen, the transcription model and the language model all want their share at once. So picking a model stopped being purely about quality and became a systems decision.

The Matrix Oracle goes for faster, lighter inference with llama3.2:1b and tiny Whisper. Pocket Doctor spends a bit more of the budget on transcription quality and uses smollm2:1.7b. That kind of call sounds philosophical until you're standing over a small hot computer at 11 p.m., wondering whether the better model is worth three more seconds of latency.

silence detection

def callback(indata, frames, time_info, status):
    state.recording_data.append(indata.copy())
    level = np.max(np.abs(indata))

    if level > SILENCE_THRESHOLD:
        has_speech = True
        silence_start = None
    elif has_speech and time.time() - silence_start > SILENCE_DURATION:
        state.stop_recording = True

streamed generation

resp = requests.post('http://localhost:11434/api/generate',
    json={'model': 'llama3.2:1b', 'prompt': full_prompt, 'stream': True},
    stream=True)

for line in resp.iter_lines():
    if line:
        token = json.loads(line)['response']
        oracle_msg.text += token

5What got in the way

The first problem was feedback. A normal app can use loaders, progress bars and microcopy to tell you what's happening. A fake terminal has fewer options. I ended up with a small visual vocabulary: red or active states while it's listening, a processing indicator during inference and a blinking cursor when it's idle. It's a tiny UX system, but it does real work.

The second was heat. Speech recognition plus local inference makes the Pi work very hard for a very small box, so active cooling became part of the architecture. A mystical oracle that needs a fan is charming, but it does make the illusion a little more mechanical.

The third was development. A full-screen Pygame UI on a Pi touchscreen is not friendly when it crashes. Remote access over VNC became essential, because it kept a terminal in view while I worked on the device's interface. So the magic mirror got debugged like any other Linux box, which is probably healthy.

6Why the limits help

Pi Oracle convinced me that local AI gets more interesting when the hardware is tight. Scarcity forces taste. You have to decide what the device is actually for, which feedback matters and which model is good enough, instead of whichever is best on paper. And the constraints come free with an $80 computer.

It also confirmed that I like building objects that are a bit theatrical, as long as the theatrics serve the interaction. This one is part assistant, part prop, part clinical reference and part thermal management exercise. I showed it to my mom and she asked it a medical question. It gave a reasonable answer. Then the fan got loud and she asked if it was okay. I'm fond of every part of that story.