Menu
Wire an ESP32, a microphone and a tiny screen to Claude, then just talk. This guide takes you from a bag of parts to a working Buddy, one checked step at a time.
This is Buddy's real screen layout with sample replies. Pick a sentence to watch one turn from start to finish.
Claude understands text, not sound, and an ESP32 is far too small to understand speech by itself. So the work is shared between your desk, your computer and the cloud.
The program on your computer that does the middle steps is called the relay. Your Claude key lives inside the relay and never goes onto the ESP32. That also means your computer has to be on, awake and running the relay whenever you want to talk to Buddy.
| Part | What it does | Good to know |
|---|---|---|
| ESP32 DevKit, 38 pins | The brain. It has Wi-Fi built in. | A full-size breadboard fits it best, because this board is wide. |
| 0.96 inch OLED screen (SSD1306, I2C, 4 pins) | Buddy's face and its written replies. | Pin order differs between brands, so read the labels printed on yours. |
| INMP441 microphone module | Hears you. | Power it from 3V3 only. 5V can destroy it. |
| MAX98357A amplifier and an 8 Ω 0.5 W speaker | Plays Buddy's voice from the ESP32. | Optional on a Mac, where the relay can use your computer's speakers instead. |
| Full-size breadboard and jumper wires | Joins everything without soldering. | About 20 male-to-male wires, plus a few spares. |
| USB data cable | Powers the ESP32 and loads programs onto it. | Some cheap cables only charge. Check whether your ESP32 has a micro-USB or a USB-C socket. |
If your OLED, microphone or amplifier came with a strip of pins that is not soldered on, solder it on. Loose pins pressed into the holes seem fine for a minute and then fail in confusing ways.
Moving wires on a powered board is the quickest way to damage a part.
Connect the INMP441 to the 3V3 pin, never to 5V or VIN. 5V can destroy it.
You will install four things: Python (it runs the relay), the project files, the Arduino IDE (it loads programs onto the ESP32) and the ESP32 add-on for the Arduino IDE. It takes about 20 minutes, most of it waiting for downloads.
Pick your computer and the steps below will match it.They follow the same flow as the Mac steps but were not run on a real PC. If one is wrong for you, the troubleshooting section near the end lists the usual fixes.
.pkg file). Versions 3.10 to 3.12 are the safest.Terminal and press Enter. Then run:python3 --versionPowerShell and press Enter. Then run:python --versionIt prints something like Python 3.12.x.
If a window offers to install developer tools instead, close it, quit Terminal, open it again and retry.
If it says python is not recognized, run the installer again and tick the PATH box, then open a new PowerShell window.
Double-click the zip in your Downloads folder. A folder called ai_buddy_project appears. Drag it into your Documents folder.
Right-click the zip in your Downloads folder, choose Extract All, and extract it into your Documents folder. You should end up with a folder called ai_buddy_project.
cd followed by one space. Do not press Enter yet.ai_buddy_project and drag the relay folder into the Terminal window. Its path appears after cd . Now press Enter.ls and press Enter. If you see relay.py in the list, you are in the right place.python3 -m pip install -r requirements.txtai_buddy_project, then the relay folder.powershell and press Enter. A PowerShell window opens already inside that folder.python -m pip install -r requirements.txtThe last lines say Successfully installed ... and there is no red error text.
If the later step fails with an error about a missing DLL, install the Microsoft Visual C++ Redistributable and try again.
Go to arduino.cc/en/software and download Arduino IDE 2.x for macOS. Choose Apple Silicon for an M1, M2, M3 or M4 Mac and Intel otherwise (Apple menu, About This Mac, tells you). Open the download and drag Arduino IDE into Applications. If your Mac refuses to open it, right-click the app and choose Open.
Go to arduino.cc/en/software and download Arduino IDE 2.x for Windows 10 and newer. Run the installer and accept the defaults.
https://espressif.github.io/arduino-esp32/package_esp32_index.jsonesp32.Open the Library Manager (also in the left bar), search for Adafruit SSD1306 and click Install. When a window asks about dependencies, click Install all.
Plug the ESP32 into your computer. In the Arduino IDE open Tools, Port.
A new entry appears, such as /dev/cu.usbserial-0001 or /dev/cu.SLAB_USBtoUART. Always pick the one that starts with cu.
A new entry appears, such as COM3. The number can be different on your PC.
If no new port appears, try a different cable first (it must carry data), then a different USB port. Next look at the small chip near the USB socket. If it says CP2102, install the CP210x driver from Silicon Labs. If it says CH340, install the CH340 driver. On a Mac you may need to allow the driver in System Settings, Privacy & Security, and restart.
Tools, Port shows a new port whenever the ESP32 is plugged in.
Change wires only while the ESP32 has no power.
Every pin on the ESP32 has a label printed beside it. Some boards print D32, some 32, IO32 or GPIO32. They all mean the same pin. 3V3 is the 3.3 volt pin, VIN is the 5 volt pin, and there are several GND pins that all work the same.
Use whichever wires you have. If you can, follow these colours, because then a red wire should only ever touch a power pin, and anything else that is red is a mistake you can see.
The microphone and the amplifier talk to the ESP32 with a standard called I2S. It always uses the same three jobs: a clock wire that keeps time, a select wire that says left or right, and a data wire. The screen uses a simpler standard called I2C, with its own clock (SCL) and data (SDA) wires.
Plug the ESP32 into the breadboard so its two rows of pins sit on either side of the middle gap. Then:
Use the printed stripes on your breadboard as the guide, even if your ground wire is black.
The order of the pins differs between brands, so read the labels printed on your own modules before you plug anything in.
On the microphone, SD means "serial data out". On the amplifier, SD means "shutdown". They are different things, so do not mix them up. Leave the amplifier's SD and GAIN pins empty.
Connect the two speaker wires to the amplifier's output terminals, the two screw terminals or solder pads marked + and −. With a single speaker it does not matter which wire goes where. If you plan to use your computer's speakers instead, you can skip the amplifier and speaker for now.
Now plug the USB cable back in.
Before Wi-Fi, your computer and Claude come into it, check that each part works on its own. A small test program in the project does this, and it will save you hours of guessing later.
hardware_check/hardware_check.ino from the project folder.Connecting........ for more than 10 seconds, press and hold the BOOT button on the ESP32 until the upload starts, then let go.Done uploading, open Tools, Serial Monitor and set the speed at the bottom right to 115200.Do not move on until the screen and the microphone work. The speaker is optional, as you will see below.
| What you see | What to check |
|---|---|
| The screen stays blank | SDA on pin 32 and SCL on pin 33 (try swapping them), plus VCC and GND. If it is still blank, upload i2c_scan/i2c_scan.ino. It lists every I2C device it finds, and your screen should appear as 0x3C. Odd addresses such as 0x0D or 0x52 mean a wire is loose. |
| No beep | Amplifier VIN to the ESP32 VIN pin, GND, then BCLK 14, LRC 13 and DIN 27. Check that the speaker wires are tight in the terminals. |
| The mic bar never moves | Microphone VDD on 3V3, L/R on GND, then SCK 26, WS 25 and SD 35. If all of that is right, open the file, change MIC_SLOT = 0 to MIC_SLOT = 1 and upload again. |
| Upload fails with "port not found" | Use a data USB cable, and see the USB check in step 1. |
You do not need the amplifier at all. On a Mac, the relay can play Buddy's voice through your computer's own speakers, while the microphone and screen stay on the ESP32. The project's Mac launcher already does this.
Playing through the computer's speakers uses a Mac command, afplay. On Windows, Buddy's voice has to come from the amplifier and speaker. The screen still shows every reply either way.
Buddy talks to Claude through an API key, a long secret password tied to your account and your balance. You only need one of the two options below. Both give you a key that starts with sk-.
OneProvider is a separate company that sells prepaid access to Claude. The project files already point at it, so this option needs no extra setup.
buddy and, if you can, set a spend limit such as $10. If the key ever leaks, nobody can spend more than that.relay/start_relay.command (Mac), remove the # in front of the line that mentions ONEPROVIDER_BASE_URL. On Windows you will type the same setting in step 5.This build was only run through OneProvider. The relay uses Anthropic's own software library, so the official API should work, but nobody has confirmed it on this exact setup yet. The setting names say ONEPROVIDER because that is what the build used. They work for either provider.
Never post it, put it in a screenshot or share it in a chat. If it ever leaks, delete it in your provider's dashboard and issue a new one. The key lives only on your computer. The ESP32 never sees it.
Open a terminal in the relay folder, the same way you did when you installed the packages. Then set your key and run the test. Put your real key between the quotes.
export ONEPROVIDER_KEY="sk-paste-your-key-here"
python3 test_key.py$env:ONEPROVIDER_KEY="sk-paste-your-key-here"
python test_key.pyIf you chose Option B, run the line export ONEPROVIDER_BASE_URL=https://api.anthropic.com first (on Windows: $env:ONEPROVIDER_BASE_URL="https://api.anthropic.com").
You see Claude says: ... followed by a short greeting, and then KEY WORKS. With OneProvider you also see the balance left on the key.
A 401 means the key was copied wrongly, so copy it again without spaces. A 429 means the balance or the key's spend limit is used up. The key setting only lasts as long as that window stays open.
The relay is the program on your computer that turns your voice into text, asks Claude, and turns the answer into speech. It has to be running whenever you talk to Buddy.
relay folder: type cd and a space, drag the folder into the window, press Enter.bash. This avoids the "Apple could not verify..." warning that macOS shows for any script that came out of a zip file.bash start_relay.commandPrefer double-clicking? Click Done on the warning, open System Settings, Privacy & Security, find the message about start_relay.command and click Open Anyway. After that, double-click works. The file is plain text, so you can open it in TextEdit first and see that it only starts the relay.
relay folder, the way you did in step 1.$env:ONEPROVIDER_KEY="sk-paste-your-key-here"
python -m uvicorn relay:app --host 0.0.0.0 --port 8000The window shows Application startup complete and Uvicorn running on http://0.0.0.0:8000. Leave the window open. Closing it stops Buddy.
The launcher uses caffeinate so your Mac does not fall asleep while the relay runs. Keep the lid open anyway, because closing it stops Buddy.
Windows will sleep on its own and stop Buddy. While you use Buddy, set Settings, System, Power, Screen and sleep to Never.
The ESP32 needs to know where on your home network to find the relay. Open a second terminal window and run:
ipconfig getifaddr en0It prints something like 192.168.1.23. Write it down. If it prints nothing, try en1 instead of en0, or read it from System Settings, Wi-Fi, Details.
ipconfigUnder Wireless LAN adapter Wi-Fi, find the line IPv4 Address. It looks like 192.168.1.23. Write it down.
Connect your phone to the same Wi-Fi and open http://192.168.1.23:8000/ in its browser, using your own address.
The page shows something like {"ok":true,"model":"claude-haiku-5-5", ...}.
If it does not load, the usual causes are a wrong address, a phone on a different network (guest Wi-Fi often blocks this), or a privacy setting. On newer macOS, open System Settings, Privacy & Security, Local Network and make sure Terminal is switched on. If your computer is on a phone hotspot, its address often starts with 172.20.10. and the hotspot must be set to the 2.4 GHz band (on an iPhone, switch on Maximize Compatibility).
This is the real Buddy program. Open ai_buddy/ai_buddy.ino in the Arduino IDE and find these lines near the top:
// ---------------- Settings ----------------
const char* WIFI_SSID = "YOUR_WIFI_NAME";
const char* WIFI_PASS = "YOUR_WIFI_PASSWORD";
const char* RELAY_HOST = "192.168.1.50"; // LAN IP of the PC running relay.py
const uint16_t RELAY_PORT = 8000;Replace the three values with your own and keep the quote marks:
WIFI_SSID is your Wi-Fi name. It must be a 2.4 GHz network.WIFI_PASS is your Wi-Fi password. Capital letters count.RELAY_HOST is the computer address you wrote down in step 5.Check that Tools, Board is still ESP32 Dev Module and the port is right, then click Upload. Hold the BOOT button if it sticks on Connecting..... Afterwards open the Serial Monitor at 115200.
The Serial Monitor says WiFi OK, IP ... and then noise floor .... The screen shows "Listening to the room... stay quiet" for about a second, then two eyes.
Stay quiet during that first second. Buddy uses it to learn how noisy your room is, so it knows what counts as someone speaking.
Just talk, normally, about 30 to 50 centimetres from the microphone. There is no button to hold. Try:
This is what happens on the screen, in this order:
Expect a few seconds between the end of your sentence and the start of the reply. It depends on your computer. Look at the relay window too: it prints heard: (what Whisper understood) and reply: (what Claude said). That window is the best place to see what is going on.
You asked something, saw the reply on the screen and heard it. That is the whole project working.
If you restart your router, your computer's address may change. If Buddy then says "Can't reach the relay", find the address again and update RELAY_HOST in the program.
You do not need to read any code to build Buddy. If you want to know why it works, these are the pieces that matter. Every snippet below is copied straight from the project files you downloaded.
// ---------------- Pins ----------------
const int PIN_BTN = 0; // BOOT button: tap = mute / unmute
const int PIN_SDA = 32, PIN_SCL = 33; // OLED
const int MIC_SCK = 26, MIC_WS = 25, MIC_SD = 35; // INMP441
const int SPK_BCLK = 14, SPK_LRC = 13, SPK_DIN = 27; // MAX98357AEach name is a job and each number is a pin on the ESP32. They match the wiring tables in step 2: SCK, WS and SD for the microphone, BCLK, LRC and DIN for the amplifier. If you wired something differently, change the numbers here to match what you built, then upload again. GPIO 35 can only be an input, which is exactly what the microphone's data wire needs.
Buddy reads the microphone about 30 times a second, in short chunks. For each chunk it asks one question: is this louder than the room normally is? These settings decide the answer.
// ---------------- Voice detection (tune these if it triggers too much or too little) ----------------
const int CHUNK_FRAMES = 512; // 32 ms of audio per chunk at 16 kHz
const int PREROLL_CHUNKS = 8; // keep 256 ms from before speech is detected so the first word isn't clipped
const int START_CHUNKS = 3; // loud chunks in a row needed to start (about 100 ms)
const uint32_t END_SILENCE_MS = 900; // this much quiet ends your sentence
const uint32_t MAX_RECORD_MS = 15000; // safety limit for one sentence
const float VAD_MIN_LEVEL = 60; // never trigger below this level
const float VAD_RATIO = 3.0; // trigger when level > background noise x this
const bool DEBUG_LEVELS = false; // true = print levels to the Serial Monitor to help tuningAnd this is the loop that applies them:
// Always listening: each chunk goes into the ring buffer and is checked for speech
float level = readChunk(preroll[prerollHead]);
if (level < 0) return;
prerollHead = (prerollHead + 1) % PREROLL_CHUNKS;
if (prerollCount < PREROLL_CHUNKS) prerollCount++;
float threshold = fmaxf(VAD_MIN_LEVEL, noiseFloor * VAD_RATIO);
if (level > threshold) {
loudRun++;
} else {
loudRun = 0;
noiseFloor = noiseFloor * 0.97f + level * 0.03f; // slowly follow the room's background noise
}
if (DEBUG_LEVELS && millis() - lastDebug > 500) {
lastDebug = millis();
Serial.printf("level %.0f noise %.0f trigger %.0f\n", level, noiseFloor, threshold);
}
if (loudRun >= START_CHUNKS) {
loudRun = 0;
converse(threshold);
// ...0.97f and 0.03f.Once speech starts, the ESP32 opens a connection to the relay and sends the saved quarter second, then streams live audio until you have been quiet for 900 milliseconds. It cannot know in advance how long you will talk, so it uses chunked transfer, a standard feature of HTTP: each piece announces its own length, and a final empty piece, the 0, means "that is all".
// 1) the audio from just before speech was detected, oldest first
int first = (prerollHead - prerollCount + PREROLL_CHUNKS) % PREROLL_CHUNKS;
for (int k = 0; k < prerollCount; k++) sendChunk(c, preroll[(first + k) % PREROLL_CHUNKS]);
prerollCount = 0;
// 2) keep streaming until you have been quiet for END_SILENCE_MS
uint32_t startMs = millis(), lastVoice = millis();
float endThreshold = threshold * 0.7f; // a bit lower than the start level so trailing words don't cut off
float level = 0;
while (c.connected() && millis() - startMs < MAX_RECORD_MS) {
level = readChunk(chunkBuf);
if (level < 0) continue;
sendChunk(c, chunkBuf);
if (level > endThreshold) lastVoice = millis();
else if (millis() - lastVoice > END_SILENCE_MS) break;
}
if (millis() - startMs >= MAX_RECORD_MS) noiseFloor = level; // constant loud noise: raise the bar
c.print("0\r\n\r\n");The relay is a small Python web server (FastAPI) with one job: take a sentence of sound and return Buddy's answer. It does that in three moves.
def transcribe(pcm: bytes) -> str:
pcm = pcm[: len(pcm) // 2 * 2]
audio = np.frombuffer(pcm, dtype=np.int16).astype(np.float32) / 32768.0
peak = float(np.abs(audio).max()) if audio.size else 0.0
if peak > 0:
audio = audio * min(0.9 / peak, 8.0) # gentle normalisation for the quiet INMP441
segments, _ = get_whisper().transcribe(audio, vad_filter=True, beam_size=1, language=WHISPER_LANG)
return " ".join(s.text.strip() for s in segments).strip()The audio is scaled up a little first, because the INMP441 is quiet. vad_filter=True makes Whisper skip silence, and beam_size=1 trades a little accuracy for speed.
messages = list(history) + [{"role": "user", "content": heard}]
while messages and messages[0]["role"] != "user":
messages.pop(0)
try:
resp = await client.messages.create(model=MODEL, max_tokens=200, system=SYSTEM, messages=messages)
reply = "".join(b.text for b in resp.content if b.type == "text").strip() or "Hmm, I have nothing to say."history remembers the last six exchanges, so Buddy can follow a conversation. The short loop makes sure the list starts with something you said, which is a rule of the API. max_tokens=200 keeps answers short.
async def tts_mp3(text: str) -> bytes:
mp3 = bytearray()
async for chunk in edge_tts.Communicate(text, VOICE).stream():
if chunk["type"] == "audio":
mp3.extend(chunk["data"])
return bytes(mp3)Edge TTS streams back the spoken reply as MP3 audio. The relay then either plays it on your Mac or converts it to the format the ESP32 speaker expects.
SYSTEM = (
"You are Buddy, a friendly little desk companion living inside an ESP32 with a tiny screen and speaker. "
"Your replies are spoken aloud, so answer in one or two short plain sentences. "
"No markdown, no lists, no emojis."The reply is spoken aloud, so the instructions ask for short, plain sentences. Without "no markdown, no lists, no emojis", the voice would read out every asterisk.
void drawFace(bool blink) {
display.clearDisplay();
if (blink) {
display.fillRect(30, 34, 24, 3, SSD1306_WHITE);
display.fillRect(74, 34, 24, 3, SSD1306_WHITE);
} else {
display.fillRoundRect(30, 14, 24, 30, 6, SSD1306_WHITE);
display.fillRoundRect(74, 14, 24, 30, 6, SSD1306_WHITE);
}
display.setTextColor(SSD1306_WHITE);
display.setTextSize(1);
display.setCursor(34, 55);
display.print("just talk :)");
display.display();
}The demo at the top of this page draws Buddy's face with exactly these numbers. Change them, upload, and Buddy gets new eyes.
Most changes are one line. Edit the file, save it, and then either click Upload again (for ai_buddy.ino) or restart the relay (for everything in the relay folder).
| You want to | Change this | In |
|---|---|---|
| Give Buddy a new personality | The text under SYSTEM = ( | relay.py |
| Have it answer only when it hears its name | Remove the # before export WAKE_WORD=buddy. After a reply you can keep chatting for 20 seconds without the name. On Windows set $env:WAKE_WORD="buddy". | start_relay.command |
| Get faster replies | WHISPER_MODEL=base, a smaller and slightly less accurate listener | start_relay.command |
| Use a different voice | TTS_VOICE. The default is en-IN-NeerjaNeural. Run edge-tts --list-voices to see the rest. | start_relay.command |
| Talk in Hindi | TTS_VOICE=hi-IN-SwaraNeural, WHISPER_LANG=hi, and ask for Hindi replies in SYSTEM. The screen font only shows English letters, so Hindi is spoken but not shown. | start_relay.command, relay.py |
| Use another Claude model | CLAUDE_MODEL. The default is claude-haiku-5-5. | start_relay.command |
| Choose where the voice comes out | SPEAKER=mac for your Mac's speakers, or SPEAKER=esp32 for the speaker wired to the board | start_relay.command |
| Change the volume | MAC_VOLUME for Mac speakers. VOLUME for the ESP32 speaker, and keep that at 0.5 or below. | start_relay.command |
The launcher lists the optional settings, switched off, so you only have to remove a #:
# Optional settings: remove the # at the start of a line to turn it on.
# export WAKE_WORD=buddy
# export WHISPER_MODEL=base
# export VOLUME=0.35
# Using Anthropic's own API instead of OneProvider? Remove the # on the next line.
# export ONEPROVIDER_BASE_URL=https://api.anthropic.comIf Buddy ignores you or jumps at every noise, set DEBUG_LEVELS to true in ai_buddy.ino and upload. The Serial Monitor then prints level, noise and trigger twice a second, so you can see what the microphone sees.
| What happens | Try this |
|---|---|
| It never reacts to you | Lower VAD_MIN_LEVEL to 30, or lower MIC_SHIFT to 13 or 12. |
| It reacts to background noise | Raise VAD_MIN_LEVEL or VAD_RATIO. |
| It cuts you off mid-sentence | Raise END_SILENCE_MS to 1200. |
| It waits too long after you finish | Lower END_SILENCE_MS to 700. |
| The microphone is silent | Set MIC_SLOT = 1, and check that the microphone's L/R pin is on GND. |
Nearly every problem in this build comes down to a loose wire, a wrong pin number or a window that was closed. Find your symptom below. If none of them fits, the last section of this page shows what to send when you ask for help.
Your ESP32 board package is too old. Open the Boards Manager and install version 3.x of "esp32 by Espressif Systems".
You are using a Python installed by Homebrew. Install Python from python.org (step 1), open a new Terminal window and try again.
Install Python from python.org (step 1), then quit and reopen Terminal.
Run the Python installer again and tick Add python.exe to PATH. Then open a new PowerShell window.
Use a cable that carries data (many cheap ones only charge) and try another USB port. Check the chip near the USB socket: install the CP210x driver for a CP2102 or the CH340 driver for a CH340.
Press and hold the BOOT button on the ESP32 until the upload starts, then let go.
Check SDA on pin 32 and SCL on pin 33, then VCC and GND. Upload i2c_scan/i2c_scan.ino: your screen should appear as 0x3C. If you only see odd addresses such as 0x0D or 0x52, a wire is loose. Push every wire in firmly and scan again.
First check that BCLK, LRC and DIN are all firmly connected to pins 14, 13 and 27, and that the amplifier's VIN is on the ESP32 VIN pin. A floating or loose pin makes this noise. If it keeps happening with correct wiring, the amplifier module itself may be faulty. See "What went wrong" below, then switch to your computer's speakers.
Amplifier VIN to the ESP32 VIN pin, GND to the rail, then BCLK 14, LRC 13, DIN 27. Check that the speaker wires are tight in the terminals.
Microphone VDD on 3V3, L/R on GND, SCK 26, WS 25, SD 35. If all that is right, change MIC_SLOT = 0 to MIC_SLOT = 1 and upload again.
Open Terminal, go to the relay folder (type cd and a space, drag the folder in, press Enter) and run bash start_relay.command. See step 5 for the other way.
The key was not entered. Start the relay again and paste your key when it asks.
The key was copied wrongly. Copy it again with no spaces.
The balance, or that key's spend limit, is used up. Top up or raise the limit in your provider's dashboard.
Check the address, that the phone is on the same Wi-Fi (guest networks often block this) and that the relay window is still open. On macOS, make sure Terminal is allowed under System Settings, Privacy & Security, Local Network. On Windows, allow Python through the firewall on private networks.
Check the Wi-Fi name and password, where capital letters count. The network must be 2.4 GHz.
The relay window is closed, RELAY_HOST is wrong, the computer is asleep or the computer and the ESP32 are on different networks. Run the phone test from step 5.
Your computer probably went to sleep or the lid was closed. Wake it. The relay window should still be there.
Set DEBUG_LEVELS = true, upload, and watch the Serial Monitor while you talk. If level never goes above trigger, lower VAD_MIN_LEVEL to 30 or MIC_SHIFT to 13.
Raise VAD_MIN_LEVEL (try 120), or use the wake word.
Raise END_SILENCE_MS to 1200.
Speak a little louder or closer, or lower MIC_SHIFT to 13.
Its voice is reaching the microphone. Lower the volume, move the microphone away from the speaker, or use the wake word.
Check that the Mac's volume is up and not muted, and that start_relay.command contains export SPEAKER=mac. The relay window prints reply: ... when it works. If it says afplay was not found, you are not on a Mac.
Use the smaller speech model: remove the # before export WHISPER_MODEL=base in start_relay.command.
The first version of anything rarely works. These are the real problems that came up while building this Buddy, and what fixed them. If you hit one, you are in good company.
The OLED worked, then showed nothing. The I2C scan reported strange addresses, 0x0D and 0x52, instead of the screen's 0x3C. Nothing was wrong with the code: a jumper wire had worked loose in the breadboard.
Lesson: when a module disappears, re-seat every wire on it before you touch the code. The scan sketch in the project exists for exactly this.
With the speaker wired up, the amplifier played a steady tone no matter what the code sent it. Connecting BCLK and LRC alone made noise, and all three signal wires made a continuous tone. Several test programs, including one that sent only silence and one that used an older audio driver, gave the same result. The module was most likely faulty.
Lesson: test the speaker early, with the hardware check, and keep a plan B. Here that was playing Buddy's voice through the Mac's speakers, which needs no amplifier at all.
The first wiring diagram used pins 32 and 33 for the screen, 26 and 25 for the microphone clock and 14 and 13 for the amplifier. The first version of the code expected different pins, so nothing worked until one side was changed to match the other.
Lesson: the pin numbers sit in three lines at the top of the program. Write down what you actually wired, and make those lines say the same thing.
Double-clicking start_relay.command showed "Apple could not verify..." because the file came out of a zip. It is a plain text script, so running it with bash start_relay.command from Terminal avoids the warning completely.
On an iPhone hotspot, the Mac's address was 172.20.10.2 instead of the usual 192.168.x.x. That is normal. The ESP32 needs the 2.4 GHz band, so the hotspot's Maximize Compatibility switch had to be on.
VOLUME at 0.5 or below.Everything you need is in one zip: the test programs, the Buddy program for the ESP32 and the relay for your computer.
ai_buddy_project/
README.md short technical reference
hardware_check/ test your screen, speaker and microphone first
i2c_scan/ finds your screen if it does not show up
speaker_test/ a plain speaker test
ai_buddy/ the Buddy program for the ESP32
relay/ the program that runs on your computer
relay.py speech to text, Claude, text to speech
requirements.txt the Python packages it needs
test_key.py checks that your Claude key works
start_relay.command Mac launcher (asks for your key)Stuck? Send three things when you ask for help: the text on the Buddy screen, the text in the Serial Monitor and the text in the relay window. With those, most problems can be found in minutes. Ask for help.
Built with the ESP32 Arduino core, the Adafruit SSD1306 and GFX libraries, faster-whisper, FastAPI, edge-tts and Claude.
This guide is independent. Claude is made by Anthropic, and OneProvider is a separate company. Neither has reviewed or endorsed this project.
© 2025 Twin Reality Technologies
All rights reserved.