Build an AI Desk Buddy with ESP32 Dev Module and Claude: Step-by-Step

Build your own AI Buddy

Build a little desk buddy you can talk to

Wire an ESP32, a microphone and a tiny screen to Claude, then just talk. This guide takes you from a bag of parts to a working Buddy, one checked step at a time.

Time
About 2 hours
Experience
None needed
Made on
A Mac, with Windows notes
Buddy's screen shows two eyes.

This is Buddy's real screen layout with sample replies. Pick a sentence to watch one turn from start to finish.

heard: What is two plus two?reply: Two plus two is four.
Contents

How Buddy works

Claude understands text, not sound, and an ESP32 is far too small to understand speech by itself. So the work is shared between your desk, your computer and the cloud.

The six stops a sentence makes: microphone, ESP32, Whisper, Claude, Edge TTS, then the reply on the screen and out loud.1Microphonehears you2ESP32spots a sentence3Whispersound to text4Claudewrites a reply5Edge TTStext to voice6Replyon screen and aloudOn your deskOn your computerIn the cloud
One spoken sentence makes this trip every time you talk to Buddy.
  1. DeskYou speak. The INMP441 microphone turns your voice into a stream of numbers.
  2. DeskThe ESP32 listens all the time. When it notices a sentence start and then stop, it sends just that sentence over Wi-Fi to your computer.
  3. ComputerWhisper writes it down. Whisper is a free speech-recognition model that runs on your own computer, so your voice is not uploaded for this step.
  4. CloudClaude replies. Your computer sends the written sentence to Claude and gets a short answer back.
  5. ComputerEdge TTS gives the answer a voice. A free text-to-speech service turns the reply into spoken audio.
  6. DeskBuddy shows it and says it. The reply appears on the OLED screen. The voice comes out of your computer's speakers, or out of a speaker wired to the ESP32.

The program on your computer that does the middle steps is called the relay. Your Claude key lives inside the relay and never goes onto the ESP32. That also means your computer has to be on, awake and running the relay whenever you want to talk to Buddy.

What you need

Parts for the build. Search for these exact names at any electronics shop. This build used parts from Robu.in.
PartWhat it doesGood to know
ESP32 DevKit, 38 pinsThe brain. It has Wi-Fi built in.A full-size breadboard fits it best, because this board is wide.
0.96 inch OLED screen (SSD1306, I2C, 4 pins)Buddy's face and its written replies.Pin order differs between brands, so read the labels printed on yours.
INMP441 microphone moduleHears you.Power it from 3V3 only. 5V can destroy it.
MAX98357A amplifier and an 8 Ω 0.5 W speakerPlays Buddy's voice from the ESP32.Optional on a Mac, where the relay can use your computer's speakers instead.
Full-size breadboard and jumper wiresJoins everything without soldering.About 20 male-to-male wires, plus a few spares.
USB data cablePowers the ESP32 and loads programs onto it.Some cheap cables only charge. Check whether your ESP32 has a micro-USB or a USB-C socket.

If your OLED, microphone or amplifier came with a strip of pins that is not soldered on, solder it on. Loose pins pressed into the holes seem fine for a minute and then fail in confusing ways.

On the computer side

  • A Mac (Apple Silicon or Intel) or a Windows 10 or 11 PC.
  • Wi-Fi on the 2.4 GHz band. The ESP32 cannot use 5 GHz, and your computer must be on the same network.
  • About 2 GB of free disk space for the speech model and Python packages.
  • A Claude API key. Step 4 shows two ways to get one.

Two safety rules

Unplug the USB cable while you change wires

Moving wires on a powered board is the quickest way to damage a part.

The microphone only ever gets 3V3

Connect the INMP441 to the 3V3 pin, never to 5V or VIN. 5V can destroy it.

Step 1: Set up your computer

You will install four things: Python (it runs the relay), the project files, the Arduino IDE (it loads programs onto the ESP32) and the ESP32 add-on for the Arduino IDE. It takes about 20 minutes, most of it waiting for downloads.

Pick your computer and the steps below will match it.
The Windows steps are untested

They follow the same flow as the Mac steps but were not run on a real PC. If one is wrong for you, the troubleshooting section near the end lists the usual fixes.

Install Python

Mac
  1. Go to python.org/downloads and download the Python 3.12 installer for macOS (a .pkg file). Versions 3.10 to 3.12 are the safest.
  2. Open the file and click through the installer.
  3. When it finishes, a Finder window may open. If you see Install Certificates.command, double-click it and wait until it says it is done.
  4. Open Terminal: press Command + Space, type Terminal and press Enter. Then run:
Terminal
python3 --version
Windows
  1. Go to python.org/downloads and download the Python 3.12 Windows installer (64-bit).
  2. Run it. On the first screen, tick Add python.exe to PATH before you click Install Now.
  3. Open PowerShell: press the Windows key, type PowerShell and press Enter. Then run:
PowerShell
python --version
Checkpoint

It prints something like Python 3.12.x.

If a window offers to install developer tools instead, close it, quit Terminal, open it again and retry.

If it says python is not recognized, run the installer again and tick the PATH box, then open a new PowerShell window.

Unzip the project

Mac

Double-click the zip in your Downloads folder. A folder called ai_buddy_project appears. Drag it into your Documents folder.

Windows

Right-click the zip in your Downloads folder, choose Extract All, and extract it into your Documents folder. You should end up with a folder called ai_buddy_project.

Install the relay's packages

Mac
  1. In Terminal, type cd followed by one space. Do not press Enter yet.
  2. In Finder, open ai_buddy_project and drag the relay folder into the Terminal window. Its path appears after cd . Now press Enter.
  3. Type ls and press Enter. If you see relay.py in the list, you are in the right place.
  4. Run this. It downloads a few hundred megabytes and takes 5 to 10 minutes:
Terminal
python3 -m pip install -r requirements.txt
Windows
  1. In File Explorer, open ai_buddy_project, then the relay folder.
  2. Click the address bar at the top, type powershell and press Enter. A PowerShell window opens already inside that folder.
  3. Run this. It downloads a few hundred megabytes and takes 5 to 10 minutes:
PowerShell
python -m pip install -r requirements.txt
Checkpoint

The last lines say Successfully installed ... and there is no red error text.

If the later step fails with an error about a missing DLL, install the Microsoft Visual C++ Redistributable and try again.

Install the Arduino IDE

Mac

Go to arduino.cc/en/software and download Arduino IDE 2.x for macOS. Choose Apple Silicon for an M1, M2, M3 or M4 Mac and Intel otherwise (Apple menu, About This Mac, tells you). Open the download and drag Arduino IDE into Applications. If your Mac refuses to open it, right-click the app and choose Open.

Windows

Go to arduino.cc/en/software and download Arduino IDE 2.x for Windows 10 and newer. Run the installer and accept the defaults.

Add ESP32 support to the Arduino IDE

  1. Mac Open the settings with Command + , (comma).
  2. Windows Open File, Preferences.
  3. Find the box called Additional boards manager URLs and paste this line into it:
Boards manager URL
https://espressif.github.io/arduino-esp32/package_esp32_index.json
  1. Click OK. Open the Boards Manager (the icon in the left bar) and search for esp32.
  2. Find esp32 by Espressif Systems and click Install. Choose the latest 3.x version. It is a big download, so wait for it to finish.

Install the screen libraries

Open the Library Manager (also in the left bar), search for Adafruit SSD1306 and click Install. When a window asks about dependencies, click Install all.

Check the USB connection

Plug the ESP32 into your computer. In the Arduino IDE open Tools, Port.

Mac

A new entry appears, such as /dev/cu.usbserial-0001 or /dev/cu.SLAB_USBtoUART. Always pick the one that starts with cu.

Windows

A new entry appears, such as COM3. The number can be different on your PC.

If no new port appears, try a different cable first (it must carry data), then a different USB port. Next look at the small chip near the USB socket. If it says CP2102, install the CP210x driver from Silicon Labs. If it says CH340, install the CH340 driver. On a Mac you may need to allow the driver in System Settings, Privacy & Security, and restart.

Checkpoint

Tools, Port shows a new port whenever the ESP32 is plugged in.

Step 2: Wire it up

Unplug the USB cable first

Change wires only while the ESP32 has no power.

Every pin on the ESP32 has a label printed beside it. Some boards print D32, some 32, IO32 or GPIO32. They all mean the same pin. 3V3 is the 3.3 volt pin, VIN is the 5 volt pin, and there are several GND pins that all work the same.

A colour code that makes mistakes easy to spot

Use whichever wires you have. If you can, follow these colours, because then a red wire should only ever touch a power pin, and anything else that is red is a mistake you can see.

  • 3.3 V power
  • 5 V power
  • Ground (GND)
  • Clock
  • Data
  • Left/right select

The microphone and the amplifier talk to the ESP32 with a standard called I2S. It always uses the same three jobs: a clock wire that keeps time, a select wire that says left or right, and a data wire. The screen uses a simpler standard called I2C, with its own clock (SCL) and data (SDA) wires.

Wiring mapThe OLED screen, microphone and amplifier on the left, the ESP32 on the right. Power and ground go through shared rails; each signal wire runs straight to its GPIO pin. The tables below list the same connections.OLED screenGNDVCCSDASCLINMP441 microphoneVDDGNDL/RSCKWSSDMAX98357A amplifierVINGNDBCLKLRCDINESP32 DevKit3V3GNDVIN (5 V)GPIO 32GPIO 33GPIO 26GPIO 25GPIO 35GPIO 14GPIO 13GPIO 273V3GND5V
Every connection in one picture. The lists below say the same thing in words, one part at a time.

Make the power rails

Plug the ESP32 into the breadboard so its two rows of pins sit on either side of the middle gap. Then:

  • Connect the ESP32 3V3 pin to the breadboard's red (+) rail.
  • Connect an ESP32 GND pin to the breadboard's blue (−) rail.

Use the printed stripes on your breadboard as the guide, even if your ground wire is black.

Connect each part

The order of the pins differs between brands, so read the labels printed on your own modules before you plug anything in.

  • OLED screen0.96 inch, 4 pins

    • GNDblue (−) rail
    • VCCred (+) rail3V3
    • SDAGPIO 32
    • SCLGPIO 33
  • INMP441 microphone3V3 only, never 5V

    • VDDred (+) rail3V3 only
    • GNDblue (−) rail
    • L/Rblue (−) railthis picks the left slot
    • SCKGPIO 26
    • WSGPIO 25
    • SDGPIO 35data out
  • MAX98357A amplifieroptional on a Mac

    • VINESP32 VIN pin5 V, not the 3V3 rail
    • GNDblue (−) rail
    • BCLKGPIO 14
    • LRCGPIO 13
    • DINGPIO 27
    • GAINleave empty
    • SDleave emptythis one means shutdown
Two pins are both called SD

On the microphone, SD means "serial data out". On the amplifier, SD means "shutdown". They are different things, so do not mix them up. Leave the amplifier's SD and GAIN pins empty.

Connect the two speaker wires to the amplifier's output terminals, the two screw terminals or solder pads marked + and −. With a single speaker it does not matter which wire goes where. If you plan to use your computer's speakers instead, you can skip the amplifier and speaker for now.

Check everything slowly

Now plug the USB cable back in.

Step 3: Test the hardware

Before Wi-Fi, your computer and Claude come into it, check that each part works on its own. A small test program in the project does this, and it will save you hours of guessing later.

  1. In the Arduino IDE choose File, Open and open hardware_check/hardware_check.ino from the project folder.
  2. Choose Tools, Board, esp32, ESP32 Dev Module.
  3. Choose your ESP32's port under Tools, Port.
  4. Click the Upload button, the right-pointing arrow at the top left. The first time, it takes a few minutes to compile.
  5. If the bottom window shows Connecting........ for more than 10 seconds, press and hold the BOOT button on the ESP32 until the upload starts, then let go.
  6. When it says Done uploading, open Tools, Serial Monitor and set the speed at the bottom right to 115200.
Checkpoint: you should see and hear all three
  • Screen: it says "Hardware check", then a "Mic level" bar.
  • Speaker: a short beep at the start, then one every 5 seconds.
  • Microphone: when you clap or talk, the bar jumps and the numbers in the Serial Monitor go up.

Do not move on until the screen and the microphone work. The speaker is optional, as you will see below.

If one of them is missing

What you seeWhat to check
The screen stays blankSDA on pin 32 and SCL on pin 33 (try swapping them), plus VCC and GND. If it is still blank, upload i2c_scan/i2c_scan.ino. It lists every I2C device it finds, and your screen should appear as 0x3C. Odd addresses such as 0x0D or 0x52 mean a wire is loose.
No beepAmplifier VIN to the ESP32 VIN pin, GND, then BCLK 14, LRC 13 and DIN 27. Check that the speaker wires are tight in the terminals.
The mic bar never movesMicrophone VDD on 3V3, L/R on GND, then SCK 26, WS 25 and SD 35. If all of that is right, open the file, change MIC_SLOT = 0 to MIC_SLOT = 1 and upload again.
Upload fails with "port not found"Use a data USB cable, and see the USB check in step 1.

No amplifier, or it only makes noise?

You do not need the amplifier at all. On a Mac, the relay can play Buddy's voice through your computer's own speakers, while the microphone and screen stay on the ESP32. The project's Mac launcher already does this.

  • Disconnect the amplifier's VIN wire so the module stays off and silent. Everything else can stay in place.
  • While Buddy speaks, the screen shows its reply and Buddy ignores sound, so it does not hear itself.
  • Keep the microphone away from your computer's speakers and keep the volume moderate.
This shortcut is Mac only

Playing through the computer's speakers uses a Mac command, afplay. On Windows, Buddy's voice has to come from the amplifier and speaker. The screen still shows every reply either way.

Step 4: Get a Claude key

Buddy talks to Claude through an API key, a long secret password tied to your account and your balance. You only need one of the two options below. Both give you a key that starts with sk-.

Option A: OneProvider (what this build used)

OneProvider is a separate company that sells prepaid access to Claude. The project files already point at it, so this option needs no extra setup.

  1. Open app.oneprovider.dev and sign in.
  2. Check that your balance is there. If it shows zero, look for the option to redeem a code. Button names may differ slightly from what is written here.
  3. Choose Issue a key, name it buddy and, if you can, set a spend limit such as $10. If the key ever leaks, nobody can spend more than that.
  4. Copy the key. It is shown in full only once, so paste it into a notes window for now.

Option B: Anthropic's own API

  1. Create an account in the Anthropic Console, add a few dollars of credit and create an API key.
  2. Tell the relay to use Anthropic's address instead of OneProvider's. In relay/start_relay.command (Mac), remove the # in front of the line that mentions ONEPROVIDER_BASE_URL. On Windows you will type the same setting in step 5.
Option B is untested

This build was only run through OneProvider. The relay uses Anthropic's own software library, so the official API should work, but nobody has confirmed it on this exact setup yet. The setting names say ONEPROVIDER because that is what the build used. They work for either provider.

Keep the key private

Never post it, put it in a screenshot or share it in a chat. If it ever leaks, delete it in your provider's dashboard and issue a new one. The key lives only on your computer. The ESP32 never sees it.

Test the key

Open a terminal in the relay folder, the same way you did when you installed the packages. Then set your key and run the test. Put your real key between the quotes.

Mac
Terminal
export ONEPROVIDER_KEY="sk-paste-your-key-here"
python3 test_key.py
Windows
PowerShell
$env:ONEPROVIDER_KEY="sk-paste-your-key-here"
python test_key.py

If you chose Option B, run the line export ONEPROVIDER_BASE_URL=https://api.anthropic.com first (on Windows: $env:ONEPROVIDER_BASE_URL="https://api.anthropic.com").

Checkpoint

You see Claude says: ... followed by a short greeting, and then KEY WORKS. With OneProvider you also see the balance left on the key.

A 401 means the key was copied wrongly, so copy it again without spaces. A 429 means the balance or the key's spend limit is used up. The key setting only lasts as long as that window stays open.

Step 5: Start the relay

The relay is the program on your computer that turns your voice into text, asks Claude, and turns the answer into speech. It has to be running whenever you talk to Buddy.

Mac
  1. Open Terminal and go to the relay folder: type cd and a space, drag the folder into the window, press Enter.
  2. Run the launcher with bash. This avoids the "Apple could not verify..." warning that macOS shows for any script that came out of a zip file.
Terminal
bash start_relay.command
  1. When it asks, paste your key and press Enter.
  2. The first time, it downloads the speech-recognition model, about half a gigabyte. Wait for it.
  3. If your Mac asks "Do you want the application Python to accept incoming network connections?", click Allow.

Prefer double-clicking? Click Done on the warning, open System Settings, Privacy & Security, find the message about start_relay.command and click Open Anyway. After that, double-click works. The file is plain text, so you can open it in TextEdit first and see that it only starts the relay.

Windows
  1. Open a PowerShell window inside the relay folder, the way you did in step 1.
  2. Set your key and start the relay. If you chose Option B, add the base URL line from step 4 above the last line.
PowerShell
$env:ONEPROVIDER_KEY="sk-paste-your-key-here"
python -m uvicorn relay:app --host 0.0.0.0 --port 8000
  1. The first time, it downloads the speech-recognition model, about half a gigabyte. Wait for it.
  2. If Windows asks whether to let Python through the firewall, allow it on private networks.
Checkpoint

The window shows Application startup complete and Uvicorn running on http://0.0.0.0:8000. Leave the window open. Closing it stops Buddy.

The launcher uses caffeinate so your Mac does not fall asleep while the relay runs. Keep the lid open anyway, because closing it stops Buddy.

Windows will sleep on its own and stop Buddy. While you use Buddy, set Settings, System, Power, Screen and sleep to Never.

Find your computer's address

The ESP32 needs to know where on your home network to find the relay. Open a second terminal window and run:

Mac
Terminal
ipconfig getifaddr en0

It prints something like 192.168.1.23. Write it down. If it prints nothing, try en1 instead of en0, or read it from System Settings, Wi-Fi, Details.

Windows
PowerShell
ipconfig

Under Wireless LAN adapter Wi-Fi, find the line IPv4 Address. It looks like 192.168.1.23. Write it down.

Test it from your phone

Connect your phone to the same Wi-Fi and open http://192.168.1.23:8000/ in its browser, using your own address.

Checkpoint

The page shows something like {"ok":true,"model":"claude-haiku-5-5", ...}.

If it does not load, the usual causes are a wrong address, a phone on a different network (guest Wi-Fi often blocks this), or a privacy setting. On newer macOS, open System Settings, Privacy & Security, Local Network and make sure Terminal is switched on. If your computer is on a phone hotspot, its address often starts with 172.20.10. and the hotspot must be set to the 2.4 GHz band (on an iPhone, switch on Maximize Compatibility).

Step 6: Put Buddy on the ESP32

This is the real Buddy program. Open ai_buddy/ai_buddy.ino in the Arduino IDE and find these lines near the top:

ai_buddy.ino (near the top)
// ---------------- Settings ----------------
const char* WIFI_SSID = "YOUR_WIFI_NAME";
const char* WIFI_PASS = "YOUR_WIFI_PASSWORD";
const char* RELAY_HOST = "192.168.1.50";  // LAN IP of the PC running relay.py
const uint16_t RELAY_PORT = 8000;

Replace the three values with your own and keep the quote marks:

  • WIFI_SSID is your Wi-Fi name. It must be a 2.4 GHz network.
  • WIFI_PASS is your Wi-Fi password. Capital letters count.
  • RELAY_HOST is the computer address you wrote down in step 5.

Check that Tools, Board is still ESP32 Dev Module and the port is right, then click Upload. Hold the BOOT button if it sticks on Connecting..... Afterwards open the Serial Monitor at 115200.

Checkpoint

The Serial Monitor says WiFi OK, IP ... and then noise floor .... The screen shows "Listening to the room... stay quiet" for about a second, then two eyes.

Stay quiet during that first second. Buddy uses it to learn how noisy your room is, so it knows what counts as someone speaking.

Step 7: Talk to Buddy

Just talk, normally, about 30 to 50 centimetres from the microphone. There is no button to hold. Try:

  • "Hello Buddy, what's your name?"
  • "What is two plus two?"
  • "Tell me a fun fact."

This is what happens on the screen, in this order:

  1. The eyes widen, so Buddy knows you are speaking.
  2. The screen says Thinking... while your sentence travels to the relay and Claude answers.
  3. The reply appears as text.
  4. You hear the reply, and the eyes come back when Buddy is ready to listen again.

Expect a few seconds between the end of your sentence and the start of the reply. It depends on your computer. Look at the relay window too: it prints heard: (what Whisper understood) and reply: (what Claude said). That window is the best place to see what is going on.

Checkpoint: Buddy answers out loud

You asked something, saw the reply on the screen and heard it. That is the whole project working.

Good to know

  • Buddy cannot hear you while it is talking. Wait for the eyes to come back.
  • Tap the BOOT button on the ESP32 to mute the microphone. The eyes become lines. Tap again to unmute.
  • Buddy answers anyone it hears, including the TV. Use the wake word in the next section if you want it to answer only to its name.
  • If you change the Arduino program, click Upload again. If you change the relay files, close the relay window and start it again.

Every day after that

  1. On your computer, start the relay and paste your key.
  2. Power the ESP32 from any USB charger. After the first upload it only needs the relay.
  3. Wait for the eyes, then talk.

If you restart your router, your computer's address may change. If Buddy then says "Can't reach the relay", find the address again and update RELAY_HOST in the program.

How the code works

You do not need to read any code to build Buddy. If you want to know why it works, these are the pieces that matter. Every snippet below is copied straight from the project files you downloaded.

The pins live in one place

ai_buddy.ino
// ---------------- Pins ----------------
const int PIN_BTN = 0;                                     // BOOT button: tap = mute / unmute
const int PIN_SDA = 32, PIN_SCL = 33;                      // OLED
const int MIC_SCK = 26, MIC_WS = 25, MIC_SD = 35;          // INMP441
const int SPK_BCLK = 14, SPK_LRC = 13, SPK_DIN = 27;       // MAX98357A

Each name is a job and each number is a pin on the ESP32. They match the wiring tables in step 2: SCK, WS and SD for the microphone, BCLK, LRC and DIN for the amplifier. If you wired something differently, change the numbers here to match what you built, then upload again. GPIO 35 can only be an input, which is exactly what the microphone's data wire needs.

Always listening, without a button

Buddy reads the microphone about 30 times a second, in short chunks. For each chunk it asks one question: is this louder than the room normally is? These settings decide the answer.

ai_buddy.ino
// ---------------- Voice detection (tune these if it triggers too much or too little) ----------------
const int CHUNK_FRAMES = 512;            // 32 ms of audio per chunk at 16 kHz
const int PREROLL_CHUNKS = 8;            // keep 256 ms from before speech is detected so the first word isn't clipped
const int START_CHUNKS = 3;              // loud chunks in a row needed to start (about 100 ms)
const uint32_t END_SILENCE_MS = 900;     // this much quiet ends your sentence
const uint32_t MAX_RECORD_MS = 15000;    // safety limit for one sentence
const float VAD_MIN_LEVEL = 60;          // never trigger below this level
const float VAD_RATIO = 3.0;             // trigger when level > background noise x this
const bool DEBUG_LEVELS = false;         // true = print levels to the Serial Monitor to help tuning

And this is the loop that applies them:

ai_buddy.ino, inside loop()
// Always listening: each chunk goes into the ring buffer and is checked for speech
float level = readChunk(preroll[prerollHead]);
if (level < 0) return;
prerollHead = (prerollHead + 1) % PREROLL_CHUNKS;
if (prerollCount < PREROLL_CHUNKS) prerollCount++;

float threshold = fmaxf(VAD_MIN_LEVEL, noiseFloor * VAD_RATIO);
if (level > threshold) {
  loudRun++;
} else {
  loudRun = 0;
  noiseFloor = noiseFloor * 0.97f + level * 0.03f;  // slowly follow the room's background noise
}

if (DEBUG_LEVELS && millis() - lastDebug > 500) {
  lastDebug = millis();
  Serial.printf("level %.0f   noise %.0f   trigger %.0f\n", level, noiseFloor, threshold);
}

if (loudRun >= START_CHUNKS) {
  loudRun = 0;
  converse(threshold);
  // ...
  • The trigger level is the larger of a fixed minimum and three times the room's background noise. A fan or a busy street raises it automatically.
  • Three loud chunks in a row, about a tenth of a second, count as the start of speech. One door slam does not.
  • Every chunk also goes into a small ring buffer. When Buddy decides you have started talking, it still has the quarter of a second before that, so your first word is not clipped.
  • While it is quiet, Buddy slowly follows the room's noise level. That is the line with 0.97f and 0.03f.

Sending a sentence to the relay

Once speech starts, the ESP32 opens a connection to the relay and sends the saved quarter second, then streams live audio until you have been quiet for 900 milliseconds. It cannot know in advance how long you will talk, so it uses chunked transfer, a standard feature of HTTP: each piece announces its own length, and a final empty piece, the 0, means "that is all".

ai_buddy.ino, inside converse()
// 1) the audio from just before speech was detected, oldest first
int first = (prerollHead - prerollCount + PREROLL_CHUNKS) % PREROLL_CHUNKS;
for (int k = 0; k < prerollCount; k++) sendChunk(c, preroll[(first + k) % PREROLL_CHUNKS]);
prerollCount = 0;

// 2) keep streaming until you have been quiet for END_SILENCE_MS
uint32_t startMs = millis(), lastVoice = millis();
float endThreshold = threshold * 0.7f;  // a bit lower than the start level so trailing words don't cut off
float level = 0;
while (c.connected() && millis() - startMs < MAX_RECORD_MS) {
  level = readChunk(chunkBuf);
  if (level < 0) continue;
  sendChunk(c, chunkBuf);
  if (level > endThreshold) lastVoice = millis();
  else if (millis() - lastVoice > END_SILENCE_MS) break;
}
if (millis() - startMs >= MAX_RECORD_MS) noiseFloor = level;  // constant loud noise: raise the bar
c.print("0\r\n\r\n");

The relay: ears, brain and voice

The relay is a small Python web server (FastAPI) with one job: take a sentence of sound and return Buddy's answer. It does that in three moves.

Ears: Whisper turns sound into text

relay.py
def transcribe(pcm: bytes) -> str:
    pcm = pcm[: len(pcm) // 2 * 2]
    audio = np.frombuffer(pcm, dtype=np.int16).astype(np.float32) / 32768.0
    peak = float(np.abs(audio).max()) if audio.size else 0.0
    if peak > 0:
        audio = audio * min(0.9 / peak, 8.0)  # gentle normalisation for the quiet INMP441
    segments, _ = get_whisper().transcribe(audio, vad_filter=True, beam_size=1, language=WHISPER_LANG)
    return " ".join(s.text.strip() for s in segments).strip()

The audio is scaled up a little first, because the INMP441 is quiet. vad_filter=True makes Whisper skip silence, and beam_size=1 trades a little accuracy for speed.

Brain: Claude writes the reply

relay.py, inside talk()
messages = list(history) + [{"role": "user", "content": heard}]
while messages and messages[0]["role"] != "user":
    messages.pop(0)
try:
    resp = await client.messages.create(model=MODEL, max_tokens=200, system=SYSTEM, messages=messages)
    reply = "".join(b.text for b in resp.content if b.type == "text").strip() or "Hmm, I have nothing to say."

history remembers the last six exchanges, so Buddy can follow a conversation. The short loop makes sure the list starts with something you said, which is a rule of the API. max_tokens=200 keeps answers short.

Voice: Edge TTS speaks it

relay.py
async def tts_mp3(text: str) -> bytes:
    mp3 = bytearray()
    async for chunk in edge_tts.Communicate(text, VOICE).stream():
        if chunk["type"] == "audio":
            mp3.extend(chunk["data"])
    return bytes(mp3)

Edge TTS streams back the spoken reply as MP3 audio. The relay then either plays it on your Mac or converts it to the format the ESP32 speaker expects.

Buddy's personality is one paragraph

relay.py
SYSTEM = (
    "You are Buddy, a friendly little desk companion living inside an ESP32 with a tiny screen and speaker. "
    "Your replies are spoken aloud, so answer in one or two short plain sentences. "
    "No markdown, no lists, no emojis."

The reply is spoken aloud, so the instructions ask for short, plain sentences. Without "no markdown, no lists, no emojis", the voice would read out every asterisk.

The face is two rounded rectangles

ai_buddy.ino
void drawFace(bool blink) {
  display.clearDisplay();
  if (blink) {
    display.fillRect(30, 34, 24, 3, SSD1306_WHITE);
    display.fillRect(74, 34, 24, 3, SSD1306_WHITE);
  } else {
    display.fillRoundRect(30, 14, 24, 30, 6, SSD1306_WHITE);
    display.fillRoundRect(74, 14, 24, 30, 6, SSD1306_WHITE);
  }
  display.setTextColor(SSD1306_WHITE);
  display.setTextSize(1);
  display.setCursor(34, 55);
  display.print("just talk :)");
  display.display();
}

The demo at the top of this page draws Buddy's face with exactly these numbers. Change them, upload, and Buddy gets new eyes.

Make it yours

Most changes are one line. Edit the file, save it, and then either click Upload again (for ai_buddy.ino) or restart the relay (for everything in the relay folder).

You want toChange thisIn
Give Buddy a new personalityThe text under SYSTEM = (relay.py
Have it answer only when it hears its nameRemove the # before export WAKE_WORD=buddy. After a reply you can keep chatting for 20 seconds without the name. On Windows set $env:WAKE_WORD="buddy".start_relay.command
Get faster repliesWHISPER_MODEL=base, a smaller and slightly less accurate listenerstart_relay.command
Use a different voiceTTS_VOICE. The default is en-IN-NeerjaNeural. Run edge-tts --list-voices to see the rest.start_relay.command
Talk in HindiTTS_VOICE=hi-IN-SwaraNeural, WHISPER_LANG=hi, and ask for Hindi replies in SYSTEM. The screen font only shows English letters, so Hindi is spoken but not shown.start_relay.command, relay.py
Use another Claude modelCLAUDE_MODEL. The default is claude-haiku-5-5.start_relay.command
Choose where the voice comes outSPEAKER=mac for your Mac's speakers, or SPEAKER=esp32 for the speaker wired to the boardstart_relay.command
Change the volumeMAC_VOLUME for Mac speakers. VOLUME for the ESP32 speaker, and keep that at 0.5 or below.start_relay.command

The launcher lists the optional settings, switched off, so you only have to remove a #:

start_relay.command
# Optional settings: remove the # at the start of a line to turn it on.
# export WAKE_WORD=buddy
# export WHISPER_MODEL=base
# export VOLUME=0.35
# Using Anthropic's own API instead of OneProvider? Remove the # on the next line.
# export ONEPROVIDER_BASE_URL=https://api.anthropic.com

Tune how it listens

If Buddy ignores you or jumps at every noise, set DEBUG_LEVELS to true in ai_buddy.ino and upload. The Serial Monitor then prints level, noise and trigger twice a second, so you can see what the microphone sees.

What happensTry this
It never reacts to youLower VAD_MIN_LEVEL to 30, or lower MIC_SHIFT to 13 or 12.
It reacts to background noiseRaise VAD_MIN_LEVEL or VAD_RATIO.
It cuts you off mid-sentenceRaise END_SILENCE_MS to 1200.
It waits too long after you finishLower END_SILENCE_MS to 700.
The microphone is silentSet MIC_SLOT = 1, and check that the microphone's L/R pin is on GND.

If something goes wrong

Nearly every problem in this build comes down to a loose wire, a wrong pin number or a window that was closed. Find your symptom below. If none of them fits, the last section of this page shows what to send when you ask for help.

Setting up your computer

Arduino says: ESP_I2S.h: No such file

Your ESP32 board package is too old. Open the Boards Manager and install version 3.x of "esp32 by Espressif Systems".

Mac: error: externally-managed-environment

You are using a Python installed by Homebrew. Install Python from python.org (step 1), open a new Terminal window and try again.

Mac: command not found: python3, or a developer tools pop-up

Install Python from python.org (step 1), then quit and reopen Terminal.

Windows: 'python' is not recognized

Run the Python installer again and tick Add python.exe to PATH. Then open a new PowerShell window.

The Arduino IDE does not show a port

Use a cable that carries data (many cheap ones only charge) and try another USB port. Check the chip near the USB socket: install the CP210x driver for a CP2102 or the CH340 driver for a CH340.

Upload sticks on Connecting........

Press and hold the BOOT button on the ESP32 until the upload starts, then let go.

Wiring and the hardware check

The screen stays blank

Check SDA on pin 32 and SCL on pin 33, then VCC and GND. Upload i2c_scan/i2c_scan.ino: your screen should appear as 0x3C. If you only see odd addresses such as 0x0D or 0x52, a wire is loose. Push every wire in firmly and scan again.

The speaker plays a steady tone or hiss

First check that BCLK, LRC and DIN are all firmly connected to pins 14, 13 and 27, and that the amplifier's VIN is on the ESP32 VIN pin. A floating or loose pin makes this noise. If it keeps happening with correct wiring, the amplifier module itself may be faulty. See "What went wrong" below, then switch to your computer's speakers.

There is no beep at all

Amplifier VIN to the ESP32 VIN pin, GND to the rail, then BCLK 14, LRC 13, DIN 27. Check that the speaker wires are tight in the terminals.

The microphone bar never moves

Microphone VDD on 3V3, L/R on GND, SCK 26, WS 25, SD 35. If all that is right, change MIC_SLOT = 0 to MIC_SLOT = 1 and upload again.

The key and the relay

Mac: Apple could not verify start_relay.command

Open Terminal, go to the relay folder (type cd and a space, drag the folder in, press Enter) and run bash start_relay.command. See step 5 for the other way.

The relay window mentions ONEPROVIDER_KEY

The key was not entered. Start the relay again and paste your key when it asks.

test_key.py says 401

The key was copied wrongly. Copy it again with no spaces.

test_key.py says 429, or Buddy says its balance has run out

The balance, or that key's spend limit, is used up. Top up or raise the limit in your provider's dashboard.

The phone cannot open the relay address

Check the address, that the phone is on the same Wi-Fi (guest networks often block this) and that the relay window is still open. On macOS, make sure Terminal is allowed under System Settings, Privacy & Security, Local Network. On Windows, allow Python through the firewall on private networks.

Talking to Buddy

The screen says WiFi failed

Check the Wi-Fi name and password, where capital letters count. The network must be 2.4 GHz.

The screen says it can't reach the relay

The relay window is closed, RELAY_HOST is wrong, the computer is asleep or the computer and the ESP32 are on different networks. Run the phone test from step 5.

Buddy worked, then stopped answering

Your computer probably went to sleep or the lid was closed. Wake it. The relay window should still be there.

Buddy never reacts to my voice

Set DEBUG_LEVELS = true, upload, and watch the Serial Monitor while you talk. If level never goes above trigger, lower VAD_MIN_LEVEL to 30 or MIC_SHIFT to 13.

Buddy reacts to every little noise

Raise VAD_MIN_LEVEL (try 120), or use the wake word.

Buddy cuts me off mid-sentence

Raise END_SILENCE_MS to 1200.

The relay window says heard: (nothing)

Speak a little louder or closer, or lower MIC_SHIFT to 13.

Buddy answers itself in a loop

Its voice is reaching the microphone. Lower the volume, move the microphone away from the speaker, or use the wake word.

Mac: the screen shows the reply but there is no sound

Check that the Mac's volume is up and not muted, and that start_relay.command contains export SPEAKER=mac. The relay window prints reply: ... when it works. If it says afplay was not found, you are not on a Mac.

Replies are slow

Use the smaller speech model: remove the # before export WHISPER_MODEL=base in start_relay.command.

What went wrong while this was built

The first version of anything rarely works. These are the real problems that came up while building this Buddy, and what fixed them. If you hit one, you are in good company.

  • The screen vanished

    The OLED worked, then showed nothing. The I2C scan reported strange addresses, 0x0D and 0x52, instead of the screen's 0x3C. Nothing was wrong with the code: a jumper wire had worked loose in the breadboard.

    Lesson: when a module disappears, re-seat every wire on it before you touch the code. The scan sketch in the project exists for exactly this.

  • The amplifier would not stop humming

    With the speaker wired up, the amplifier played a steady tone no matter what the code sent it. Connecting BCLK and LRC alone made noise, and all three signal wires made a continuous tone. Several test programs, including one that sent only silence and one that used an older audio driver, gave the same result. The module was most likely faulty.

    Lesson: test the speaker early, with the hardware check, and keep a plan B. Here that was playing Buddy's voice through the Mac's speakers, which needs no amplifier at all.

  • The diagram and the code disagreed

    The first wiring diagram used pins 32 and 33 for the screen, 26 and 25 for the microphone clock and 14 and 13 for the amplifier. The first version of the code expected different pins, so nothing worked until one side was changed to match the other.

    Lesson: the pin numbers sit in three lines at the top of the program. Write down what you actually wired, and make those lines say the same thing.

  • The Mac refused to run the launcher

    Double-clicking start_relay.command showed "Apple could not verify..." because the file came out of a zip. It is a plain text script, so running it with bash start_relay.command from Terminal avoids the warning completely.

  • The hotspot network looked different

    On an iPhone hotspot, the Mac's address was 172.20.10.2 instead of the usual 192.168.x.x. That is normal. The ESP32 needs the 2.4 GHz band, so the hotspot's Maximize Compatibility switch had to be on.

Safety, cost and privacy

  • Speaker: the small speaker is rated 0.5 W and the amplifier can deliver far more. Keep VOLUME at 0.5 or below.
  • Power: the microphone gets 3V3 only, and you unplug the USB cable before you move wires.
  • Cost: with a small model such as the default, a short chat costs a tiny fraction of a cent. Check your provider's price list, and set a spend limit on the key so a mistake can never get expensive.
  • What travels where: Buddy is always listening, but sound only goes from the ESP32 to your own computer, and only when it hears speech. The written text of your sentence goes to Claude through your provider. The text of Buddy's reply goes to Microsoft's Edge text-to-speech service so it can be spoken. Your voice recording is not sent to either.
  • Mute button: tap BOOT on the ESP32 whenever you do not want to be heard.
  • Third parties: OneProvider is a separate company, not Anthropic, and what it does with your text is its own policy. Do not tell Buddy anything you would not type into any online service.

Get the project files

Everything you need is in one zip: the test programs, the Buddy program for the ESP32 and the relay for your computer.

What is inside the download
ai_buddy_project/
  README.md                short technical reference
  hardware_check/          test your screen, speaker and microphone first
  i2c_scan/                finds your screen if it does not show up
  speaker_test/            a plain speaker test
  ai_buddy/                the Buddy program for the ESP32
  relay/                   the program that runs on your computer
    relay.py               speech to text, Claude, text to speech
    requirements.txt       the Python packages it needs
    test_key.py            checks that your Claude key works
    start_relay.command    Mac launcher (asks for your key)

Stuck? Send three things when you ask for help: the text on the Buddy screen, the text in the Serial Monitor and the text in the relay window. With those, most problems can be found in minutes. Ask for help.

Ideas for what to build next

  • Switch on the wake word so Buddy only answers to its name.
  • Give it a new voice or a second language.
  • Draw a different face: more expressions for thinking, speaking and muted.
  • Print or build a small case so it sits on your desk without a breadboard.
  • Run it from a power bank and carry it around the house.

Built with the ESP32 Arduino core, the Adafruit SSD1306 and GFX libraries, faster-whisper, FastAPI, edge-tts and Claude.

This guide is independent. Claude is made by Anthropic, and OneProvider is a separate company. Neither has reviewed or endorsed this project.

NVIDIA Omniverse vs Unity: Which One Do You Actually Need?

If you’ve spent any time around 3D, XR, or digital twin conversations lately, you’ve probably heard someone say “we should look at Omniverse” in a meeting where everyone was quietly thinking “isn’t that just Unity with ray tracing?” It isn’t. And the difference matters more than the marketing pages let on — because choosing the wrong one costs you months, not days.

At TwinReality, we work at the intersection of both worlds — the AR glasses and spatial computing devices where experiences get delivered, and the simulation platforms where serious 3D content increasingly gets built. So here’s the honest, practitioner’s answer to a question we keep getting asked.

The one-line answer

Unity is a game engine for building and shipping an application. Omniverse is a simulation and collaboration platform for connecting tools and producing physically accurate ground truth.

If that sentence fully lands for you, you can stop reading. If it sounds like two ways of saying the same thing — keep going, because the confusion is exactly where expensive mistakes happen.

Why they look similar (and why that’s misleading)

On the surface, the two feel like cousins. Both give you a real-time 3D viewport. Both have a scene hierarchy, materials, lighting, physics, and scripting. Both even use NVIDIA’s PhysX physics engine under the hood. Open either one and you’ll feel at home within an hour.

But the resemblance is skin-deep, because the two platforms are built for opposite ends of the 3D pipeline — and they make opposite trade-offs at almost every layer beneath the viewport.

The philosophical difference in one image

Think of it this way:

Unity is a destination. Assets flow in from everywhere, get assembled, and leave as a finished application. The scene exists to become a build.

Omniverse is a hub. Tools and data connect to it and stay connected. The scene exists to be the shared, accurate, always-current model that everything else — design reviews, robot training, marketing renders, AI pipelines — plugs into.

Neither philosophy is better. They’re answers to different questions.

A side-by-side that actually helps

 UnityNVIDIA Omniverse
Core purposeBuild & ship an interactive appAggregate, simulate & collaborate on 3D data
Typical outputA game/app running on a deviceDigital twins, synthetic data, photoreal renders, streamed 3D
RenderingRasterised, speed-firstRTX path-traced, accuracy-first
Physics (both PhysX)Tuned for game feelTuned for real-world fidelity
Scene formatProprietaryOpenUSD (open, interoperable)
Runs onPhones, headsets, consoles, browsersRTX workstations & cloud GPUs
Deploys to AR glasses?Yes — its home turfNot directly; export or stream
LicensingPer-seat/revenue-based tiersFree for development, production & redistribution (as of 2026)
Best mental modelA destinationA hub

The question that actually decides it

Forget feature lists. Ask yourself one question:

Is my end product an application, or is it accurate 3D truth?

Choose Unity if: your deliverable runs on a device. A mobile AR try-on. A training app on Quest. An AR game. A branded experience — the kind of interactive content that lives alongside things like AR filters for brands. Anywhere the user holds or wears the endpoint, Unity (or Unreal) is your answer.

Choose Omniverse if: your deliverable is a faithful digital version of something physical. A factory layout you want to validate before pouring concrete. A warehouse where you’ll train robot fleets virtually. A product you need rendered with physical accuracy across a thousand marketing variations. Synthetic camera data to train a vision model. This is why Indian manufacturers like Ola Electric built their factory digital-twin platform on Omniverse, and why Reliance is using it to plan its Jamnagar solar gigafactory — these are truth problems, not app problems.

Choose both if: you’re doing what we think is the most interesting work happening right now — which brings us to the part most comparisons miss.

Unity: a destination

Unity’s entire architecture is organised around one goal: shipping a runtime application to a device. You import assets, assemble a scene, write gameplay or interaction logic, and export a build that runs on a phone, a Quest headset, a console, or a browser.

Everything about Unity is optimised for that endpoint:

Its renderer is rasterised and aggressively performance-tuned, because it must hit 60–90fps on a mobile chipset, not on a workstation GPU. Its physics is tuned for game feel — believable, responsive, cheap to compute — rather than laboratory accuracy. Its asset pipeline is a one-way street: content flows in, gets baked and compressed, and ships out as part of a build. And its scene format is proprietary, which is fine, because a Unity scene’s job is to become an app, not to be shared with other software.

For AR specifically, this is why Unity has dominated for a decade. When we cover devices like the Xreal One Pro or the Ray-Ban Meta smart glasses on this site, nearly every third-party experience running on hardware like that traces back to Unity (or Unreal). If your deliverable is an experience running on a device in someone’s hand or on someone’s face, Unity is not just an option — it’s the default.

Omniverse: a hub

Omniverse starts from a completely different question: not “how do I ship an app?” but “how do I get every 3D tool, every dataset, and every team looking at one accurate, living model of something real?”

That reframing explains every design decision NVIDIA made:

It’s built on OpenUSD, not a proprietary format. Universal Scene Description — the open scene format originally developed at Pixar — is the beating heart of Omniverse. USD isn’t just a file type; it’s a live, layered scene graph that Maya, Blender, Revit, CATIA, Unreal and dozens of other tools can all read and write. Omniverse doesn’t want to replace your tools. It wants to connect them, with one source of truth in the middle.

Its rendering is physically accurate, not just pretty. Omniverse renders with RTX path tracing — light behaves the way light actually behaves. That sounds like a luxury until you realise why it exists: if you’re using a virtual factory to train a computer-vision model, or validating how a sensor perceives a scene, “looks about right” is worthless. You need ground truth. The render isn’t the product; the accuracy is.

Its physics is simulation-grade, not game-grade. Same PhysX foundation as Unity, tuned in the opposite direction — toward real-world fidelity for robotics, manipulation, and industrial processes, where the point is to predict what will happen in reality, not to feel fun.

It doesn’t ship apps to phones. Omniverse runs on RTX workstations or cloud GPUs. Its outputs are renders, simulations, synthetic training data, and streamed experiences — not an APK you sideload onto a headset.

The part nobody tells you: they’re a pipeline, not rivals

Here’s the insight that reframes the whole “versus” question: Omniverse and Unity aren’t competing for the same seat. They’re two stations on the same assembly line.

A realistic modern workflow looks like this:

  1. Build truth in Omniverse. Aggregate CAD, scans, and BIM data into one OpenUSD scene. Simulate it. Validate it. Render it with physical accuracy. This is your single source of truth.
  2. Deliver experience through Unity. Export or translate the relevant slice of that USD scene into Unity, optimise it for mobile-class hardware, add interaction, and ship it to the device in someone’s hand — or increasingly, on their face.

The physically accurate factory twin becomes the lightweight AR walkthrough a site manager views through smart glasses. The path-traced product configurator becomes the mobile AR try-before-you-buy experience. The simulation feeds the experience; the experience never had to compromise the simulation.

This upstream/downstream split is also visible in the market. Boutique integrators who started in Unity for industrial visualisation have been migrating their simulation work to Omniverse and OpenUSD — while keeping game engines for the delivery layer. The tools found their natural positions once teams stopped treating them as substitutes.

And with smart glasses finally maturing into genuine computing devices — a shift we’ve tracked closely, from Meta’s push into its own smart glasses to the growing question of everyday practicality like prescription support — the demand for this exact bridge is only going to grow. The industrial world is building accurate twins in Omniverse. The consumer world is putting displays on faces. Somebody has to connect the two ends of that pipeline.

What changed in 2026 (and why it lowers the stakes)

One practical update worth knowing: as of mid-2026, NVIDIA made Omniverse free for development, production, and even redistribution — no enterprise subscription required (paid support remains optional). The old cost barrier that made “let’s just try Omniverse” an expensive experiment is gone. The main remaining requirement is hardware: you’ll need an RTX GPU to run it seriously.

That means the honest advice today isn’t “pick one.” It’s: keep Unity for everything device-facing, and start learning OpenUSD and Omniverse for everything truth-facing — because the cost of exploring the second one just dropped to zero.

The bottom line

“Omniverse vs Unity” is the wrong frame. The right frame is:

  • Unity answers: how do I put an experience on a device?
  • Omniverse answers: how do I create and maintain an accurate digital version of something real?

Most serious spatial-computing work over the next few years will need both answers — accurate twins upstream, delivered experiences downstream. The teams that understand both ends of that pipeline, and how OpenUSD carries content between them, are the ones who’ll be genuinely hard to replace.

That intersection — where physically accurate simulation meets the AR devices people actually wear — is exactly where we’re focused at TwinReality. If you’re exploring what Omniverse, digital twins, or OpenUSD workflows could mean for your product or content pipeline, get in touch — we’d love to talk.


Related reading on TwinReality:

Can You Get Prescription Lenses for Smart Glasses? (Ray-Ban Meta, Fury & More)

meta description

Key Takeaways

    • Yes — most smart glasses, including Ray-Ban Meta and Meta Fury, support prescription lenses.
    • You can order official prescription lenses through Meta or Ray-Ban directly, or use third-party opticians.
    • Third-party services like Fuse Lenses and Lensology are popular options for custom prescription swaps.
    • “Optics” models (like Ray-Ban Meta Blazer/Scribe) are built specifically for prescription wear, with thinner frames designed for opticians.
    • Standard sunglass-style smart glasses can technically be fitted with prescriptions too, but with slightly more difficulty.

Why This Question Comes Up So Often

If you’ve watched any Ray-Ban Meta or Meta Fury review, the comments section is full of one question: “Can I actually wear these if I need glasses?” It’s one of the most common points of confusion for buyers, because smart glasses are marketed like sunglasses and tech gadgets — not like prescription eyewear.

This isn’t unique to one brand either. As augmented reality glasses go mainstream — from camera glasses to full display-based AR — prescription support is quickly becoming a baseline expectation rather than a nice-to-have.

The short answer is yes, prescription options exist. But how you get them, and how well they fit, depends on which model you choose.

Option 1: Order Prescription Lenses Directly from Meta or Ray-Ban

Both Meta and Ray-Ban allow you to add prescription lenses when purchasing certain smart glasses models directly through their websites. During checkout, you can typically select:

    • Single-vision prescription lenses
    • Transition (light-adaptive) lenses
    • Blue-light or polarized options, depending on the model

This is the most straightforward path, since the lenses are fitted for you before the glasses ship. The tradeoff is that Meta and Ray-Ban’s supported prescription range is limited — very high or complex prescriptions may not be supported through official channels.

Meta Glass website
Image:

Option 2: Use Third-Party Prescription Lens Services

If your prescription falls outside what Meta or Ray-Ban officially supports, or you simply want more lens options, third-party services are a popular workaround. Two names come up repeatedly among smart glasses owners:

    • Fuse Lenses — offers replacement prescription and polarized lenses designed to fit specific smart glasses frames.
    • Lensology (UK-based) — cuts custom prescription lenses for smart glasses frames sent in by the customer.

The general process: you buy the smart glasses frame (without prescription), then send the frame — or your prescription details — to the lens service, who fits or ships lenses matched to your prescription.

fuse lens
Source: Outdoor Sportswire

Option 3: Visit a Local Optician

Many opticians can cut and fit prescription lenses into smart glasses frames, similar to how they would with any other eyewear. This works best with frames designed for easier lens installation.

This is where the distinction between regular smart glasses and “optics” models matters.

Source: Istock
Source: Istock

Why Some Models Cost More: Optics vs. Standard Frames

If you’ve compared prices between something like the Ray-Ban Meta Wayfarer and the Ray-Ban Meta Blazer or Scribe “Optics” versions, you may have noticed a steep price jump — often $200–300 more, despite near-identical tech (same camera, chip, and microphone count).

The difference isn’t the electronics — it’s the frame construction:

    • Optics models use thinner frames specifically engineered for prescription lens fitting, making it much easier for opticians to install RX lenses accurately.
    • Standard sunglass-style models are built primarily as sunglasses first, which makes prescription fitting possible but slightly more finicky for opticians to work with.

If you plan to wear your smart glasses as your primary, all-day eyewear, an Optics model may save you a trip to a specialist optician. If you’re comfortable working with a third-party lens service or a flexible local optician, the standard (cheaper) models can still get the job done.

What About Farsightedness or High Prescriptions?

If you have a prescription outside the standard range Meta or Ray-Ban support, your best options are:

    1. Check if a third-party lens service (like Fuse Lenses) supports your prescription strength.
    2. Ask a local optician if they can fit your lenses into the frame — standard sunglass frames can usually accommodate this, just with more manual work.
    3. Wait for wider third-party support, as more lens manufacturers are catching up to smart glasses’ popularity.

Prescription support is just one part of choosing the right pair. If you’re still deciding between frame styles and features, our Ray-Ban Meta review and Meta’s own smart glasses launch breakdown cover the fit, comfort, and tech differences in more depth.

Quick Comparison

 

PathBest ForTradeoff
Official Meta/Ray-Ban checkoutSimplicity, standard prescriptionsLimited prescription range
Fuse Lenses / LensologyCustom or high prescriptionsExtra step, added cost
Local opticianHands-on fitting, immediate helpFit quality depends on frame type
Optics models (Blazer/Scribe)All-day prescription wearersHigher upfront price

Frequently Asked Questions

Can you get prescription lenses on Ray-Ban Meta glasses?

Yes. You can order them directly through Meta or Ray-Ban’s website, or have prescription lenses fitted by a third-party service or optician.

Do prescription lenses work with Meta Fury glasses?

Yes, Meta Fury supports prescription lenses through the same channels — official checkout, third-party lens services, or a local optician.

What is the difference between “Optics” models and regular smart glasses for prescriptions?

Optics models (like Blazer and Scribe) use thinner frames built specifically for easier, more precise prescription lens fitting. Regular sunglass-style frames can still support prescriptions, but may be more difficult for opticians to work with.

Are third-party prescription lenses reliable for smart glasses?

Many owners use services like Fuse Lenses and Lensology successfully. Reliability depends on the provider, so check reviews and confirm they support your specific frame model before ordering.

Do I need special lenses if I have a high prescription?

If your prescription is outside the standard range supported by Meta or Ray-Ban directly, a third-party lens provider or local optician is usually the better route.

I Used XREAL One Pro as My Only Work Monitor for 5 Days — Here’s What Broke

XREAL One Pro

Most XREAL reviews end after 20 minutes.

Someone unboxes it, connects it to a laptop, waves their head around, says “wow, it’s like a floating screen,” and calls it a review. That’s useful for buying decisions. It tells you almost nothing about what actually happens when you try to work in these glasses — not casually, but seriously. Eight hours a day. Calls, code, documents, deadlines.

So I tried exactly that.

For five working days, I closed my external monitor, put on the XREAL One Pro, and used it as my only screen. No backup. No “I’ll just quickly check this on the monitor.” Just the glasses, a laptop, and a full work week.

Here’s what I learned — including the specific moments it fell apart.

The Setup

Hardware: XREAL One Pro (medium size), connected via USB-C to a Windows laptop. No Beam Pro. No external hub. Just the glasses and the cable.

Work type: Writing, Notion, Google Docs, email, two video calls per day, occasional browser research.

What I was hoping to prove: That you can genuinely go monitor-free — especially useful in India where desk space is limited, power cuts mean moving rooms, and working from a café means carrying a portable monitor or squinting at a 14-inch screen.

I was not trying to game or watch movies. I was trying to work.

Day 1: The Honeymoon

The first morning was genuinely exciting.

I opened Notion, set the virtual screen to about 130 inches, and anchored it in place. The text was readable. Not phone-sharp, not monitor-sharp — somewhere in between, like a good projector in a lit room. Acceptable.

The thing that surprised me most was posture. I sit badly at my desk — chin down, neck forward, hunched. With the glasses, your natural gaze is straight ahead. Within an hour I noticed my shoulders were further back. That part worked exactly as promised.

I got through a full morning of writing without a single issue.

Then I got on my first video call.

Where It First Cracked: Video Calls

Here’s what nobody tells you about using AR glasses on a video call: you look like you’re staring slightly past everyone.

The screen is floating in front of you — but your camera is pointing at your face, not at the screen. So your eyes are tracking slightly upward and to the left of where the camera is. To the other person on the call, you appear to be looking just off-camera the entire time.

I didn’t notice this until my second call when a colleague said, “Are you looking at something else?”

I wasn’t. I was looking directly at his face — on my virtual screen. But it looked like I wasn’t.

This is a real problem for client calls. A mild annoyance for internal ones. By Day 3, I’d adjusted my setup so the screen was positioned lower and more central, which helped. But it never fully went away.

Day 2: The Text Readability Wall

Day 2 is when I hit the problem that multiple commenters on XREAL reviews flag but nobody explains properly.

The XREAL One Pro is 1080p. At 130 inches of virtual screen, that’s a lot of pixels spread over a lot of space.

At default screen size, reading dense text — say, a Google Sheet with small cells, or a codebase in VS Code with 12pt font — is uncomfortable. Not impossible. Uncomfortable. You find yourself leaning forward slightly, which defeats the posture benefit.

The fix is simple in theory: reduce the virtual screen size to around 80–90 inches. That concentrates the resolution, and suddenly text becomes noticeably crisper. But at 80 inches, the experience starts feeling less like a big monitor and more like a laptop screen floating in space. The main selling point (the size) disappears.

This is the tension nobody talks about: the bigger you go, the worse text looks. The smaller you go, the less point there is.

For writing and reading long documents — fine. For spreadsheets, code, or anything with dense small text — you’re making a compromise.

reading text
Credits: The Verge

Day 3: Heat and the Three-Hour Limit

By Day 3 I found something that several One Pro owners mention in passing but nobody quantifies: the glasses get warm.

Not hot. Not uncomfortable-hot. But warm — specifically, right at the bridge of the nose where the device sits. After about 2.5 to 3 hours of continuous use, I noticed myself pushing them up the nose more often. By hour 3.5, I took them off for 10 minutes.

This isn’t a dealbreaker. But it means the glasses are not a seamless 8-hour work device. They’re more of a 2–3 hour sprint tool. If you work in 90-minute blocks with breaks, you’ll never hit this problem. If you’re the kind of person who sits down at 10am and doesn’t look up until 1pm, you’ll feel it.

I also noticed that on my MacBook (tested on Day 4), the heat issue was slightly less pronounced than on Windows. This could be a coincidence, or it could be related to how each system drives the display — I don’t know enough to say definitively.

Credits: How To Geek

Day 4: The Conference Call That Made Me Look Like a Spy

Day 4 included a longer external call — a client presentation where I was sharing my screen.

Here’s a scenario the product marketing does not prepare you for: when you share your screen on a video call while wearing AR glasses, the other person sees your actual laptop display, not your virtual screen.

That sounds obvious in retrospect. But it means that if you’ve moved windows around to work comfortably on your virtual screen — resizing things, rearranging layouts — what you share is the small, cluttered laptop screen you’ve been ignoring all day. My Notion document looked fine to me in the glasses. To my client, it was a tiny window in the corner of a desktop covered in browser tabs.

I had to quickly rearrange my laptop screen mid-call while wearing glasses that weren’t showing me the laptop screen. It was not graceful.

The fix: always keep your laptop screen tidy even when you’re working primarily in the glasses. Or use the glasses withyour laptop screen active, treating them as a second monitor rather than a replacement. Which is probably how most people actually use them — but defeats the “replace your monitor” premise.

Day 5: What I Actually Kept Using

By Day 5, I’d stopped fighting the glasses and started using them for what they’re genuinely good at.

What worked well:

Writing long-form content — articles, documents, anything where you’re in a single window, text is large enough to read comfortably, and you don’t need to switch constantly. The floating screen, the posture improvement, the focus that comes from having only one visible app — all real benefits.

Working from a café or a moving environment — I tested this on a train journey on Day 5. With the screen anchored, I could work while the world moved around me. The screen stayed fixed. This is the use case where the XREAL genuinely has no competition. No laptop screen bouncing with the train, no neck craned toward a tiny display.

Battery-saving on phone — one commenter noted you can disable your primary laptop screen and use the glasses as the only output, which significantly extends battery life. I tested this and it works, though setting it up each time added friction.

What didn’t work:

Spreadsheets and dense data. Video calls where eye contact matters. Any task involving dragging windows across different parts of the screen. Screen sharing.

The Honest Verdict After 5 Days

The XREAL One Pro is not a monitor replacement. Not yet.

It’s a focused work companion — exceptional for single-task work, travel, and environments where a physical monitor is impossible. The moment your work requires multi-window precision, small text, or professional video calls, you start making compromises that pile up over a full day.

The comments under every XREAL review are full of people asking “can I use this for coding?” or “will it replace my monitors?” The honest answer — from five days of actually trying — is: it depends less on the glasses and more on your work type. If your job is writing, content, reading, or anything document-based, you’ll find a rhythm. If your job is data, development, or visual design, you’ll be back to your monitor by Day 3.

One thing I didn’t expect to feel: the glasses made me more focused. When there’s only one screen and it’s floating in front of you, you can’t really multitask. There’s no peripheral monitor pulling your attention. For a certain kind of deep work, that’s not a limitation — it’s the point.

At ₹40,000+, it’s a serious ask. But for the right use case, it’s the most interesting productivity experiment I’ve run in a long time.

What Would Make It Work Properly for Full-Day Use

A 1440p display — the single biggest upgrade that would fix the text legibility problem at large screen sizes.

Wireless connectivity — several commenters mentioned the wire as a dealbreaker for portability. Agreed. The cable is manageable on a desk. On a train, it’s a constant reminder that this is still gen-one technology.

Better thermal management — 3 hours before warmth becomes noticeable isn’t enough for a work tool.

Screen size that adapts to content type — automatically tightening resolution for spreadsheets or code, loosening for movies.


Interested in how XREAL compares to other AR glasses available in India? Read our full AR glasses comparison guide. Or if you’re curious about the glasses that went a different direction entirely — AI without a display — see our Ray-Ban Meta review and the latest Meta standalone glasses launch.

Meta Launches Its Own Smart Glasses

Meta New Smart Glasses

Meta just made its biggest move yet in the race to put AI on your face. On June 24, 2026, the company officially launched a new line of wearables simply called Meta Glasses — dropping the Ray-Ban co-branding it’s been riding for years and stepping into the spotlight under its own name.

The starting price? $299. That’s cheaper than its previous Ray-Ban collab models, and a world away from Snap’s competing Specs glasses that launched just a week earlier at $2,195.

A New Identity, Same Trusted Partner

The glasses are still built in partnership with EssilorLuxottica — the same company behind Ray-Ban and Oakley — but you won’t find either of those logos on the frame. Meta is clearly signaling that it wants to own this space outright, not share the marquee with a legacy eyewear brand.

They come in several designs and color combinations, including a Kylie Jenner-designed model that even plays a custom chime when you put them on. Meta AI can also speak in an AI-generated version of Jenner’s voice. Whether that’s cool or uncanny is up to you.

What Can They Actually Do?

There’s no display built into the lenses — so this isn’t full augmented reality glasses yet. But don’t let that undersell what’s here. The glasses come with a camera, built-in speakers, and a dedicated button that summons Meta AI instantly.

Point them at a bowl of food and they’ll estimate the calories. Spot a sign in Arabic and they’ll translate it on the spot, spoken right into your ear. Need directions? A new “Pedestrian Navigation” feature gives turn-by-turn walking guidance — no phone needed, no screen required. Meta is also rolling out live translation support for 14 new languages, including Mandarin, Japanese, Hindi, and Korean.

The Bigger Picture

Smart glasses shipments surged 167% in the first quarter of 2026 compared to the same period last year. Meta already holds roughly 69% of the market. Daily users of Meta’s glasses have tripled year-over-year, according to CEO Mark Zuckerberg.

But the competition is heating up fast. Google and Samsung are collaborating on AI glasses launching later this year. Apple is reportedly developing its own smart glasses. And OpenAI is working on a hardware product too. The race to become the go-to AI wearable is very much on.

Meta executives have been clear that these display-less glasses are just the beginning — a stepping stone toward lenses with actual screens, and eventually an AI that sees and understands your world around the clock.

Why It Matters

For years, smart glasses felt like a novelty. Expensive, awkward, slightly weird to wear in public. Meta is trying to change that math — by making them affordable, useful, and genuinely good-looking.

At $299 with real AI utility baked in, the question is no longer “would you ever wear smart glasses?” It’s becoming “why aren’t you wearing them yet?”

The answer to that question could define the next decade of personal tech.