Introducing LilL3x, the Desktop AI Sidekick

Who hasn’t wanted their own little AI sidekick? Imagine having a friendly little chatbox sitting on your desk, checking in with you and always up for conversation. This beginner’s Raspberry Pi project will set you up!

Features

• Integration with multiple LLMs, including:
  • ChatGPT
  • Ollama
  • Claude
  • Gemini
  • And others!
• Wake Word functionality (with PicoVoice Porcupine or Vosk)
• Integration with multiple text-to-speech engines, including:
  • ChatGPT
  • ElevenLabs
  • Amazon Polly
  • Google
  • TypeCast
• A mounted Pi Camera will let the AI know when you are there, as well as comment on the surroundings
• Fully configurable via web interface

Shopping List

• A Raspberry Pi 4B 8 GB
  Less RAM will work, but more is better.
• ReSpeaker 2-Mics Pi HAT
• Mini Speaker
• Raspberry Pi Cooling Fan
• Micro Connector
• Raspberry Pi Camera Module V2
• OLED I2C IIC Display 128×64
• 128 GB SD Card
• Jumper wires (short):
  • 4 F2F
  • 2 M2F
• 3D printer and filament, including clear filament
• Crimpers and related hardware to shorten wires
• glue

Building LilL3x

The first thing that you want to do is to setup your Pi and SSH in. Instructions for this can be found here: Getting started with your Raspberry Pi

Once you’ve done that, you are ready to put the module together. Unplug the Pi and:

• Install the Pi Camera . See this guide for assistance.
• Attach the 90 degree connector to the GPIO pins on the PI. (This will allow you to install a hat while also taking advantage of the GPIO pins.)
• Set up the screen:
  • connect the SDA pin on the screen to pin 3
  • connect the SCL to pin 5
  • Connect the power and the ground to pins 2 and 6 respectively.
• Attach the ReSpeaker 2-Mics Pi HAT to the connector
• Attach the speaker to the hat in the rightmost port. It’s tricky to get in, the exposed wires face downward.
• Use jumper wires to attach pins 1 and 9 to the power (red) and ground (black) of the fan. Note: it might be good to skip this step until you are ready to put the unit in the case.
• Insert the unit into the printed case. Feel free to skip this step until you have a chance to test the screen and camera to make sure that they are hooked up correctly.

3D Printing the Case

The Onshape 3D Model

Print all parts x1 except:

• Part 8 (peg with a notch removed) – print x4
• Part 4 (peg with half head) – print x2
• Part 9 (cylinder) – print x2

Use the following color scheme:

• Print the three big pieces (the case, the door and the lid) in the primary color
• Print the small piece that looks like a hat (piece 6) in clear
• Print the rest of the pieces in the secondary color

You might want to consider printing extras of the smaller peices!

Next, SSH into the Pi and install the software.

Installation

Installing LilL3x is easy, just download and run the setup script:

wget https://raw.githubusercontent.com/followkim/LilL3x/refs/heads/v2/install/install.sh
bash install.sh

Note:

• You will be asked to reboot at one point during the process. Say no.
• There will be an error installing Whisper (no space left on device) but you can ignore this as the error is handled later in the script.
• When you are done, you will still need to setup the audio card, see below.
• The install process takes about 45 minutes. While you are waiting, you should sign up for Pico and Imgur, and your favorite LLM
• GitHub link just for your reference

Setting the audio device

When the install script completes, set the default audio device. The instructions differ depending if you are in bookworm or trixie.

For Bookworm run raspi-config

sudo raspi-config 

In the raspi-config configuration Utility:

1. Select System Options (1)
2. Select Audio
3. Select “seeed-2mic-voicecard”

As of Trixie, raspi-config can no longer set the sound card. I am working to automate this, but currently you need to find and set the audio sink on your own:

• Get the status of Pipewire by calling wpctl status
• Look for the sinks listed in Audio: Sinks, and try the one that isn’t selected. Remember the ID.
• Inspect the ID with wpctl inspect to make sure it’s the right sound card (seeed-2mic-voicecard)
• Set it as default with wpctl set-default

wpctl statuswpctl inspect <id>
wpctl set-default <id>

So in this case above, you’d want to inspect sink 58 to see if it’s the correct card. If it matches, then select it as the default.

Test your sound card with this command, It will record then play back what it just heard, so be sure to make noise after executing it!

arecord -D "plughw:3,0" -f S16_LE -r 16000 -d 2 -t wav test.wav;aplay -D "plughw:3,0" test.wav

You can also test microphone and output separately this way:

cd /~LilL3x
python speech_tools.py # test the speaker
python listen_tools.py # test the microphone (say 'quit' to end)

Start It Up!

You have installed all the needed software for LilL3x, and it’s time to reboot your PI and it will run the software. If everything is going well you will see a screen welcoming you to your project and showing your IP. Press the button on the ReSpeaker to continue.

• To see what is going on type log. This will tail the log, control-c to exit.
• Press the button on the audio HAT to wake the system.

The AI that comes pre-programmed is “Eliza“, one of the earliest known chatbots built by Joseph Weizenbaum at MIT. It won’t be able say anything interesting until you configure the system, which means installing the website. We will do this next.

Troubleshooting

If LilL3x doesn’t boot up, run it manually and see what error you get and share it with me.

cd ~/LilL3x
./lillex.py

Configuring LilL3x

If you go to the Raspberry PI’s IP on a web browser, you should now see a menu and an configuration link.

Most of the defaults are fine, but you will want to fill out:

• Your info and your AI’s name
• The AI engine that you will use (ie ChatGPT), along with an API key and other related credentials
• Wake: If you’ve setup Pico , add your Accesskey and select a wakeword
• Speech Tools: Choose a speech engine and enter your credentials and voice
• Logging Level: set to “Debug Messages” for now

When you are done click the “set” button at the bottom left of the page. Your settings will take place within 30 seconds. Type log to watch the updates as they happen.

Setting up a Wake Word

Go to PicoVoice:

• Sign up for a free account (will need a GitHub account)
• Create a new wake word and download the zip to your computer
• Unzip the file and use FTP to move the file to ~/LilL3x/wake. Do not change the file name.

If PicoVoice is not an option, you can use Vosk which will use you AI’s name as a wake word. Word of warning: this does not work well.

Get an Imgur Key

An Imgur key is needed to upload files to imgur. Some models, such as ChatGPT, need a web address to an image file.

• Go to https://imgur.com/ and register for a free account
• Go to account settings (in the profile picture on the upper right)
• Select “Applications” from the left sidebar
• Create a new Client ID and insert it into the “Camera” section of the configuration page, in field for the Imgur Client ID.