Raspberry Pi, Ai Camera, SONY IMX500 Object Recognition. Halloween Project.

In a quiet suburban neighborhood, October had arrived, and the air buzzed with Halloween excitement. At the same time, a new Raspberry Pi AI camera came into my hands, and that means it's time to scare someone. People often fear AI because they believe it will take away jobs, but this time, it will help me create a new kind of fear.

Project Architecture

The concept is simple: a reaper that only comes to life when someone walks by, thanks to object detection powered by the new AI camera. Here's how the system works:

An ESP32 and a relay are connected to the reaper, and the ESP32 is linked to a Wi-Fi network.
The reaper is placed near the sidewalk.
The AI camera and Raspberry Pi are hidden nearby, facing the sidewalk, and connected to the same Wi-Fi network.
The AI detection runs continuously, and when the Raspberry Pi detects people, it sends a UDP broadcast message, 'BOO!'
The ESP32 receives the message and activates the reaper, scaring passersby.

Raspberry PI & AI Camera (Sony IMX500) Setup

I used a Raspberry Pi with a standard Debian Bookworm system and some predefined Wi-Fi settings in Pi Imager.

To keep the AI module discreet, I designed an analog of a wildlife camera holder.

Then I put a Raspberry PI and an AI Camera inside and closed it with a cover.

The next step was to install the software. I have a repository that has all the required scripts:

https://github.com/Nerdy-Things/raspberry-pi-sony-imx500-halloween-project

For the Raspberry Pi, I needed to install the following dependencies. It is located in the /raspberry/install.sh

#!/bin/bash

# https://www.raspberrypi.com/documentation/accessories/ai-camera.html

# Required - installing dependencies

sudo apt update && sudo apt full-upgrade

sudo apt install -y imx500-all

sudo apt install -y python3-opencv python3-munkres

#Optional

pip install model_compression_toolkit --break-system-packages

pip install imx500-converter[pt] --break-system-packages

# Download models from https://github.com/raspberrypi/imx500-models

git submodule init

git submodule update

sudo reboot now

The script will work only if it runs inside the cloned repository. Otherwise, all models can be found here.

AI Camera Sony IMX500 Object Detection

After installation, I ran the recognition.py script, which is similar to the AI Camera example. But with the critical changes at the bottom:

while True:

last_results = parse_detections(picam2.capture_metadata())

# Record file to SD card

data_folder = f"../data/images/{DateUtils.get_date()}/"

try:

picam2.capture_file(f"{data_folder}/{DateUtils.get_time()}.jpg")

except:

FileUtils.create_folders(data_folder)

if (len(last_results) > 0):

for result in last_results:

if result.category == 0:

print("Person detected, sending BOO!")

send_udp_message(MESSAGE, PORT)

The script uses an imx500_network_ssd_mobilenetv2_fpnlite_320x320_pp model and gets detection results from the AI Camera, saves an image to an SD card, and if a person is detected (result.category == 0), it sends a UDP broadcast message:

interfaces = socket.getaddrinfo(host=socket.gethostname(), port=None, family=socket.AF_INET)

allips = [ip[-1][0] for ip in interfaces]

# Create the socket once

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)

sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)

for ip in allips:

try:

print(f'sending on {ip}')

sock.sendto(message, ("255.255.255.255", port))

except Exception as e:

print(f"Error sending message on {ip}: {e}")

sock.close()

The remaining part was to place the hidden camera somewhere.

And run the script.

ESP32 & Relay

I want to keep my relay wireless, so I came up with the following schema.

Three AA batteries are connected to a voltage converter that stabilizes the output to 5V.

The converter is connected to the ESP32's 5V and ground pins, and the same wires are connected to the relay’s DC+ and DC-.

The rest was to replace the button with a relay and hang the reaper.

Raspberry Pi, Ai Camera, SONY IMX500 Object Recognition. Halloween Project.