- Possible Fix for Safary
- Bottom line
Fruitnanny is a code name for a DIY geek baby monitor. It uses RaspberryPi, a NoIR camera module, infrared lights, temperature and humidity sensors, and a custom Web UI. Chrome and Firefox with native WebRTC are used as clients. Right now it means all major platforms like Windows, Linux, Android, MacOS, and iOS soon are supported.
I assume that a reader worked with RaspberryPI system before and understand Linux systems. Feel free to contact me using Disqus or my email firstname.lastname@example.org.
When my son was born in March 2016 I got a holly mission to find a video baby monitor. After some research, I didn’t find good candidates(too expensive or didn’t have some features) and, being a geek person, decided to build my own device. That time I had RaspberryPi v1 Model B which had been used for media center. My wife wasn’t happy with the idea but she didn’t have a choice.
At the beginning I thought it would be an easy task, probably someone already had built something similar. Google found dozens of projects but none of them had real-time audio and video capabilities which I wanted to have in my project. Some projects were trying to use VLC streaming, MJpeg or others technics. I was trying to use all of them but wasn’t satisfied. Then I stumbled upon UV4L project and it was promising, especially WeRTC part. I chose this project as a main software part of the project.
Another big part of the project was an additional hardware. Connect a camera to Raspberry is not a big deal, but because baby monitor works mostly during the night, it must have some infrared lights to allow night vision. Plus, it’s good to have sensors data like temperature and humidity.
I didn’t design electronics from the college and I had to refresh a knowledge.
Due to all of these initial problems I managed to build Alpha version only after two months. I called it “lunch box” version:
This version worked good but had several problems:
- Audio quality was horrible. There was a loud noise(like a helicopter)and it was almost impossible to hear a baby’s cry.
- Power problems. After 20-30 minutes system rebooted itself.
- Ugly “lunch box” case :)
- UV4L is distributed only through binaries what prevented any modifications. The biggest concern for me was a fact that I couldn’t modify a WebUI.
Some of the issues were related to low CPU power and some circuits problems in RaspberryPi v1. I didn’t find other way but upgrade to RaspberryPi v3. It’s much more powerful and suites better for video and audio processing and as a bonus contains built-in wifi.
I couldn’t spend a lot of time on the project next several months because of the baby. Progress was very slow but at the end, project was finished and contained completely changed architecture, a new WebUI and, as a bonus, a new case.
The final version of my device contains next parts. Some parts are mandatory(like RaspberryPi) and some are optional (infrared lights, DHT sensor, case). Parts can be bought in different places. I used AliExpress and Amazon. It’s possible to replace some parts with different compatible models (like Camera module, microphone, and all electronics).
|RaspberryPI 3||Adafruit||A main part of the device|
|A camera module for better night video capture. Can be replaced with v2 module|
|Cheap iPhone lens||Amazon||Lens are used only to increase a view angle|
|Power Adapter 2A||Amazon||Because device has external lights, sensors and intensively uses WiFi it’s recommended to use 2A power adapter|
|Microphone||Amazon||The cheapest and smallest(important!) mic. Not all are good. I replaced several before find working one|
|DHT22 sensor||Amazon||A sensor to gather temperature and humidity of environment|
|12 x Infrared LEDs
|Adafruit||Allow device to see in the darkness. More LEDs more light more power consumption|
|4 x resistors
(27 Ohm, 1/4 W)
|Amazon||Resistors to pair with InfraRed LEDs|
|Adafruit||A transistor is used to control all LEDs through GPIO.|
|1 x resistors
(270 Ohm, 1/4 W)
|Amazon||A resistor to pair with the transistor|
When all parts are in place it’s time to assemble them. Below is a simplified connection scheme.
It requires 3 wires to connect DHT22. First leg(+) connects to pin 17 on raspberry (3.3V). Second(control) to pin 18 (GPIO24), third one is skipped and fourth(-) connects to pin 20 (Ground).
A light system is divided into 4 groups of three infrared LEDs and one 27 Ohm resistor, and TIP120 transistor. This “how to” can be useful to understand LED parameters.
All LEDs anodes(long leg) are soldered together and connected to TIP120 leg #3 (Emitter). Each of cathodes(short legs) connects to a resistor and then all together to pin 6(Ground) on raspberry.
I used next formula and datasheet to calculate resistor’s value:
R = (Vsupply – (VF x No. of LED’s)) / IF
25 Ohm = (5V - 3*1.5V) / 0.02A
27 Ohm resistor has the closest value(up) to the result.
TIP120’s #1 leg(Base) connects to pin 3 (GPIO2) through 270 Ohm transistor. This channel to control programmatically lights by changing GPIO value. Leg #2 connects to pin 2 (5V).
Several words about the transistor. RaspberryPi’s gpio pin output is maxed 16mA and 3.3V. So transistor must be able to switch state based on that pretty small amperage. After a small research I found Darlington type of transistors and bought TIP 120. It was designed for higher loads(up to 5A and 60V through Collector-Emitter) but it suits needs of this device as well.
Example of soldering:
Camera and Mic connections
Connect a microphone to any USB ports and a camera module to CSI port.
When everything is assembled it’s time to breathe a life into the device.
The system is based on raspbian OS and next applications:
- Janus WebRTC gateway - setup a WebRTC connections between browser and media streams.
- Nginx - is a proxy to Nodejs and Janus allowing to use single URL access.
- GStreamer is a library to create media pipelines. It forwards a H264 encoded video stream from camera module without modifications to browser. An audio stream is encoded using Opus codec before forwarding.
1. Raspbian setup
Get a new or existing micro SD card and flash a raspbian into it:
2. Basic configuration
Install basic software:
sudo apt-get update sudo apt-get install vim git nano emacs libraspberrypi-dev autoconf automake libtool pkg-config \ alsa-base alsa-tools alsa-utils
Install NodeJS to serve WebUI:
curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash - sudo apt install -y nodejs
Run raspberry config to enable camera and resize partition (options 1 and 5):
Upgrade the Raspberry Pi’s firmware:
sudo apt-get install rpi-update sudo rpi-update
Follow this instruction to setup WiFi. It’s good to disable WiFi “power save mode” which is enabled by default:
sudo iw dev wlan0 set power_save off
Add line to
Access trough .local domain (for instance pi.local or fruitnanny.local)
sudo apt-get install avahi-daemon
3. Download FruitNanny’s source code
FruitNanny repository contains web application, configs and additional tools. They all will be used later in the tutorial.
cd /opt sudo mkdir fruitnanny sudo chown pi:pi fruitnanny git clone https://github.com/ivadim/fruitnanny
4. Audio and Video pipeline setup
Install GStreamer and media plugin to allow media processing:
sudo apt-get install gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-bad \ gstreamer1.0-plugins-ugly gstreamer1.0-plugins-bad libgstreamer1.0-dev \ libgstreamer-plugins-base1.0-dev gstreamer1.0-alsa
Official repository doesn’t contain a gstreamer plugin for rpi camera module, it need to be built from sources:
git clone https://github.com/thaytan/gst-rpicamsrc /tmp/gst-rpicamsrc cd /tmp/gst-rpicamsrc ./autogen.sh --prefix=/usr --libdir=/usr/lib/arm-linux-gnueabihf/ make sudo make install
Files /opt/fruitnanny/bin/video.sh and /opt/fruitnanny/bin/audio.sh contain media pipepline setups
gst-launch-1.0 -v rpicamsrc name=src preview=0 exposure-mode=night fullscreen=0 bitrate=1000000 annotation-mode=time+date annotation-text-size=20 ! video/x-h264,width=960,height=540,framerate=8/1 ! queue max-size-bytes=0 max-size-buffers=0 ! h264parse ! rtph264pay config-interval=1 pt=96 ! queue ! udpsink host=127.0.0.1 port=5004 sync=false
Gstreamer asks rpicamsrc for annotated with datetime watermark frames in 960x540 px format and 8 fps then parses them and sends to udp port 5004.
gst-launch-1.0 -v alsasrc device=hw:1 ! audioconvert ! audioresample ! opusenc ! rtpopuspay ! queue max-size-bytes=0 max-size-buffers=0 ! udpsink host=127.0.0.1 port=5002
Gstreamer asks alsa driver for the audio stream, parses it, resamples, encodes it with opus codec and sens it to udp port 5002.
Janus listens both 5002 and 5004 ports for incoming streams.
5. Janus WebRTC Gateway
Janus WebRTC Gateway is a WebRTC server which setup communication between browser and media streams.
# install prerequisites sudo apt-get install libmicrohttpd-dev libjansson-dev libnice-dev \ libssl-dev libsrtp-dev libsofia-sip-ua-dev libglib2.0-dev \ libopus-dev libogg-dev pkg-config gengetopt libsrtp2-dev # get Janus sources git clone https://github.com/meetecho/janus-gateway /tmp/janus-gateway cd /tmp/janus-gateway git checkout v0.2.5 # build binaries sh autogen.sh ./configure --disable-websockets --disable-data-channels --disable-rabbitmq --disable-mqtt make sudo make install
Fruitannany’s specific Janus configuration files are located in folder
janus.cfg- general janus config. Compare to original file it disables only some unused plugins.
janus.transport.http.cfg- enables http access to janus.
janus.plugin.streaming.cfg- configures media streams. 5002 port for audio and 5004 for video.
Copy these files into Janus config directory:
cp /opt/fruitnanny/configuration/janus/janus.cfg /usr/local/etc/janus cp /opt/fruitnanny/configuration/janus/janus.plugin.streaming.cfg /usr/local/etc/janus cp /opt/fruitnanny/configuration/janus/janus.transport.http.cfg /usr/local/etc/janus
SSL certificates need to be generated to access Janus gateway through https protocol, same certificates will be used for Nginx:
cd /usr/local/share/janus/certs sudo openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 \ -keyout mycert.key -out mycert.pem
Sensors and LEDs access
Enable access to GPIO without root.
sudo adduser $USER gpio
The application need to have ability to turn on and off infrared LEDs from browser. I added a simple shell script to the Fruitnanny which can blink with infrared light. The same file is used by NodeJS Web app.
Install Adafruit DHT module:
git clone https://github.com/adafruit/Adafruit_Python_DHT /tmp/Adafruit_Python_DHT cd /tmp/Adafruit_Python_DHT sudo apt-get install build-essential python-dev python-pip sudo python setup.py install
After module installation Python can get data from the DHT22 sensor. As for LEDs, Fruitnanny contains a python script which prints out current temperature and humidity. The same script is run by the NodeJS Web app.
Autostart Audio, Video, NodeJS and Janus
Now we are one step closer to the final step. Main applications were installed and configured and it’s time to start them.
sudo cp /opt/fruitnanny/configuration/systemd/audio.service /etc/systemd/system/ sudo cp /opt/fruitnanny/configuration/systemd/video.service /etc/systemd/system/ sudo cp /opt/fruitnanny/configuration/systemd/janus.service /etc/systemd/system/ sudo cp /opt/fruitnanny/configuration/systemd/fruitnanny.service /etc/systemd/system/
And time to start everything:
sudo systemctl enable audio sudo systemctl start audio sudo systemctl enable video sudo systemctl start video sudo systemctl enable janus sudo systemctl start janus sudo systemctl enable fruitnanny sudo systemctl start fruitnanny
At this point you should have everything up and running. To disable some services run
sudo systemctl stop SERVICE_NAME.
fruitnanny_config.js to configure the baby monitor.
baby_name- baby’s name to display in UI
baby_birthday- baby’s birthday
temp_unit- temperature to display in Celsius (
C) or Fahrenheit(
To update baby’s picture you need to replace file
To be able to serve janus and nodejs request from one entry point URL(
http://RASPBERRY_IP/) I set a Nginx proxy to forward requests to the NodeJS(
http://127.0.0.1:7000) app and to the Janus(
Run next commands to install and configure Nginx:
# install nginx sudo apt-get install nginx # remove default site sudo rm -f /etc/nginx/sites-enabled/default # copy fruitnanny configs sudo cp /opt/fruitnanny/configuration/nginx/fruitnanny_http /etc/nginx/sites-available/fruitnanny_http sudo cp /opt/fruitnanny/configuration/nginx/fruitnanny_https /etc/nginx/sites-available/fruitnanny_https # enable new configs sudo ln -s /etc/nginx/sites-available/fruitnanny_http /etc/nginx/sites-enabled/ sudo ln -s /etc/nginx/sites-available/fruitnanny_https /etc/nginx/sites-enabled/
These sites enable ‘basic authentication’ to protect access to the system. To add a new user and password pair run:
sudo sh -c "echo -n 'fruitnanny:' >> /etc/nginx/.htpasswd" sudo sh -c "openssl passwd -apr1 >> /etc/nginx/.htpasswd"
http://RASPBERRY_IP/ you will be asked to enter credentials and after can see a web page with Video Player.
Possible Fix for Safary
Looks like libnice which is part of Raspberry PI disto is pretty outdated and doesn’t work with Safari browser (iOS/MacOS). To fix the problem the new version of this library need to be compiled from sources.
First, remove old version of libnice
sudo apt-get purge -y libnice-dev
Install build tools:
sudo apt-get install gcc autoconf automake libtool pkg-config gtk-doc-tools gettext python3 gengetopt
Build libnice from sources:
git clone https://gitlab.freedesktop.org/libnice/libnice /tmp/libnice cd /tmp/libnice git checkout 0.1.15 sed -i -e 's/NICE_ADD_FLAG(\[-Wcast-align\])/# NICE_ADD_FLAG(\[-Wcast-align\])/g' ./configure.ac sed -i -e 's/NICE_ADD_FLAG(\[-Wno-cast-function-type\])/# NICE_ADD_FLAG(\[-Wno-cast-function-type\])/g' ./configure.ac ./autogen.sh --prefix=/usr --disable-gtk-doc make sudo make install
I used SketchUp to create 3D models. It was my first experiment with 3D printing and I would fail if not help from my friend Christos. The first model I created was not designed properly and slicer(special software to transform a basic 3d model into a 3d printer language) couldn’t even read it. The second model collapsed on the printing bed and only third attempt was successful…
All models are available on Thingiverse and GitHub:
Case contains next parts:
- Main case - where RaspberryPI and main electronics live
- Top cover - holds LEDs and Camera module
- Cap - for easy access to Ethernet port
- DHT22 cradle - isolates sensor from hot Raspberry PI
I love how the case looks like but next time I would make it bigger and increase size and number of ventilation holes. Currently, all electronics and wires are tightly coupled and it causes problem with heat, the case needs more airflow.
The guide turned out into a huge instruction but I hope it will help someone who has a similar idea of building a baby monitor. Feel free to adjust the project to your needs, you can add or remove some sensors, add or remove some functionality, create a completely different case. All my sources are opensources
On my own, I had more ideas but didn’t have time resolve all of them. Some of them:
- iOS application.
- Motion detection and notifications - motion application for Raspberry can detect movements using a camera, the only problem that video stream must be splittet between Janus and Motion. I almost accomplished this with GStreamer, ffmpeg and v4l2loopback. Ping me if you are interested in my findings.
- logging temperature and humidity into a database for future analysis.
Thank you for your attention!