• Intro
  • About project
  • Disclaimer
  • History
• Build guide
  • Hardware
    • DHT22 connection
    • LEDs connection
    • Camera and Mic connections
  • Software
    • 1. Raspbian setup
    • 2. Basic configuration
    • 3. Download FruitNanny’s source code
    • 4. Audio and Video pipeline setup
    • 5. Janus WebRTC Gateway
      • Installation
    • Sensors and LEDs access
      • LEDs
      • DHT22
    • Autostart Audio, Video, NodeJS and Janus
    • Nginx
  • Possible Fix for Safary
  • Case
• Bottom line

Intro

About project

Fruitnanny is a code name for a DIY geek baby monitor. It uses RaspberryPi, a NoIR camera module, infrared lights, temperature and humidity sensors, and a custom Web UI. Chrome and Firefox with native WebRTC are used as clients. Right now it means all major platforms like Windows, Linux, Android, MacOS, and iOS soon are supported.

Disclaimer

I assume that a reader worked with RaspberryPI system before and understand Linux systems. Feel free to contact me using Disqus or my email iva9im@gmail.com.

History

When my son was born in March 2016 I got a holly mission to find a video baby monitor. After some research, I didn’t find good candidates(too expensive or didn’t have some features) and, being a geek person, decided to build my own device. That time I had RaspberryPi v1 Model B which had been used for media center. My wife wasn’t happy with the idea but she didn’t have a choice.

At the beginning I thought it would be an easy task, probably someone already had built something similar. Google found dozens of projects but none of them had real-time audio and video capabilities which I wanted to have in my project. Some projects were trying to use VLC streaming, MJpeg or others technics. I was trying to use all of them but wasn’t satisfied. Then I stumbled upon UV4L project and it was promising, especially WeRTC part. I chose this project as a main software part of the project.

Another big part of the project was an additional hardware. Connect a camera to Raspberry is not a big deal, but because baby monitor works mostly during the night, it must have some infrared lights to allow night vision. Plus, it’s good to have sensors data like temperature and humidity.
I didn’t design electronics from the college and I had to refresh a knowledge.

Due to all of these initial problems I managed to build Alpha version only after two months. I called it “lunch box” version:

This version worked good but had several problems:

• Audio quality was horrible. There was a loud noise(like a helicopter)and it was almost impossible to hear a baby’s cry.
• Power problems. After 20-30 minutes system rebooted itself.
• Ugly “lunch box” case :)
• UV4L is distributed only through binaries what prevented any modifications. The biggest concern for me was a fact that I couldn’t modify a WebUI.

Some of the issues were related to low CPU power and some circuits problems in RaspberryPi v1. I didn’t find other way but upgrade to RaspberryPi v3. It’s much more powerful and suites better for video and audio processing and as a bonus contains built-in wifi.

I couldn’t spend a lot of time on the project next several months because of the baby. Progress was very slow but at the end, project was finished and contained completely changed architecture, a new WebUI and, as a bonus, a new case.

Build guide

Hardware

The final version of my device contains next parts. Some parts are mandatory(like RaspberryPi) and some are optional (infrared lights, DHT sensor, case). Parts can be bought in different places. I used AliExpress and Amazon. It’s possible to replace some parts with different compatible models (like Camera module, microphone, and all electronics).

Picture	Name	Links	Description
	RaspberryPI 3	Adafruit	A main part of the device
	NoIR Camera	Adafruit Amazon	A camera module for better night video capture. Can be replaced with v2 module
	Cheap iPhone lens	Amazon	Lens are used only to increase a view angle
	Power Adapter 2A	Amazon	Because device has external lights, sensors and intensively uses WiFi it’s recommended to use 2A power adapter
	Microphone	Amazon	The cheapest and smallest(important!) mic. Not all are good. I replaced several before find working one
	DHT22 sensor	Amazon	A sensor to gather temperature and humidity of environment
	12 x Infrared LEDs (940 nm)	Adafruit	Allow device to see in the darkness. More LEDs more light more power consumption
	4 x resistors (27 Ohm, 1/4 W)	Amazon	Resistors to pair with InfraRed LEDs
	TIP120 Darlington transistor	Adafruit	A transistor is used to control all LEDs through GPIO.
	1 x resistors (270 Ohm, 1/4 W)	Amazon	A resistor to pair with the transistor

When all parts are in place it’s time to assemble them. Below is a simplified connection scheme.

DHT22 connection

It requires 3 wires to connect DHT22. First leg(+) connects to pin 17 on raspberry (3.3V). Second(control) to pin 18 (GPIO24), third one is skipped and fourth(-) connects to pin 20 (Ground).

LEDs connection

A light system is divided into 4 groups of three infrared LEDs and one 27 Ohm resistor, and TIP120 transistor. This “how to” can be useful to understand LED parameters.

All LEDs anodes(long leg) are soldered together and connected to TIP120 leg #3 (Emitter). Each of cathodes(short legs) connects to a resistor and then all together to pin 6(Ground) on raspberry.

I used next formula and datasheet to calculate resistor’s value:

R = (Vsupply – (VF x No. of LED’s)) / IF

25 Ohm = (5V - 3*1.5V) / 0.02A

27 Ohm resistor has the closest value(up) to the result.

TIP120’s #1 leg(Base) connects to pin 3 (GPIO2) through 270 Ohm transistor. This channel to control programmatically lights by changing GPIO value. Leg #2 connects to pin 2 (5V).

Several words about the transistor. RaspberryPi’s gpio pin output is maxed 16mA and 3.3V. So transistor must be able to switch state based on that pretty small amperage. After a small research I found Darlington type of transistors and bought TIP 120. It was designed for higher loads(up to 5A and 60V through Collector-Emitter) but it suits needs of this device as well.

Example of soldering:

Camera and Mic connections

Connect a microphone to any USB ports and a camera module to CSI port.

Software

When everything is assembled it’s time to breathe a life into the device.

The system is based on raspbian OS and next applications:

• Janus WebRTC gateway - setup a WebRTC connections between browser and media streams.
• NodeJS - is a server-side javascript environment. It’s used for 2 main purposes - 1. serve html and other content to browser, 2. run scripts on the server side.
• Nginx - is a proxy to Nodejs and Janus allowing to use single URL access.
• GStreamer is a library to create media pipelines. It forwards a H264 encoded video stream from camera module without modifications to browser. An audio stream is encoded using Opus codec before forwarding.

1. Raspbian setup

Get a new or existing micro SD card and flash a raspbian into it:

• https://www.raspberrypi.org/documentation/installation/installing-images
• https://www.howtoforge.com/tutorial/howto-install-raspbian-on-raspberry-pi/

2. Basic configuration

Install basic software:

sudo apt-get update
sudo apt-get install vim git nano emacs libraspberrypi-dev autoconf automake libtool pkg-config \
    alsa-base alsa-tools alsa-utils

Install NodeJS to serve WebUI:

curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
sudo apt install -y nodejs

Run raspberry config to enable camera and resize partition (options 1 and 5):

sudo raspi-config

Upgrade the Raspberry Pi’s firmware:

sudo apt-get install rpi-update
sudo rpi-update

Follow this instruction to setup WiFi. It’s good to disable WiFi “power save mode” which is enabled by default:

sudo iw dev wlan0 set power_save off

Add line to /etc/network/interfaces

wireless-power off

Access trough .local domain (for instance pi.local or fruitnanny.local)

sudo apt-get install avahi-daemon

3. Download FruitNanny’s source code

FruitNanny repository contains web application, configs and additional tools. They all will be used later in the tutorial.

cd /opt
sudo mkdir fruitnanny
sudo chown pi:pi fruitnanny
git clone https://github.com/ivadim/fruitnanny

4. Audio and Video pipeline setup

Install GStreamer and media plugin to allow media processing:

sudo apt-get install gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-bad \ 
    gstreamer1.0-plugins-ugly gstreamer1.0-plugins-bad libgstreamer1.0-dev \
    libgstreamer-plugins-base1.0-dev gstreamer1.0-alsa

Official repository doesn’t contain a gstreamer plugin for rpi camera module, it need to be built from sources:

git clone https://github.com/thaytan/gst-rpicamsrc /tmp/gst-rpicamsrc
cd /tmp/gst-rpicamsrc
./autogen.sh --prefix=/usr --libdir=/usr/lib/arm-linux-gnueabihf/
make
sudo make install

Files /opt/fruitnanny/bin/video.sh and /opt/fruitnanny/bin/audio.sh contain media pipepline setups

Video pipeline:

gst-launch-1.0 -v rpicamsrc name=src preview=0 exposure-mode=night fullscreen=0 bitrate=1000000 annotation-mode=time+date annotation-text-size=20 ! video/x-h264,width=960,height=540,framerate=8/1 ! queue max-size-bytes=0 max-size-buffers=0 ! h264parse ! rtph264pay config-interval=1 pt=96 ! queue ! udpsink host=127.0.0.1 port=5004  sync=false

Gstreamer asks rpicamsrc for annotated with datetime watermark frames in 960x540 px format and 8 fps then parses them and sends to udp port 5004.

Audio pipeline:

gst-launch-1.0 -v alsasrc device=hw:1 ! audioconvert ! audioresample ! opusenc ! rtpopuspay ! queue max-size-bytes=0 max-size-buffers=0 ! udpsink host=127.0.0.1 port=5002

Gstreamer asks alsa driver for the audio stream, parses it, resamples, encodes it with opus codec and sens it to udp port 5002.

Janus listens both 5002 and 5004 ports for incoming streams.

5. Janus WebRTC Gateway

Janus WebRTC Gateway is a WebRTC server which setup communication between browser and media streams.

Installation

# install prerequisites
sudo apt-get install libmicrohttpd-dev libjansson-dev libnice-dev \
    libssl-dev libsrtp-dev libsofia-sip-ua-dev libglib2.0-dev \
    libopus-dev libogg-dev pkg-config gengetopt libsrtp2-dev
    
# get Janus sources
git clone https://github.com/meetecho/janus-gateway /tmp/janus-gateway
cd /tmp/janus-gateway
git checkout v0.2.5

# build binaries
sh autogen.sh
./configure --disable-websockets --disable-data-channels --disable-rabbitmq --disable-mqtt
make
sudo make install

Fruitannany’s specific Janus configuration files are located in folder /opt/fruitnanny/configuration/janus/:

• janus.cfg - general janus config. Compare to original file it disables only some unused plugins.
• janus.transport.http.cfg - enables http access to janus.
• janus.plugin.streaming.cfg - configures media streams. 5002 port for audio and 5004 for video.

Copy these files into Janus config directory:

cp /opt/fruitnanny/configuration/janus/janus.cfg /usr/local/etc/janus
cp /opt/fruitnanny/configuration/janus/janus.plugin.streaming.cfg /usr/local/etc/janus
cp /opt/fruitnanny/configuration/janus/janus.transport.http.cfg /usr/local/etc/janus

SSL certificates need to be generated to access Janus gateway through https protocol, same certificates will be used for Nginx:

cd /usr/local/share/janus/certs
sudo openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 \
  -keyout mycert.key -out mycert.pem

Sensors and LEDs access

Enable access to GPIO without root.

sudo adduser $USER gpio

LEDs

The application need to have ability to turn on and off infrared LEDs from browser. I added a simple shell script to the Fruitnanny which can blink with infrared light. The same file is used by NodeJS Web app.

DHT22

Install Adafruit DHT module:

git clone https://github.com/adafruit/Adafruit_Python_DHT /tmp/Adafruit_Python_DHT
cd /tmp/Adafruit_Python_DHT
sudo apt-get install build-essential python-dev python-pip
sudo python setup.py install

After module installation Python can get data from the DHT22 sensor. As for LEDs, Fruitnanny contains a python script which prints out current temperature and humidity. The same script is run by the NodeJS Web app.

Autostart Audio, Video, NodeJS and Janus

Now we are one step closer to the final step. Main applications were installed and configured and it’s time to start them.

sudo cp /opt/fruitnanny/configuration/systemd/audio.service /etc/systemd/system/
sudo cp /opt/fruitnanny/configuration/systemd/video.service /etc/systemd/system/
sudo cp /opt/fruitnanny/configuration/systemd/janus.service /etc/systemd/system/
sudo cp /opt/fruitnanny/configuration/systemd/fruitnanny.service /etc/systemd/system/

And time to start everything:

sudo systemctl enable audio
sudo systemctl start audio

sudo systemctl enable video
sudo systemctl start video

sudo systemctl enable janus
sudo systemctl start janus

sudo systemctl enable fruitnanny
sudo systemctl start fruitnanny

At this point you should have everything up and running. To disable some services run sudo systemctl stop SERVICE_NAME.

Modify fruitnanny_config.js to configure the baby monitor.

Params:

• baby_name - baby’s name to display in UI
• baby_birthday - baby’s birthday
• temp_unit - temperature to display in Celsius (C) or Fahrenheit(F)

To update baby’s picture you need to replace file public/project/img/baby.png.

Nginx

To be able to serve janus and nodejs request from one entry point URL(http://RASPBERRY_IP/) I set a Nginx proxy to forward requests to the NodeJS(http://127.0.0.1:7000) app and to the Janus(https://127.0.0.1:8089) gateway.

Run next commands to install and configure Nginx:

# install nginx
sudo apt-get install nginx

# remove default site
sudo rm -f /etc/nginx/sites-enabled/default

# copy fruitnanny configs
sudo cp /opt/fruitnanny/configuration/nginx/fruitnanny_http /etc/nginx/sites-available/fruitnanny_http
sudo cp /opt/fruitnanny/configuration/nginx/fruitnanny_https /etc/nginx/sites-available/fruitnanny_https

# enable new configs
sudo ln -s /etc/nginx/sites-available/fruitnanny_http /etc/nginx/sites-enabled/
sudo ln -s /etc/nginx/sites-available/fruitnanny_https /etc/nginx/sites-enabled/

These sites enable ‘basic authentication’ to protect access to the system. To add a new user and password pair run:

sudo sh -c "echo -n 'fruitnanny:' >> /etc/nginx/.htpasswd"
sudo sh -c "openssl passwd -apr1 >> /etc/nginx/.htpasswd"

Now accessing http://RASPBERRY_IP/ you will be asked to enter credentials and after can see a web page with Video Player.

Possible Fix for Safary

Looks like libnice which is part of Raspberry PI disto is pretty outdated and doesn’t work with Safari browser (iOS/MacOS). To fix the problem the new version of this library need to be compiled from sources.

First, remove old version of libnice

sudo apt-get purge -y libnice-dev

Install build tools:

sudo apt-get install gcc autoconf automake libtool pkg-config gtk-doc-tools gettext python3 gengetopt

Build libnice from sources:

git clone https://gitlab.freedesktop.org/libnice/libnice /tmp/libnice 
cd /tmp/libnice
git checkout 0.1.15
sed -i -e 's/NICE_ADD_FLAG(\[-Wcast-align\])/# NICE_ADD_FLAG(\[-Wcast-align\])/g' ./configure.ac
sed -i -e 's/NICE_ADD_FLAG(\[-Wno-cast-function-type\])/# NICE_ADD_FLAG(\[-Wno-cast-function-type\])/g' ./configure.ac
./autogen.sh --prefix=/usr --disable-gtk-doc
make
sudo make install

Case

I used SketchUp to create 3D models. It was my first experiment with 3D printing and I would fail if not help from my friend Christos. The first model I created was not designed properly and slicer(special software to transform a basic 3d model into a 3d printer language) couldn’t even read it. The second model collapsed on the printing bed and only third attempt was successful…

All models are available on Thingiverse and GitHub:

• https://www.thingiverse.com/thing:2311567
• https://github.com/ivadim/fruitnanny/tree/master/case

Case contains next parts:

• Main case - where RaspberryPI and main electronics live
• Top cover - holds LEDs and Camera module
• Cap - for easy access to Ethernet port
• DHT22 cradle - isolates sensor from hot Raspberry PI

I love how the case looks like but next time I would make it bigger and increase size and number of ventilation holes. Currently, all electronics and wires are tightly coupled and it causes problem with heat, the case needs more airflow.

Bottom line

The guide turned out into a huge instruction but I hope it will help someone who has a similar idea of building a baby monitor. Feel free to adjust the project to your needs, you can add or remove some sensors, add or remove some functionality, create a completely different case. All my sources are opensources

On my own, I had more ideas but didn’t have time resolve all of them. Some of them:

• iOS application.
• Motion detection and notifications - motion application for Raspberry can detect movements using a camera, the only problem that video stream must be splittet between Janus and Motion. I almost accomplished this with GStreamer, ffmpeg and v4l2loopback. Ping me if you are interested in my findings.
• logging temperature and humidity into a database for future analysis.

Thank you for your attention!