Local CCTV with Google Coral ML using Frigate NVR and Home Assistant

Let's talk about home security and surveillance. I'm not too keen on the idea of sending security camera footage sent to a cloud server, let alone reading the continuously changing T&Cs and dealing with subscription fees.

This project is about setting up a CCTV system that uses machine learning for object recognition, and does this without relying on cloud-based services. We're talking about leveraging Frigate and Home Assistant - two powerful tools that, when combined, create a formidable, cloud-free security system.

Why Frigate and Home Assistant?

Frigate is an open-source edge-based object detection system that runs on your local network. It’s incredibly efficient and can process CCTV footage in real-time. It integrates seamlessly with Home Assistant, the open-source home automation platform.

Most guides I found online were either outdated or extremely long, so I'm sharing how I set it up in under an hour.

Getting Started: What You Need

• Cameras: You’ll need IP cameras. Frigate recommends using Dahua, Hikvision, and Amcrest cameras. Ensure your cameras support RTSP as Frigate uses this protocol. Prior to setup, configure the cameras with an RTSP user and password for accessing the RTSP feed. If the password must contain a special character, use _ as I found that Frigate could not escape other characters initially.
• Google Coral USB TPU: Don't skip this. This is the component that is essentially the brain behind the real-time object recognition capabilities and a CPU will not be able to compete with its computational power. The Coral TPU is going to review each frame and recognize objects.
• Storage: For Frigate to save the footage when movement is detected. The "R" in NVR.
• A Computer for Frigate: Depends on the number of cameras and their resolution. You could use Raspberry Pi 4 with a powered USB hub for the storage drive and the Coral TPU for a few lower resolution cameras, or a more powerful PC, depending on your setup’s scale and complexity. I picked up an old HP ProDesk dedicated to running Frigate. This machine will not run a hypervisor, as I wanted to achieve the lowest possible latency with Frigate having direct access to the USB ports.
• Home Assistant Setup: This guide assumes you already have HA set up, at least in out of the box configuration. I recommend using a device separate from the one running Frigate for smoother performance.
• Networking: Ensure your network is configured to allow communication in this order: Cameras > Frigate server > Home Assistant.

High-level Overview

Outlining the process:

1. Set up and harden a Linux machine.
2. Install docker and docker compose.
3. Set up the storage.
4. Install MQTT.
5. Install and configure Frigate.
6. Connect Frigate to Home Assistant.
7. Configure Home Assistant.

Preparing the Host

• Install Ubuntu
  • Install the latest version of Ubuntu with the default settings.
  • Set the hostname to something descriptive, such as 'Frigate'
  • Set a static IP for the machine or use DHCP and reserve the IP in your router.
• Generate an SSH keypair
  • Install PuTTY on your local machine and launch the PuTTY Key Generator (PuTTYGen.exe)
  • Generate a new key pair with a strong passphrase, then save both private and public keys on your local machine.
• Set up access
  • Append the public SSH key.
  • Paste in the public key from PuTTY, then save and close the authorized_keys file with Ctrl+X.
• Update package repositories and upgrade packages.

mkdir ~/.ssh
chmod 700 ~/.ssh
nano ~/.ssh/authorized_keys
# Paste in the public key, then save and exit the editor.
sudo apt-get update && sudo apt-get upgrade -y

• Change the default SSH port, disable root login and force key authentication
  • Edit sshd_config with nano.
  • Change and uncomment the SSH port. In this case, port 22041 will be used.
  • Change and uncomment PermitRootLogin to no.
  • Change UsePAM to no.
  • Close and save with Ctrl+X

sudo nano /etc/ssh/sshd_config
  Port 22 #uncomment change it to 22041 
  PermitRootLogin yes #uncomment and change it to no
  UsePAM yes #change it to no
  PasswordAuthentication no
# Save and exit the editor.

• Configure the firewall
  • Set the default policies in UFW by denying all incoming traffic and allowing all outgoing traffic.
  • Update UFW to allow traffic from the camera(s), Home Assistant and your client device.
  • Restart SSH and UFW to apply the new settings. Port 22 will drop the connection.
  • Start a new SSH session with the private key on port 22041.

sudo ufw default deny incoming
sudo ufw default deny outgoing
sudo ufw allow from $IP_ADDRESS_1 # replace with your camera's IP address, repeat for every camera
sudo ufw allow from $IP_ADDRESS_2 # replace with your Home Assistant's IP address
sudo ufw allow out to $IP_ADDRESS_2 # replace with your Home Assistant's IP address
sudo ufw allow from $IP_ADDRESS_3 # replace with your client (the device you are configuring the server from) IP address
sudo ufw allow out to $IP_ADDRESS_3 # replace with your client (the device you are configuring the server from) IP address
sudo systemctl restart ssh
sudo ufw enable
sudo ufw reload

Hardening the Host

• Updates and Essentials
  • Install Fail2Ban.
  • Enable Automatic Updates.

sudo apt-get install fail2ban
sudo systemctl enable fail2ban
sudo systemctl start fail2ban
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo apt install unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades

• Optional Security Settings
  • Install CrowdSec. This requires a CrowdSec account. CrowdSec is an open-source security monitoring and response tool that analyzes user behavior and traffic on your server to identify and block potential security threats.
  • Configure Fail2Ban.

curl -s https://packagecloud.io/install/repositories/crowdsec/crowdsec/script.deb.sh | sudo bash
apt install crowdsec
sudo cscli console enroll <your ID> --overwrite #get the ID from your CrowdSec console's Security Engine

sudo nano /etc/fail2ban/jail.local
  bantime = 1h #update to 2h
  ignoreip = 127.0.0.1/8 ::1 your_home_assistant_ip #whitelist your Home Assistant IP address by replacing 'your_home_assistant_ip' with it to avoid potential login bans due to failed attempts
  maxretry = 5 #update to 3
  enable jails by adding enabled = true to each jail defined by '[]'

Installing Docker

• Install Docker and Docker Compose
  • Install Docker and Docker Compose.
  • Verify Docker is running.

sudo apt update
sudo apt install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt update
sudo apt install docker-ce
sudo apt-get install docker-compose-plugin
sudo apt-get install docker-compose

docker-compose --version
sudo systemctl status docker

Install the TPU runtime

• Install the Coral TPU runtime

echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list

curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

sudo apt-get update

sudo apt-get install libedgetpu1-std

Configuring the Storage Drive

• Prepare and Mount the Drive.
  • Find the drive with lsblk.
  • Create the new partition (replace X with the drive letter). Press n, leave everything on default, then press w to write the changes.
  • Format the partition as ext4.
  • Create the directory for mounting the new drive.
  • Take a note of the UUID of the drive for mounting it.
  • Mount the drive automatically on startup using fstab.

lsblk
sudo fdisk /dev/sdX (w, then enter to save).
sudo mkfs.ext4 /dev/sdX1
sudo mkdir /mnt/frigate
sudo mount /dev/sdX1 /mnt/frigate
sudo blkid #note down UUID
sudo nano /etc/fstab
UUID=your-uuid /mnt/frigate ext4 defaults 0 2 #add this line to the bottom of fstab
#Save and Exit
cd /mnt/frigate
mkdir config
mkdir storage

Setting up MQTT via Home Assistant

MQTT facilitates the communication between Home Assistant and Frigate. If MQTT is already installed on your instance, skip this part. The credentials will be required for Frigate.

• Set up MQTT:
  • Access your Home Assistant server, navigate to Settings > Add-Ons > Search for Mosquitto broker.
  • Enable Start on Boot, enable Watchdog, disable Auto Update.
  • Click Install and wait for it to complete.
  • Navigate to Settings > People > Add Person.
  • Create a dedicated account for MQTT with the username "mqtt-broker" and a strong password.
  • Restart Home Assistant.
  • According to the official guide, the MQTT integration should be automatically discovered, but in my case, I had to set it up manually.
  • Go to Settings > Devices & Services > Integrations > Add Integration > MQTT.
    • Broker: core-mosquitto
    • Port: 1883
    • Username: mqtt-broker
    • Password: password for mqtt-broker
  • Click Next and Save.

Installing and Configure Frigate

• Install Frigate and WatchTower
  • Create the compose.yml file.
  • Create the Frigate configuration file.
  • Spin up docker compose.
  • Verify that the containers are running. WatchTower is used to ensure the containers are automatically updated when a new version is available.

cd /mnt/frigate
nano compose.yml

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true
    restart: always
    image: ghcr.io/blakeblackshear/frigate:stable
    shm_size: "512mb"
    devices:
      - /dev/bus/usb:/dev/bus/usb
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /mnt/frigate/config/config.yml:/config/config.yml
      - /mnt/frigate/clips:/media/frigate
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "password" #replace with a strong password

  watchtower:
    image: containrrr/watchtower
    restart: always
    environment:
      WATCHTOWER_SCHEDULE: "0 15 05 * * *" 
      WATCHTOWER_CLEANUP: 1
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

nano /mnt/frigate/config/config.yml

mqtt:
  host: [REDACTED_IP] #Home Assistant's Host
  user: [REDACTED_USER]
  password: [REDACTED_PASSWORD]

detectors:
  coral:
    type: edgetpu
    device: usb

go2rtc:
  streams:
    Office: #Change the name of the main stream
      - rtsp://[REDACTED_USER]:[REDACTED_PASSWORD]@[CAMERA IP]:554/stream1
    Office_sub: #Change the name of the sub stream
      - rtsp://[REDACTED_USER]:[REDACTED_PASSWORD]@[CAMERA IP]:554/stream2

cameras:
  Office:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Office #Change the name of the camera to match with the main stream's name. Do not change the localhost IP as this is used for restreaming the footage on your Frigate server.
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/Office_sub #Change the name of the camera to match with the sub stream's name. Do not change the localhost IP as this is used for restreaming the footage on your Frigate server.
          roles:
            - detect

record:
  enabled: True
  retain:
    days: 14
    mode: motion
  events:
    objects:
      - person

snapshots:
  enabled: True
  retain:
    default: 30

rtmp: 
  enabled: false

sudo docker-compose up -d
docker ps

• Navigate to http://{Frigate IP}/5000 and verify the camera is visible.

Connecting and Configuring Frigate to Home Assistant

• Install the Frigate Integration:
  • Access Home Assistant, navigate to Settings > Devices & Services > Add Integration > Search for Frigate.
  • Enter the URL for Frigate and click Submit.
  • The camera should now be visible under Settings > Devices & Services > Frigate.
• Add the Frigate Lovelace Card
  • To add the camera to the Home Assistant dashboard, open > HACS > Frontend > "Explore & Add Integrations" > Frigate Card and install it.
  • Navigate to the Dashboard > Edit > Add Card > Add and configure the Frigate Card by selecting the Frigate camera.

Conclusion

And there you have it - a private and efficient home CCTV system, powered by Frigate and Home Assistant, all running on your local network. This approach provides a user-controlled, reliable surveillance system, emphasizing data privacy and system performance.