Monitor Fritz!Box connection statistics with Grafana, InfluxDB and Raspberry Pi

I’ve recently stumbled over an article in the german magazine C’T about visualisations of your Fritz!Box’s connection. The solution looked quite boring and outdated, since it used MRTG for the graph creation.

I’ve started searching for a better solution using Grafana, InfluxDB and my Raspberry Pi and found this great blog post. I’ve already explained how to install Grafana and InfluxDB in this post, so I’ll concentrate on the Fritz!Box related parts:

Start with the installation of fritzcollectd. It is a plugin for collectd.

sudo apt-get install -y python-pip
sudo apt-get install -y libxml2-dev libxslt1-dev
sudo pip install fritzcollectd

Now create a user account in the Fritz!Box for collectd. Go to System, Fritz!Box-user and create a new user with password, who has access from internet disabled. The important part is to enable „Fritz!Box settings“.

Additionally make sure that your Fritz!Box is configured to support connection queries using UPnP. You can configure this under „Home Network > Network > Networksettings“. Select „Allow access for applications“ as well as „Statusinformation using UPnP“.

Next part is the installation and configuration of collectd:

sudo apt-get install -y collectd
sudo nano /etc/collectd/collectd.conf

Enable the python and network plugins by removing the hashtag

LoadPlugin python
[...]
LoadPlugin network

Scroll down till you’ll see the plugin configuration and configure the port and IP for collectd

<Plugin network>
    Server "127.0.0.1" "25826"
</Plugin>

Enable the python plugin and configure the module with the username and password of the user you’ve created. Make also sure to use the right address.

<Plugin python>
    Import "fritzcollectd"

    <Module fritzcollectd>
        Address "fritz.box"
        Port 49000
        User "user"
        Password "password"
        Hostname "FritzBox"
        Instance "1"
        Verbose "False"
    </Module>
</Plugin>

Since you’ve already got a running InfluxDB, you’ll just need to enable collectd as data source:

sudo nano /etc/influxdb/influxdb.conf

Search for the [collectd] part and replace it with

[[collectd]]
  enabled = true
  bind-address = "127.0.0.1:25826"
  database = "collectd"
  typesdb = "/usr/share/collectd/types.db"

Reboot collectd and influx to activate the changes made

sudo systemctl restart collectd
sudo systemctl restart influxdb

Login to your grafana installation and configure a new datasource. Make sure to set the collectd database. If you’re using credentials for the InfluxDB, you can add them now. If you’re not using authentication you can disable the „With credentials“ checkbox.

Check if your configuration is working by clicking on „Save & Test“.

If everything worked, you can proceed to importing the Fritz!Box Dashboard from the Grafana.com dashboard. The ID is 713. Make sure to select the right InfluxDB during the import setup.

After clicking on import, you’ll should be able to see your new Dashboard. It might take a few minutes/hours until you’ve gathered enough data to properly display graphs.

Be aware though that if you start gathering this much data you’ll might end up with „insufficient memory“ errors. You’ll might want to tweak your InfluxDB settings accordingly.

Auto mount NFS shares on Raspbian

I’m using influxdb on my Raspberry Pi in combination with a NFS mount. The NFS mount is on my Synology NAS and should store the database data of influxdb. Reason for this setup is that I fear that the SD card won’t survive the many write/read cycles caused by a database writing to it.

The shared folder on my Synology is configured to be accessible by various IPs in my network:

The problem with Raspbian is that I’ve tried to auto mount the NFS share on startup, so that the influxdb service can directly write to the NFS mount. 

I’ve used these settings in my /etc/fstab to mount the volume automatically:

<DS IP>:/volume1/databases /mnt/databases nfs auto,user,rw,nolock,nosuid 0 0

This doesn’t work properly since my influxdb is often dead after a restart, but if I check the mounted volumes I see the NFS volume mounted properly.

However, there’s a tool called autofs which already helped me with a similar problem on my Mac when I moved my iTunes library to the Synology share.

Install autofs using

sudo apt-get install autofs

Open the file /etc/auto.master and add something like this

/mnt    /etc/auto.databases     -nosuid,noowners

Now create a file called /etc/auto.databases with this content

databases       -fstype=nfs,user,nolock,nosuid,rw <DS IP>:/volume1/databases

Unmount the existing NFS share. Remove/comment out the line for the nfs mount in your /etc/fstab so that it doesn’t conflict with autofs. Restart autofs with

sudo service autofs restart

Now check the content of your mount point with e.g.

ls /mnt/databases

Autofs should now automatically mount the NFS share. This might take a while, which is a good sign that the mount is loaded. You can also verify with

mount

that your NFS share is mounted to e.g. /mnt/databases. If you’ll restart now, influxdb should be happy on restart. When it tries to start, autofs will see the access to the mounted folder and will mount the NFS share before influxdb can start up properly.

Howto install InfluxDB and Grafana on a Raspberry Pi 3

Inspired by a friend I’ve decided to install InfluxDB and Grafana on my Raspberry Pi 3. InfluxDB is a database optimized for storing time related data like measurements of my recently installed particle sensor. Grafana is used to create beautiful graphs to display the stored data.

The InfluxDB installation can be done in a few simple steps:

This will install the InfluxDB without a user and any rights. You can read up further on that topic. Ideally you should setup an user for authentication but since some IoT devices do not support this I’m not going to explain it here.

The Grafana installation is similar simple:

Please make sure that you’ll get the most current version from github and replace it in the wget command:

First login to Grafana:

Now you’re ready to configure Grafana. Go to http://<ip-of-grafana-machine>:3000 and setup a new username and password for the webinterface. The default is admin/admin

Configure InfluxDB as datasource in Grafana:

You need to configure a datasource under http://<ip-of-grafana-machine>:3000/datasources

Enter as name the name of the database you’ve created earlier. In this case it was topic.

The type of the database is InfluxDB.

The HTTP connection URL is http://localhost:8086

Hit Save & Test, once you’ve configured everything to your liking. The connection to the database should work now.

Howto setup local DNS domains and adresses in pihole

If you use the AVM FritzBox you’ll now about this dreaded DNS suffix „fritz.box“ which every device will get in your network, if you decide to use the DNS server of the FritzBox. I wanted to have something different which doesn’t collide with domains on the internet, e.g. „stuff.local“. As I already use pihole as adblocker on DNS level I needed a solution to configure it in pihole. The following info is based on the pihole forum.

Create a file called lan.list in /etc/pihole and fill it with content in the following format:

<ip-address> <hostname>.stuff.local <hostname>

Create a second dnsmasq config file which references the file we’ve just created:

echo "addn-hosts=/etc/pihole/lan.list" | sudo tee /etc/dnsmasq.d/02-lan.conf

Restart the dns services in pihole:

sudo pihole restartdns

You should now be able to lookup your stuff.local hostnames on your pi with e.g.

nslookup box.stuff.local
Server:		127.0.0.1
Address:	127.0.0.1#53

Name:	box.stuff.local
Address: 192.168.100.1

DIY HomeKit with Raspberry Pi and Xiaomi Robot Vacuum

Inspired by tweets of Steven Troughten-Smith, people started to experiment with Raspberry Pi’s as DIY HomeBridge devices. One of them is Wojtek Pietrusiewicz. He wrote a nice article which I used as base for this post.

Introduction

I own a Xiaomi Robot Vacuum. This robot can be controlled by the Xiaomi app, however, I don’t like it very much. The idea is to control this robot over HomeKit. To use HomeKit, I use an old Raspberry Pi 1B. The software will be HomeBridge.

Setting up the Pi

Installation and Configuration of Raspbian

Use Etcher to write the image to the SD card. Remount that SD card and add a file called „ssh“ in the root of the mounted partition. This will enable SSH from the beginning so that you can login directly to the Pi. I don’t want to attach a screen or keyboard to that machine so it will only be reachable over the network. Now boot your Pi from this SD card.

Identify the Pi’s IP (e.g. by looking at the network overview in your router). Now connect to that IP with user „pi“. The default password is „raspberry“. Please change the password now with passwd and assign a new user password.

Update everything with apt:

 

Install nodejs 8.9.4

According to Wojtek only this version works currently with HomeBridge. I did not test any other version so I’m just describing what I did on my machine:

  1. Download nodejs for the PI into your users homefolder, e.g. with wget https://nodejs.org/dist/v8.9.4/node-v8.9.4-linux-armv6l.tar.xz
  2. Unpack the file with tar -xvf node-v8.9.4-linux-armv6l.tar.xz
  3. cd node-v8.9.4-linux-armv61
  4. sudo cp -R * /usr/local/
  5. add export PATH=$PATH:/usr/local/bin to e.g. ~/.bashrc
  6. node should be now available

Install HomeBridge

  1. Install the necessary libraries: sudo apt-get install libavahi-compat-libdnssd-dev
  2. Install HomeBridge with npm: sudo npm install -g --unsafe-perm homebridge
  3. open /etc/default/homebridge and safe it with this content:
  4. open /etc/systemd/system/homebridge.service and safe it with this content:

Setting up HomeBridge with Xiaomi Robot Vacuum

  1. sudo npm install -g homebridge-xiaomi-mi-robot-vacuum miio
  2. open ~/.homebridge/config.json and safe it with this content:

    Generate a new MAC address separated by : using this website. You’ll need the IP address of your Xiaomi robot as well as the token. There are several ways to get the token. I’ve extracted mine from the iOS backup. Instead of uploading the token I’ve used this command on the token taken from the sqlite database:
  3. Check if everything is working by starting homebridge for the first time. It should show a QR code. If it does, cancel the process with ctrl+c

Automate HomeBridge startup

Adding the HomeBridge to iOS devices

  1. Install the Home app, if you’ve removed it from your device. You can reinstall it from the App store.
  2. Open the Home app and add a new device
  3. If you’ve give the app access to your camera, you can scan the QR code you’ve seen earlier. However, HomeBridge is now running as a daemon in the background so you won’t see that QR code. You can add the bridge manually by using the PIN you’ve set in the config.