How to Graph Home Router Metrics

Isn't the tech-world wonderful? All these great services and free softwares popping up everywhere. I would like to share my first experience with Grafana and InfluxDB with the purpose of making persistent, beautiful, flexible graphs of your router stat's.

The problem: My router metrics are to be found all over the place in the Web-UI. Some measurements offer real-time graphs over the last couple of minutes, some don't. All stat's are reset when I reboot. I want to back in time and see throughput, temperatures, etc.

The solution: InfluxDB for persistent storage and Grafana for visualization.

InfluxDB is a so called Time Series Database (TSDB) which is specialized for storing data history, tag-value-time.

Grafana is a free self contained web-based graphing tool for InfluxDB (and other TSDBs).

Both softwares are easy to get started with yet powerful. By the way, I am in no way affiliated with eiter project. This project is just to get you going. Once you get started you may find that yourself graphing all sorts of things. This is very well suited for IOT-applications. E. g. I am using the same setup for home automation sensor metrics (temperatures, humidity, etc.) in conjunction with Openhab.

Flames and praise in the comments, please. And let me know if something doesn't work so I can fix it!

The router

In order to get started with this project you need a Linux based router with the following features:

1. Command line login root access (telnet or ssh)
2. Cron support
3. Local file store on internal JFFS or USB storage. This is for storing scripts.

The above features usually don't come in stock firmware so you probably have to go with DDWRT, Tomato or similar. In my case I use ASUSWRT Merlin. ASUS had the good sense to open-source their stock firmware and the Merlin build adds minimal but crucial features. I used a ASUS RT-N66U for this project.

The server

The second pre-req is a x86-based Linux server. It doesn't have to be super powerful. For this project I used a HP microserver with Ubuntu Server 13.04 LTS and 4GB RAM. In theory you could run this off a different processor architecture (e. g. ARM) but you would not be able to use the pre-built packages. The server doesn't have to be dedicated for InfluxDB.

A PC

You need a PC or Mac with terminal software (e. g. Putty or MobaXterm. I prefer the latter).

Some knowledge

This Instructable is for people who have a basic understanding of command-line Linux,

This is valid for ASUS RT-N66U, i. e. you can't follow the instruction to the letter if you have a different router.

Disclaimer! Don't load custom firmware on your router unless you know what you are doing or at least accept that there is a slight risk that you mess your router up to a point of no return (bricking). I have loaded lots of custom firmware on routers and never had any problems but I know problems can happen. However, with Merlin for Asus, the risk is low since it is based on stock firmware.

1. On a PC, download MerlinWRT
2. Extract the downloaded zip. The .trx file holds the firmware.
3. Browse to the router admin interface (usually found at http://192.168.1.1)
4. Go to: Administration->Firmware Upgrade. Choose the downloaded .trx-file and upload.
5. After the router reboots you are on Merlin. Log in again.
6. Go to: Administration->System. Enable all the JFFS-stuff. Press Apply.
7. Reboot router.

Verify:

1. Using your terminal software, log in to the router using the same user and password as in the Web admin.
2. Verify that /jffs is there and contains "configs" and "scripts" directories (see screenshot).

Now the router is ready for custom scripts!

As noted in the pre-req's, you need a small x86-based (Intel, AMD) Linux server for InfluxDB and Grafana. Below instructions work on Ubuntu. Check the Grafana and InfluxDB documentation for installation guides for other distributions.

InfluxDB 0.8 is not the newest version but at time of writing offers best compatibility with applications. This is how to install:

1. Log in to a command line on the server
2. If on 64-bit OS:
   $ wget https://s3.amazonaws.com/influxdb/influxdb_0.8.8_amd64.deb
   $ sudo dpkg -i influxdb_0.8.8_amd64.deb
   
3. If on 32-bit OS:
   $ wget https://s3.amazonaws.com/influxdb/influxdb_0.8.8_i686.deb
   $ sudo dpkg -i influxdb_0.8.8_i686.deb
4. Start daemon:
   $ sudo /etc/init.d/influxdb start
5. Make start on reboot:
   $ sudo update-rc.d influxdb defaults

Grafana 2.1.1 install:

1. Install Grafana as described here: http://docs.grafana.org/installation/debian/

Verify:

1. Verify influxdb by browsing to the influxdb admin gui: http://yourserver:8083. Log in as user root, password root
2. Verify grafana by browsing to http://yourserver:3000/login. Login as user admin, password admin

1. Log in to influxdb admin (root/root) on http://yourserver:8083/
2. Create a metrics database (mydb) with default settings. See screenshot. You can use a different name but you will have to change references to mydb later in the instructable.

Now would be a good time to decide what you want to measure and how. I decided on the below (for which I will provide script examples). For router stat's I started exploring the wl command and I will continue to do so. It is vast. It seems to be the main command line interface to router functionality. In fact, I think you can do everything you can do in the Web UI and a lot more. I think it is a broadcom proprietary command so you will probably have to look for alternatives if you are on a different chipset.

• CPU use. The vmstat command (which I would have preferred to use) is not installed on my router but the top command is. CPU use can be extracted from the following command output:
  
  $ top -bn1 | head -3
  
  
• Memory use. Free and used memory can also be extracted from the top command. See CPU.
  
• Temperature. There are temperature readings per wifi chip to be found, deeply hidden in the wl-command. The result has to be converted, however (see script). Example:
  
  $ wl -i eth1 phy_tempsense

• Ping. I decided to benchmark my external access measuring ping access to a couple of established web sites. Example:
  
  $ ping -c1 -W1 http://www.google.com
  
  
• Throughput. I don't think throughput is available without calculation. Counters, however, can be found in in more than one location. /proc/net/dev is a good counter source. Example:
  
  $ cat /proc/net/dev | tail +3
  
  
• Wireless Clients. Number of connected wireless devices per interface/chip (2.4GHz and 5.0GHz) through the wl-command:
  
  $ wl -i eth1 assoclist

I have ideas for other measurements but I think this is a pretty good set to get started with. Something to keep in mind is that a small home router is not a powerful processing device. Too many, too frequent or too complex measurements will effect router performance adversely.

If you come up with a killer-metric, please share (comment)!

Attached zip-archive contains:
router_assoclist.sh - Sample script for reporting number of attached wireless clients
router_cpu.sh - CPU utilization script
router_mem.sh - Memory utilization script
router_net.sh - Network statistics script. Uses traffic counters. Compensates for counter rollover.
router_ping_ext.sh - Ping roundtrip for one or more destinations
router_temp.sh - Temperatures on 2.4GHz and 5GHz chips in Celsius. If you prefer Fahrenheit, do the math here.
routerstats.sh - main script which fires off the others
todb.sh - a script which takes three arguments: series name, columns and data points. String data is automatically quoted and data is formatted and sent to target database.

This is how you install the sample scripts:

1. Log in to the router command line

2. Create a directory on persistent (jffs) file system:
   
   # mkdir /jffs/scripts/routerstats

3. Extract the zip archive and move the files to the new directory on the router. There are several ways to transfer the files, e. g. you can use a USB-stick or activate SSH in the admin GUI (also activates SCP file transfer). If you decide on SSH/SCP, use Filezilla, modaXterm or similar to transfer files to the router.

4. If you don't know how the vi editor works, now would be a good time to look it up. It is the only file editor on the router. Edit settings in todb.sh: vi /jffs/scripts/routerstats/todb.sh. Edit the following lines to match your setup. You probably only need to change the dbhost-line:
   
   dbname="mydb"
   dbhost="srv4:8086"
   user="root"
   passwd="root"
   
   If you don't want to learn vi, edit the file before moving it to the router.
5. Make the script run every 30s. The following adds two lines to the services-start script that inserts two cron scheduler entries at bootup. On regular linux distros cron entries are persistent and you would just run the cru-commands once and be done with it. Cron only executes on minute intervals so to get 30s intervals the second line is needed. The second line triggers at the same time as the first but waits 30s before doing anything useful.
   
   # echo 'cru a routerstats "* * * * * /jffs/scripts/routerstats/routerstats.sh"' >> /jffs/scripts/services-start
   # echo 'cru a "routerstats+30" "* * * * * (sleep 30; /jffs/scripts/routerstats/routerstats.sh)"' >> /jffs/scripts/services-start

Verify:

1. Log in to the InfluxDB admin GUI (http://yourserver:8083)
2. Click on "Explore Data" next to mydb
3. Enter "list series" as the query (see screenshot)

Grafana visualization is fun to play around with. I will provide you with a sample dashboard but I encourage you to play around with it and tweak it. The are links to good video tutorials on the Grafana home page.

Creating the data source:

1. Log in to grafana (http://yourserver:3000)
2. Create datasource mydb (see screenshot

Import the sample dashboard:

1. Download the attached dashboard file. It contains the json-definition of the entire dashboard. Check it out if you want. It is human readable and can be edited. For instance, if you created a datasource with any other name than mydb you can either search-replace in this file or change the datasource in the GUI later.
2. Import the dashboard (see screenshot)

Now you should have a dashboard that will gradually be filling up with data!

Good luck!