On This Page...
When trying to diagnose why the internet is slow, it can be quite challenging to figure out exactly which device on the network is eating up all the bandwidth. Many solutions to this problem require software to be installed on every device to be monitored. Instead, I endeavored to build a Raspberry Pi network monitor.
When retrofitting our cabin in the woods to a smart-home, this problem was especially challenging. There internet providers out here advertise about 30 Mbps (down) and 2 Mbps (up), which isn’t much to work with. Plus, with so many home-made IOT devices scattered around the house, who knows where the traffic is going?
My goal was to not require any special software, yet monitor the internet traffic for every device on the network.Ideally, I also wanted to create a beautiful Grafana dashboard and also see what sites the devices were connecting to.
It seemed like a couple others on Reddit were also interested in a solution to this problem, so I decided to give it a crack. The result was an open-source Python script / Docker container, meant to be run on a Raspberry Pi, that exports data to Prometheus. While I used a Raspberry Pi, the code should run on any Linux distribution (and there are Docker images for both
Skip to the end of this post for source code & installation.
But first, let’s consider the possible approaches…
Approaches to Traffic Monitoring
Based upon experience and some research, these are the possibilities I came up with:
- Pi as a router
The obvious way to monitor network traffic with a Raspberry Pi is to have the RPi sitting between the devices to be tracked and the internet (e.g., acting as a router or access point). Unfortunately, this can have serious implications for the network, which causes many to avoid the approach (see the next section).
- Router reporting
Some modern routers provide features along these lines. But generally custom firmware is required.
- Device reporting
The standard protocol for this is SNMP, which will rely upon device side installations to self-report. It integrates well with Prometheus/Grafana though.
- Packet sniffing
You could theoretically monitor the wireless traffic (if all you care about is WiFi) in the same way attackers are able to sniff traffic on a WiFi network.
Each of these approaches has its drawbacks. I did not want to buy a new router, so router reporting was not an option. I could not install the necessary software on all the IOT devices, which precluded device reporting. And packet sniffing is an interesting idea, but I wanted to be able to handle wired as well as wireless traffic.
This left only one approach…
Using the Pi as a Router
If all internet traffic is going to pass through a device, caution is warranted.
The first concern is that of security. I won’t say too much about that here, except to mention that a firewall of some kind is a good idea. I went with Uncomplicated Fire Wall (
ufw) because it is, well, uncomplicated.
A less obvious concern is that of speed. When traffic passes through a router/switch, the primary bottleneck is that of the ethernet hardware itself (not CPU). This was something of a problem with the Raspberry Pi 3B (and lower), but the Raspberry Pi 4 has an upgraded on-board 1000 Mbps
This seemed promising, so I drew up a design to test it out:
Some important points:
- The WiFi router is in Bridge mode, meaning that
eth0must act as the DHCP server (assign IP addresses to the network).
- Traffic between devices on the network will not flow through the Raspberry Pi (except for DNS resolution) — see the Performance Tests, below.
- This means that the Pi is only a bottleneck for internet traffic. With 1000 Mbps bridged
eth1ports and an ISP that only provides 30Mbps, we won’t be hitting this limit any time soon.
There are many ways to set up the
eth0 connection. You could configure this using internet bonding software, which means you could add another internet connection (
eth2) to make the internet connection even faster. For a more complete DIY Raspbery Pi router solution:
Pi as a Packet-Sniffer
You could also use the exact same approach, but put a router in front of the Pi.
The network diagram looks a lot like the one above, except you’d have a modem with its own firewall between the Raspberry Pi and the internet. From the Pi’s perspective, nothing really changes.
The installation process (at the end of this post) is still the same.
In either case, the device bridges (passes through) the traffic. The key point is that any router placed behind the Pi is in bridge mode. Or, simply use switches (not routers) behind the Pi. Therefore, the Pi’s
eth0 is able to see traffic passing in and out of the LAN.
After implementing the Pi as a router, I saw no decrease in speed for intra-network traffic. This was tested with iperf3, which showed an average of 910 Mbits/sec for two computers connected via a physical switch, and 180 Mbits/sec when separated by a WiFi hop.
Shockingly, I saw improved external (internet) speeds with the Raspberry Pi as a router. I already had a Node RED instance running a speedtest every 5 minutes and recording the data to Home Assistant + Prometheus. When using the CenturyLink provided DSL router, I rarely saw speeds above 25 Mbps (down). Now, we’re consistently seeing speeds in the ~33 Mbps range. This is likely because the Raspberry Pi is using
pppoeconf to establish the DSL connection directly, and it does a better job managing this connection than the modem provided by the ISP.
Using Prometheus for throughput/bandwidth will not be perfectly accurate on a short time scale (due to the way a
rate is averaged over an interval). However, by downloading a large file, I was able to compare the reported download speed from Chrome with that of the traffic graph:
In addition, the total download size matched that reported by Chrome:
Screenshots, Installation, & Source Code
This project is open-source and available as a Python script or Docker image.
Find the code & detailed documentation in the network-traffic-metrics Github repository.
It also includes a Grafana dashboard:
The most important part of the configuration is setting up the
tcpdump filters. For example, the following will restrict the captured traffic to that which flows in or out of the
(src net 192.168.0.0/24 and not dst net 192.168.0.0/24) or (dst net 192.168.0.0/24 and not src net 192.168.0.0/24)
For more help setting up the filters, check out this blog post by Daniel Miessler on isolating traffic with tcpdump. If you’re having trouble with anything else, please comment below or contact me directly.
I’ll try to give copy and paste instructions below. However, as I mentioned in the “Using the Pi as a Router” section, some of these steps may be highly individual. The bridge steps, in particular, can depend on the exact Linux distribution and version you have installed. I’ll assume you…
eth1(WAN) on your device
- Pi will reside at
192.168.0.2connecting to a router at
- IP addresses will be handed out on the
- You want to use Google’s DNS servers (
1. Build the Router
Please see this article.This section become too large to be contained within this post.
2. Run the script
sudo apt-get install git python3-pip pip3 install argparse prometheus_client git clone https://github.com/zaneclaes/network-traffic-metrics.git cd ./network-traffic-metrics sudo python3 ./network-traffic-metrics.py (src net 192.168.0.0/24 and not dst net 192.168.0.0/24) or (dst net 192.168.0.0/24 and not src net 192.168.0.0/24)
If all went well, you can open your web browser to
http://192.168.0.2:8000/metrics to see the counters being exported for Prometheus.
To make the script start on reboot, type
sudo crontab -e and add:
@reboot python3 /home/pi/network-traffic-metrics/network-traffic-metrics.py (src net 192.168.0.0/24 and not dst net 192.168.0.0/24) or (dst net 192.168.0.0/24 and not src net 192.168.0.0/24) &
3. Install Prometheus
Prometheus and Grafana can be run anywhere on the same network. I’d recommend not running them on the same device doing the metric exporting so that you’re not slowing down the machine. The following instructions are copied more or less exactly from the official Prometheus docs:
Download the latest release of Prometheus for your platform, then extract and run it:
tar xvfz prometheus-*.tar.gz cd prometheus-*
global: scrape_interval: 15s external_labels: monitor: 'network-traffic-metrics' scrape_configs: - job_name: 'network-traffic-metrics' static_configs: - targets: ['192.168.0.2:8000']
Check that you can access Prometheus: localhost:9090/metrics
4. Install Grafana
Again, the official docs are a good place to start (these are copied fairly directly):
sudo apt-get install -y apt-transport-https sudo apt-get install -y software-properties-common wget wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add - sudo apt-get update sudo apt-get install grafana-enterprise
http://localhost:3000/ to find Grafana. Add Prometheus as a data source:
- Open the side menu by clicking the Grafana icon in the top header.
- In the side menu under the
Dashboardslink you should find a link named
- Click the
+ Add data sourcebutton in the top header.
Prometheusfrom the Type dropdown.
The default options should match your installation from above.
Add Dashboard from the home screen. Use the gear in the upper-right to open the settings and choose the JSON model. Paste
grafana.json from the github repository in step (2).