Posts from — July 2013

Recently, I’ve been setting up a home sensor array using Arduino modules, which feed data into a Graphite instance running on a Raspberry Pi.  Along the way, through faulty wiring, faulty coding, or the effects of sunspots, I’ve ended up with some bad data ending up in Graphite.

And by bad data, I mean that my living room sensor once reported that it was 289,268 degrees C one day.  Since my house and the surrounding neighborhood show no signs of having been vaporized, I am forced to believe that reading is incorrect.

Since having a temperature reading of several hundred thousand degrees throws off the range of the graph somewhat, I wanted to get rid of that errant data point.  I looked around for a Whisper DB editor, but didn’t find anything.  Graphite’s all open source, so I probably could build one myself, but I really didn’t feel like going that far.  Then I remembered the “Feeding in Your Data” page in the Graphite docs, where it talks about using the command line to send data values.  Maybe if you can write datapoints using this method, you can overwrite datapoints using this method, too.  So, I figured I’d give it a shot.

The example they gave on the page is this:

echo "local.random.diceroll 4 `date +%s`" | nc ${SERVER} ${PORT};

The “local.random.diceroll” part is your counter name.

The “4” part is the data you want to write.

And the “date +%s” is just a fancy way of saying the Unix Epoch Timestamp of Now.

So, in my case, I’d overwrite the counter “stats.gauges.Temperature.LivingRoom” with a comfortable room temparature value of somewhere between 20 and 25.  But what about the timestamp?  I can’t just go in and hope I get the timestamp.  I can guess from that graph that it happened sometime around 00:32:30, but who knows if I’m right.  And what timezone is Whisper using?  Local time?  UTC?  And if it’s UTC, is that + 8 hours or +7 hours this time of year…?  I needed a better way to get the exact timestamp.

Fortunately, Graphite makes that relatively easy.  You see, you can get the data in json format if you want.  While it’s not as pretty as the graph, it does have the exact values of the timestamp you’re looking for.  Getting the json is fairly straightforward.  First, you go to the Graphite Browser and find the data you want to fix.  Once you have it in the window, right click and open the graph in a new tab (Or copy/paste the graph URL, either way works).  From there, edit the URL of the graph, by adding “&format=json” to the end.  Load that page and your screen will fill with json datapoints.  Like so:

In that sea of numbers, find the value you want to replace.  In my case, it’s easy to find, because it’s the one claiming that my living room jumped up to 6 times hotter than the surface of the sun for a moment.  The second value in that datapoint is the timestamp I need to wipe out the errant temperature spike.  Now that I have it, I can run the following command to put things right:

echo "stats.gauges.Temperature.LivingRoom 22.8 1373614500" | nc localhost 2003

And now my living room is back to a comfortable 22.8 degrees.