Monthly Science: Patio Ground Temperature

The patio at the rear of our house is a six inch concrete slab with a five foot “crawlspace” below it, with a HOBO datalogger pendant in the dirt near the pole at the middle of the I beam supporting the slab. The pendant is very close to the surface, so there’s a big diurnal temperature variation, but it still gives a reasonable picture of seasonal change.

A configuration setting in Hoboware determines whether it stores / exports dates with years having two digits or four digits. As you might expect, over the course of five years the dates have both formats, but there’s always a blank separating the date and the time:

grep "/12 " /tmp/data.csv | head -1
1,11/09/12 08:45:00 ,49.050,,,,
grep "/2012 " /tmp/data.csv | head -1
2235,01/01/2012 00:00:00,44.560,0.0,,,,

This burst of sed regex line noise normalizes all years to four digits:

sed 's/\/\([0-9][0-9]\) /\/20\1 /' whatever.csv

The parenthesized subexpression matches the digits of a two digit year preceding a blank, then the \1 plugs it into the right spot in the output. This suffers from the usual failure when the century rolls over, but frankly, my dear readers, I don’t give a damn. The backslashes escape forward slashes and parentheses, in addition to making the regex pretty much write-only.

The plot shows the expected annual variation:

Under patio - Ground - Center

Under patio – Ground – Center

The periodic upward spikes happen when I carry the logger to the Token Windows Laptop and read it out; the air temperature upstairs is always warmer than the dirt under the patio.

The Bash and gnuplot script that produced the graph:

#!/bin/sh
#-- overhead
export GDFONTPATH="/usr/share/fonts/truetype/"
base="${1%.*}"
echo Base name: ${base}
ofile=${base}.png
tfile=$(tempfile)
echo Input file: $1
echo Temporary file: ${tfile}
echo Output file: ${ofile}
#-- prepare csv Hobo logger file
sed 's/^\"/#&/' "$1" | sed 's/^.*Logged/#&/' | sed 's/ ,/,/' | sed 's/\/\([0-9][0-9]\) /\/20\1 /' > ${tfile}
#-- do it
gnuplot << EOF
#set term x11
set term png font "arialbd.ttf" 18 size 950,600
set output "${ofile}"
set title "${base}"
set key noautotitles
unset mouse
set bmargin 4
set grid xtics ytics
set timefmt "%m/%d/%Y %H:%M:%S"
set xdata time
set xlabel "Date"
set format x "%Y-%m"
#set xrange [1.8:2.2]
set xtics font "arial,12"
#set mxtics 2
#set logscale y
#set ytics nomirror autofreq
set ylabel "Temperature - F"
#set format y "%4.0f"
#set yrange [30:90]
#set mytics 2
#set y2label "right side variable"
#set y2tics nomirror autofreq 2
#set format y2 "%3.0f"
#set y2range [0:200]
#set y2tics 32
#set rmargin 9
set datafile separator ","
#set label 1 "label text" at 2.100,110 right font "arialbd,18"
#set arrow from 2.100,110 to 2.105,103 lt 1 lw 2 lc 0
plot	\
    "${tfile}" using 2:3 with lines lt 3
EOF
About these ads
  1. #1 by smellsofbikes on 1-March-2013 - 15:25

    That sed has broken my head.

    • #2 by Ed on 1-March-2013 - 16:34

      Believe: I did not write that in one pass!

  2. #3 by Jin Choi on 14-February-2014 - 03:19

    The use of slash for s/// is purely traditional. It can be any character you want, and if you don’t use slash you don’t have to escape them. http://www.grymoire.com/Unix/Sed.html#uh-2

    • #4 by Ed on 14-February-2014 - 12:05

      I recently saw a sed example with semicolons as delimiters. It works, but my first reaction was “Wait. What?”

      Thanks for the reminder…

      • #5 by david on 14-February-2014 - 12:18

        If you really want to confuse people, use ‘s’ as your delimiter…

        • #6 by Ed on 14-February-2014 - 15:17

          If you ever do that, I will hunt you down and hurt you badly by forcing you to sight-read a few hundred lines from cpan

  3. #7 by david on 14-February-2014 - 21:16

    So I guess using backspaces as the delimiter is Right Out, then? :)

    • #8 by Ed on 15-February-2014 - 08:49

      Depends on your ability to properly escape them and withstand aesthetic pain…

      That also applies if you use a blank space.

      • #9 by solaandjin on 15-February-2014 - 10:10

        So I had the same sort of thoughts. What is this most interesting character you could use for the s/// delimiter?

        Sadly, it appears to be limited to a single byte even when the LANG environment variable is set to be something with Unicode. So, no using the code point for an emoji strawberry, then: s🍓happy🍓sad🍓. (That will probably only show up for people running OS X…)

        ASCII BEL (07) works just fine, though, as does NP (0C). You can type:

        echo “sed ‘s^Ga^Gb^G'” > test

        and then cat test. Or replace the ^G with a ^L for an interesting visual result. (Use ctrl-v ctrl-g or ctrl-v ctrl-l to insert the control characters in a terminal, or use your editor of choice.)

  1. Monthly Science: Air Temperatures | The Smell of Molten Projects in the Morning
  2. Monthly Science: Town Water Inlet Temperature | The Smell of Molten Projects in the Morning