WWVB Glitchiness Histogram in 3D

The character based Glitchiness histograms described there work pretty well for short time scales, but more than a screen full is too much. It turns out that Gnuplot can chew up the histograms and spit out a perfectly serviceable 3D map plot.

The trick is to extract the histogram characters into a file, then persuade Gnuplot to regard the file as a binary array, with the ASCII character values giving the Z height of the dot for each XY cell.

Click for bigger picture:

Gnuplot Glitchiness

Gnuplot Glitchiness

The axes:

  • Front edge = 51 pulse durations, 0 – 1 second, 20 ms resolution
  • Right edge = 1363 histograms = 22.7 hours of WWVB reception
  • Z axis = histogram counts

The flat plane has the vast majority of points having zero (or just a few) counts.

The three front-to-back hillocks show the durations of the binary-zero, binary-one, and frame markers within each second; the resolution is 20 ms per sample perpendicular to those lines.

The fuzzy mountain peaks along the left edge represent intense noise; you’re looking for the very few intervals of zero noise when the WWVB signal is readable. Those would be flat lines from the left to right edges, with just three bumps at the proper durations.

The valley between the mountain peaks is the nighttime reception, when the noise drops to bearable intensity and RF propagation brings in enough WWVB signal to make a difference. The fact that you can see the proper pulse widths through much of the day suggests the signal is in there, but it’s so noisy you (well, I) can’t make make much use of it.

How to get the graph…

The clock produces three lines of output every minute that look like this:

UTC: 10 013 16:36:00.0 Loc=11 Age=367   LY=0 LS=0 DST=0 Chg=0 UT1=1 Mon=1 DOM=13
Glitchiness:  268 Histogram: W!ieTHG3A35412132.11...............................
Light: 02CA Min=0005 Max=038B

Extract just the lines with histograms:

grep Histo 2010-01-12\ LR\ Window\ 80\ cm\ V\ on\ shelf\ -\ shield\ box.log > 1.txt

Chop out the histogram data, which has a leading space:

cut -d ':' -f 3 1.txt > 2.txt

Discard the leading space and put the histogram text in the final file:

cut -d ' ' -f 2 2.txt > histo.txt

The last few lines of that file look like this:


You could do that all in one gargantuan Bash line, piping the results from one filter to the next, but that’s hard to explain.

Now, fire up Gnuplot and have at it:

set xyplane at 0
set zrange [0:128]
splot 'histo.txt' binary format="%uint8" record=52x1363 using 1 with points lt 3 pt 0

The doc suggests record=52xInf should work, but that draws a useless picture. If the record value is bigger than the number of actual records (found with wc -l histo.txt, the plot ends at the end of file; if it’s smaller, then you get only that many records. I suppose you could just use 99999; it’d work well enough.

The 52 comes from the number of characters in the line: 51 histogram bytes per line, plus a newline character at the end. The newline produces the distinct line below everything else along the right edge of the plot. You could get rid of the newline characters and turn it into a binary file before plotting, but that’s sort of cheating, I think.

You’ll recall the counting sequence in each histogram character:

  • “.” = 0
  • 1 through 9 = obvious
  • A through Z = 10 – 35
  • a through z = 36 – 61
  • ! = more than 61

Unfortunately, the “!” has a lower ASCII value than the other characters, so those are the dots below the plane on the left side; they should be along the top surface. I’ll change that to “|” and make the answer come out right.

From here on, it’s a matter of the usual Gnuplot futzing to get a decent-looking plot.

Rotating the view may be useful. For example, set view 60,80 produces this:

Gnuplot Glitchiness - rotated

Gnuplot Glitchiness - rotated

Now you’re looking more-or-less parallel to the samples for each minute. If you twiddled with the ranges, you could probably see the few valleys where it’d be possible to extract a valid time code.

The alert reader will note that I used record=52×4344 to generate those plots. Homework: why?

  1. #1 by randomdreams on 2010-01-28 - 12:06

    I’m guessing 52 because it’s 51 data points plus an end-of-line, since you’re interpreting characters in a file? I haven’t played with ‘record’; my plots tend to be like ‘splot “data” using 1:4:5 with line’. Gnuplot is an amazing piece of work. I’ve used it to allow us to do graphical analysis of conversion efficiency for buck/boost controllers. I wrote a bit of gpib that runs the buck/boost and records 100,000 or so datapoints and dump that into gnuplot, and we can see all sorts of strange details that reflect the poles of the output filter and weirdness in the controller function.

    That’s a sweet plot, and a clever idea to directly plot the text.

    • #2 by Ed on 2010-01-28 - 12:45

      51 data points plus an end-of-line

      Ah, but where did the 4344 come from?

      Truly embarassing: I counted the lines in the original file from the clock, with all the other junk surrounding the Histogram lines: it’s a bit more than a factor of three too high. That’s when I discovered that Inf doesn’t work and you don’t have to be spot-on accurate. A silver lining if I ever saw one.

      we can see all sorts of strange details

      This started as a sort of “Huh. I wonder what would that look like?” experiment and … it worked!

      More as I get more data…

      • #3 by randomdreams on 2010-01-28 - 15:02

        Oh, geez, again with the reading comprehension problems: you *said* it was 51 + EOF, and I missed it. Gah. Some day I’ll start reading for content again, I promise.
        Hm. Is it a side-effect of reading a binary matrix and having to cast each chunk into a 32 bit format?

        • #4 by Ed on 2010-01-28 - 20:38

          Nope, it was a pure & simple blunder… worked perfectly, despite my error, though.

          I like finding problems where I just screwed up, though, because they’re easy to fix. Eerrors with subtle causes and elaborate chains of happenstance really annoy me…