The Smell of Molten Projects in the Morning

Ed Nisley's Blog: Shop notes, electronics, firmware, machinery, 3D printing, laser cuttery, and curiosities. Contents: 100% human thinking, 0% AI slop.

Category: Software

General-purpose computers doing something specific

  • Choosing SSID Names

    It turns out that WiFi SSIDs can be quite long and contain blanks:

    SSID Names
    SSID Names

    Now, if I wanted to capture your private bits, I’d put up a public access point with a friendly name and no security at all…

  • Upstart vs. rc.local

    In the process of figuring out how to set up the isolated WiFi Internet link on the file server, I discovered that the /etc/rc.local file runs before the eth0 interface that connects to the outside world comes up. As a result, my DynDNS host address hadn’t been updated in quite some time.

    Worse, trying to set up eth1 failed, apparently because there’s a bunch of other network infrastructure that doesn’t start until eth0 comes online. Part of that infrastructure involves iptables; the added rules simply vanished.

    The solution seems to require writing an upstart script that waits for whatever events it needs, does what needs to be done, and then goes away. The whole upstart mechanism and its event list seems, um, lightly documented, as I discovered there, but the custom setup formerly in /etc/rc.local now lives in /etc/init/local.conf:

    description "Stuff that used to be in /etc/rc.local"
    author "Ed Nisley - KE4ZNU"
    
    start on (local-filesystems and net-device-up IFACE=eth0)
    stop on shutdown
    
    script
    
    logger Starting local init...
    
    logger Updating dyndns
    ddclient -force
    
    logger Bringing up eth1
    ifconfig eth1 192.168.3.1 netmask 255.255.255.0 up
    
    logger Setting iptables
    iptables -A FORWARD -i eth1 --destination 192.168.0.0/16 -j REJECT
    iptables -A INPUT -i eth1 --destination 192.168.0.0/16 -j REJECT
    iptables -A POSTROUTING -t nat -j MASQUERADE
    
    logger Ending local init
    
    end script
    

    That code assumes the outbound network interface will be eth0, which won’t work on a system using a pure wireless connection on, say, wlan0 or anything more exotic. I haven’t a clue how to parameterize that selection. Most likely, one would write another upstart script that would emit a custom signal based on the usual suspect …

    It also assumes the networking infrastructure triggered by eth0 lighting up has hauled itself to its feet and is ready to roll. That seems to be true, although I’ll admit the script is, at best, lightly tested.

    With the eth1 NIC up and iptables rules added, I think this script will restart eth1 when it goes down, but it’s not clear where the requisite network-device-down event comes from (certainly not from any script in /etc/init/*conf):

    description "Restart eth1 when it dies"
    author "Ed Nisley - KE4ZNU"
    
    start on net-device-down IFACE=eth1
    stop on net-device-up IFACE=eth1
    
    script
    
    logger Restarting eth1...
    ifconfig eth1 192.168.3.1 netmask 255.255.255.0 up
    
    logger Ending eth1 setup
    
    end script
    

    But, eh, at least the isolated interface comes up and packets go where they should (and not where they shouldn’t). Early results are encouraging…

  • Isolated Internet Access for Guests

    We provide a camping spot for touring bicyclists riding through the Hudson Valley and, as you’d expect, most of them arrive toting netbooks, tablets, and other net-enabled doodads. While I’m a nice guy and they’re uniformly nice folks, I’d rather not hand them the keys to our house network, so I recently set up a WiFi Internet-only access point that’s firewalled from the LAN.

    The general idea:

    • Use a stock WiFi router to handle DHCP / DNS / WiFi for guests (192.168.2.x)
    • Add a second NIC to the file server as eth1 (192.168.3.1), connected to the router’s WAN port (192.168.3.2)
    • Forward packets between eth0 (house network 192.168.1.x) and eth1, except …
    • Use iptables to prevent router clients from seeing the house network

    The network layout:

    Guest Internet Access Overview
    Guest Internet Access Overview

    The parts came from the Big Box o’ Network Stuff:

    • Linksys / Cisco WRT54G router (Version 8, so OpenWRT won’t run)
    • NetGear 10/100 Mb/s Ethernet PCI card

    The router setup:

    • Static WAN at 192.168.3.2
    • Router base address 192.168.2.1
    • DHCP range 192.168.2.100 through .149, lease time 1 hour
    • DNS entries 4.2.2.1 (L3), 65.88.88.2 (NY Public Library), 129.250.35.250 (NTT)
    • WiFi access to the web admin page disabled (admin only via CAT5 in the Basement Laboratory)
    • Non-broadcast SSID, not that it matters very much
    • WPA2-PSK with an XKCD-style password

    The NIC Just Worked: the drivers come along with the kernel. Because it’s not a general-purpose network interface from the server side, eth1 setup doesn’t require much effort:

    ifconfig eth1 192.168.3.1 netmask 255.255.255.0
    

    I discovered the hard way that trying to define the eth1 interface with Network Manager caused no end of heartache & confusion, not least of which is that having two NICs somehow activates Ubuntu’s internal firewalling & port forwarding. Suffice it to say, just set the NM’s GUI to Ignore the eth1 NIC and do what needs to be done manually.

    With one NIC, Ubuntu runs iptables in “let it be” mode: everything’s allowed, nothing’s blocked, and all packets get forwarded. The tables are empty and the default ACCEPT policy passes everything.

    Adding a rule to the FORWARD chain prevents the router from sending packets to the house network:

    iptables -A FORWARD -i eth1 --destination 192.168.0.0/16 -j REJECT
    

    That still allows a ping response from the file server’s eth0 NIC at 192.168.1.2 back to the WiFi clients, because packets addressed to the server pass through the INPUT chain. This rule squelches those packets:

    iptables -A INPUT -i eth1 --destination 192.168.0.0/16 -j REJECT
    

    Although packet forwarding is enabled by default, another rule turns on the NAT machinery required to shuttle packets between the 192.168.3.x network and the outside world:

    iptables -A POSTROUTING -t nat -j MASQUERADE
    

    While fiddling with iptables rules that involve packet state tracking (which these do, at least implicitly, I think), you must reset the packet state memories to ensure new packets aren’t regarded as part of an established connection. Install the conntrack utilities, then reset the state as needed:

    sudo conntrack -F
    

    And then it Just Worked.

    Now, back in the day, you’d just put those configuration lines in /etc/rc.local and be done with it. Unfortunately, nowadays the upstart process kicks off rc.local well before the system is in a usable state: somewhat before eth0 is active, which means any automagic network-related activity falls flat on its face.

    So an upstart configuration script is in order… more on that later.

    Some useful, albeit occasionally befuddling references:

    One could, of course, buy dedicated hardware to do all that and more, but it’s nothing you couldn’t accomplish with a bit more configuration on a stock Linux box. Heck, you could even serve an Upside-Down-Ternet to anyone who deserves it; the original has some other suggestions that made the big time.

    A tip o’ the cycling helmet to Dragorn of Kismet for getting me started…

  • Macro Lens Focus Stacking

    Begin by mounting the Canon SX230HS on the macro lens adapter, zooming to about the maximum, fiddling with a ruler to put the end at the closest focus point, and eventually get an overall view like this:

    Ruler - macro mid-focus
    Ruler – macro mid-focus

    The images below were batch cropped from similar views with ImageMagick:

    for f in $(seq 17 22) ; do convert -crop '1500x1126+1900+1800' \
       img_18${f}.jpg img_18${f}-crop.jpg ; done
    

    Yes, I’ve taken a bit over 1800 images since getting that camera… the old DSC-F505V recently rolled over at 10K images.

    Take a set of six identically exposed pictures starting with the focus at infinity (about 95 mm in real life):

    macro far focus
    macro far focus

    And ending with the closest focus at about 1 meter for this zoom setting (and 80 mm in real life):

    macro near focus
    macro near focus

    Then apply enfuse (from the Ubuntu repositories) with a handful of parameters suggested there that combine the sharpest parts of each image into a single image:

    enfuse --verbose --exposure-weight=0 --saturation-weight=0 \
       --contrast-weight=1 --hard-mask --output=stacked.jpg \
       img_18??-crop.jpg
    

    Which produces this nice result:

    Ruler - macro combined focus
    Ruler – macro combined focus

    It’s not perfect, it needs a few more intermediate images, there’s fringing around high-contrast edges, and so forth and so on, but for a first pass it ain’t bad at all.

    I bar-clamped the camera & macro adapter to the desk in order to eliminate all motion. My usual tripod mount for the macro setup isn’t all that stable and the microscope stand isn’t particularly rigid, either, so I must improve a bunch of mechanical structures. In principle, you can post-process the pictures to realign them, although the tolerances seem daunting enough to make mechanical fixturing look downright attractive by comparison.

    Now, if it should turn out that the SX230HS supports the CHDK USB remote trigger, that’d be nice. Or maybe the right way to proceed involves converting the problem to A Simple Matter of Software by writing a CHDK script that tweaks the focus by multiples rather than increments.

  • Canon Hack Development Kit for SX230HS

    Although the macro lens adapter and microscope mount work well enough, the relatively small sensor and lens in my Canon SX230HS make for a razor-thin depth of field:

    Macro lens depth of field
    Macro lens depth of field

    Those are, of course, millimeter divisions on the ruler.

    A bit of rummaging leads to the notion of Focus Stacking, which involves taking a sequence of images with identical exposure settings and different focus points, then compositing the in-focus parts of each image to produce a single image with everything in focus. Although some examples show a manual process involving layers in, say, The GIMP or Photoshop, I think an automated process would be better.

    Given that I have a Canon SX230HS camera, the first step is to download the proper version of the Canon Hack Development Kit, unpack it onto a spare SD card, and get used to it.

    As it turns out, the focus bracketing works exactly as intended, but doesn’t do quite what I need: it changes the focus in linear steps by adding a constant bracketing distance. The macro lens adapter drags the “infinity” focus point inward to maybe 15 mm beyond the innermost focus point, but the camera’s focus range still shows 1 m to ∞. Stepping in 1 m increments generates a bazillion pictures that don’t differ by much at all after 5 m, but you still need a few near the far end.

    However, it seems the only way to get a bazillion pictures is by holding the shutter button down with the drive mode set to Continuous, as the camera’s Custom Timer mode has a 10 shot upper limit. If I must do that, I may as well adjust the focus manually: the assumption being that the camera shall be firmly mounted to keep the pix in alignment, which currently isn’t true in any of my setups and certainly won’t be true with my finger on the button.

    The camera already has exposure bracketing, although not to the extreme range available through CHDK. RAW images (or the roughly equivalent DNG format) might come in handy at some point, but right now they’re just a temping digression available only through CHDK.

    If I’m going to keep using CHDK, I must conjure up an artificial NB-5L battery with an external power source. Those cheap eBay batteries work fine for the usual duty cycle, but constant zooming & focusing & suchlike chew them pretty hard…

  • Dropbox vs. Little Bobby Tables

    The Android version of the Dropbox interface (on the Kindle, anyway) lets you create a password like this:

    ab&CDef{gHi

    Come to find out that, although the web-based Dropbox interface doesn’t reject that password, it kvetches that your userid and password don’t match. Yes, even if you cut-and-paste from a text file copied through the USB link.

    Fortunately, the web interface has a password reset mechanism that’s missing from the Kindle app.

    Little Bobby Tables rides again!

  • Kindle Fire: Palm Gnukeyring to KeePass

    My Zire 71 stores all my userids, passwords, credit card numbers, and similar sensitive bits in a convenient offline package, using the obsolete Palm gnukeyring app. The KeePass app on my Kindle Fire can, in principle, do the same thing. Minus the convenient and offline parts, that is.

    So the problem becomes how to export about 150 entries, each containing at least one character string carefully chosen for maximum obscurity and typing difficulty, from the Zire to the Kindle. Ideally, there’s no retyping involved and it’d be nice to not leave the unencrypted contents lying around for very long.

    This has a remarkable number of moving parts…

    It turns out gnukeyring can export each entry into an unencrypted Palm Notepad file through a manual operation, but the cough user experience on the Palm goes something like this:

    • Start gnukeyring, select entry-to-export
    • Select Options, pick export to Notepad
    • Notepad automagically starts up with the new note selected, so close it
    • Exit Notepad, return to Home
    • Iterate

    That gets old very quickly, not to mention that the Notepad file uploads to the PC in the Palm’s slightly binary PDB format that’s basically useless without further hackage and exposes the passwords in unencrypted form.

    Inter-app cutting and pasting on the Kindle is exceedingly tedious, because each app runs more-or-less independently in full-screen mode: it is not feasible to manually transfer the database one field at a time. So we’re looking at a PC-based solution.

    There’s a Java utility that can export the whole encrypted Palm file into spreadsheet CSV format, putting each entry into a separate row and each field into a separate column. OpenOffice / LibreOffice can read the resulting file with no problem. Ctrl-A will select the entire Comment field for each entry, which means I can transfer that in one shot.

    It can also export into an XML file, which seems more useful.

    The utility is a command line thing, with an invocation something like this (use .csv to get that type of output):

    java -jar export.jar Keys-Gtkr.pdb ‘password’ output.xml

    As it turns out, my password contains an exclamation mark and Bash requires fancy escaping (the usual single or double quotes don’t work, because the exclamation mark has higher priority):

    java -jar export.jar Keys-Gtkr.pdb Fancy\!PassWord\! output.xml

    The XML files have this overall structure:

    <U+FEFF><?xml version="1.0" encoding="UTF-8"?>
    <pwlist>
    <pwentry>
      <title>A useful name</title>
      <category>Web</category>
      <notes>The collection of
    notes, asides, and suchlike
    in a multi-line format.
    </notes>
      <lastmodtime>2000-01-01</lastmodtime>
    </pwentry>
    </pwlist>
    

    Notice that each entry contains its category, which makes a certain sense.

    On the KeePass side, there exist Windows and Linux versions of KeePass, although one must be careful to use the 1.x level of the database on Windows, because that’s all the Android and Linux versions know about. It can import XML files and turn them into a database with a unique password; it cannot add entries from an XML file to an existing database.

    KeePassX (the Linux version) XML files looks like this:

    <!DOCTYPE KEEPASSX_DATABASE>
    <database>
     <group>
      <title>Internet</title>
      <icon>1</icon>
      <entry>
       <title>The Entry Title</title>
       <username>your userid goes here</username>
       <password>the mystery password</password>
       <url>example.com</url>
       <comment>Presumably multiline comments work OK</comment>
       <icon>0</icon>
       <creation>2012-05-08T18:48:11</creation>
       <lastaccess>2012-05-08T19:00:49</lastaccess>
       <lastmod>2012-05-08T19:00:49</lastmod>
       <expire>2999-12-28T11:59:59</expire>
      </entry>
     </group>
    </database>
    

    KeePass puts the entries within overall groups, the inverse of the Palm gnukeyring structure. I suppose there’s a way to undo that manually, but … drat!

    So, back to CSV. Although OpenOffice Calc can’t export XML directly, there’s an extension for that. Given a spreadsheet like this:

    asdf dfg
    1 2
    3 4
    5 6

    The XML file looks like this:

    <?xml version="1.0" encoding="UTF-8"?>
    <ooo_calc_export scriptVersion="2.2.0" scriptUpdate="2010-12-19"
                     scriptURL="http://www.digitalimprint.com/misc/oooexport/"
                     scriptAuthor="vjl">
       <ooo_sheet num="1" name="Sheet1">
          <ooo_row><asdf>1</asdf><dfg>2</dfg></ooo_row>
          <ooo_row><asdf>3</asdf><dfg>4</dfg></ooo_row>
          <ooo_row><asdf>5</asdf><dfg>6</dfg></ooo_row>
       </ooo_sheet>
    </ooo_calc_export>
    

    Missing entries generate empty <column-name></column-name> sequences and embedded newlines produce multiline comments.

    So the overall plan:

    • Convert Palm database into CSV
    • Import into OpenOffice Calc
    • Change column names to match KeePass fields
    • Add creation field with current timestamp
    • Add expire field to celebrate the next millennium
    • Sort by Category, delete Category column
    • Export to XML, open in text editor
    • Clean up header & footer tags to suit
    • Insert group tags for category
    • Change ooo_row tags to entry tags
    • Import as new KeePass database
    • Transfer to Kindle Fire
    • Iterate until it works

    While I had the file pinned down in the text editor, I cleaned up some cruft and moved userids / passwords from comments to the appropriate fields.

    I hadn’t used the gnukeyring password field to avoid overwriting an existing password by mistake; that disaster always lay just two screen taps away. It’s a little harder to do on the Kindle, but …

    And then it Just Worked…