The Smell of Molten Projects in the Morning

Ed Nisley's Blog: Shop notes, electronics, firmware, machinery, 3D printing, laser cuttery, and curiosities. Contents: 100% human thinking, 0% AI slop.

Category: Software

General-purpose computers doing something specific

  • Isolated Internet Access for Guests

    We provide a camping spot for touring bicyclists riding through the Hudson Valley and, as you’d expect, most of them arrive toting netbooks, tablets, and other net-enabled doodads. While I’m a nice guy and they’re uniformly nice folks, I’d rather not hand them the keys to our house network, so I recently set up a WiFi Internet-only access point that’s firewalled from the LAN.

    The general idea:

    • Use a stock WiFi router to handle DHCP / DNS / WiFi for guests (192.168.2.x)
    • Add a second NIC to the file server as eth1 (192.168.3.1), connected to the router’s WAN port (192.168.3.2)
    • Forward packets between eth0 (house network 192.168.1.x) and eth1, except …
    • Use iptables to prevent router clients from seeing the house network

    The network layout:

    Guest Internet Access Overview
    Guest Internet Access Overview

    The parts came from the Big Box o’ Network Stuff:

    • Linksys / Cisco WRT54G router (Version 8, so OpenWRT won’t run)
    • NetGear 10/100 Mb/s Ethernet PCI card

    The router setup:

    • Static WAN at 192.168.3.2
    • Router base address 192.168.2.1
    • DHCP range 192.168.2.100 through .149, lease time 1 hour
    • DNS entries 4.2.2.1 (L3), 65.88.88.2 (NY Public Library), 129.250.35.250 (NTT)
    • WiFi access to the web admin page disabled (admin only via CAT5 in the Basement Laboratory)
    • Non-broadcast SSID, not that it matters very much
    • WPA2-PSK with an XKCD-style password

    The NIC Just Worked: the drivers come along with the kernel. Because it’s not a general-purpose network interface from the server side, eth1 setup doesn’t require much effort:

    ifconfig eth1 192.168.3.1 netmask 255.255.255.0
    

    I discovered the hard way that trying to define the eth1 interface with Network Manager caused no end of heartache & confusion, not least of which is that having two NICs somehow activates Ubuntu’s internal firewalling & port forwarding. Suffice it to say, just set the NM’s GUI to Ignore the eth1 NIC and do what needs to be done manually.

    With one NIC, Ubuntu runs iptables in “let it be” mode: everything’s allowed, nothing’s blocked, and all packets get forwarded. The tables are empty and the default ACCEPT policy passes everything.

    Adding a rule to the FORWARD chain prevents the router from sending packets to the house network:

    iptables -A FORWARD -i eth1 --destination 192.168.0.0/16 -j REJECT
    

    That still allows a ping response from the file server’s eth0 NIC at 192.168.1.2 back to the WiFi clients, because packets addressed to the server pass through the INPUT chain. This rule squelches those packets:

    iptables -A INPUT -i eth1 --destination 192.168.0.0/16 -j REJECT
    

    Although packet forwarding is enabled by default, another rule turns on the NAT machinery required to shuttle packets between the 192.168.3.x network and the outside world:

    iptables -A POSTROUTING -t nat -j MASQUERADE
    

    While fiddling with iptables rules that involve packet state tracking (which these do, at least implicitly, I think), you must reset the packet state memories to ensure new packets aren’t regarded as part of an established connection. Install the conntrack utilities, then reset the state as needed:

    sudo conntrack -F
    

    And then it Just Worked.

    Now, back in the day, you’d just put those configuration lines in /etc/rc.local and be done with it. Unfortunately, nowadays the upstart process kicks off rc.local well before the system is in a usable state: somewhat before eth0 is active, which means any automagic network-related activity falls flat on its face.

    So an upstart configuration script is in order… more on that later.

    Some useful, albeit occasionally befuddling references:

    One could, of course, buy dedicated hardware to do all that and more, but it’s nothing you couldn’t accomplish with a bit more configuration on a stock Linux box. Heck, you could even serve an Upside-Down-Ternet to anyone who deserves it; the original has some other suggestions that made the big time.

    A tip o’ the cycling helmet to Dragorn of Kismet for getting me started…

  • Macro Lens Focus Stacking

    Begin by mounting the Canon SX230HS on the macro lens adapter, zooming to about the maximum, fiddling with a ruler to put the end at the closest focus point, and eventually get an overall view like this:

    Ruler - macro mid-focus
    Ruler – macro mid-focus

    The images below were batch cropped from similar views with ImageMagick:

    for f in $(seq 17 22) ; do convert -crop '1500x1126+1900+1800' \
       img_18${f}.jpg img_18${f}-crop.jpg ; done
    

    Yes, I’ve taken a bit over 1800 images since getting that camera… the old DSC-F505V recently rolled over at 10K images.

    Take a set of six identically exposed pictures starting with the focus at infinity (about 95 mm in real life):

    macro far focus
    macro far focus

    And ending with the closest focus at about 1 meter for this zoom setting (and 80 mm in real life):

    macro near focus
    macro near focus

    Then apply enfuse (from the Ubuntu repositories) with a handful of parameters suggested there that combine the sharpest parts of each image into a single image:

    enfuse --verbose --exposure-weight=0 --saturation-weight=0 \
       --contrast-weight=1 --hard-mask --output=stacked.jpg \
       img_18??-crop.jpg
    

    Which produces this nice result:

    Ruler - macro combined focus
    Ruler – macro combined focus

    It’s not perfect, it needs a few more intermediate images, there’s fringing around high-contrast edges, and so forth and so on, but for a first pass it ain’t bad at all.

    I bar-clamped the camera & macro adapter to the desk in order to eliminate all motion. My usual tripod mount for the macro setup isn’t all that stable and the microscope stand isn’t particularly rigid, either, so I must improve a bunch of mechanical structures. In principle, you can post-process the pictures to realign them, although the tolerances seem daunting enough to make mechanical fixturing look downright attractive by comparison.

    Now, if it should turn out that the SX230HS supports the CHDK USB remote trigger, that’d be nice. Or maybe the right way to proceed involves converting the problem to A Simple Matter of Software by writing a CHDK script that tweaks the focus by multiples rather than increments.

  • Canon Hack Development Kit for SX230HS

    Although the macro lens adapter and microscope mount work well enough, the relatively small sensor and lens in my Canon SX230HS make for a razor-thin depth of field:

    Macro lens depth of field
    Macro lens depth of field

    Those are, of course, millimeter divisions on the ruler.

    A bit of rummaging leads to the notion of Focus Stacking, which involves taking a sequence of images with identical exposure settings and different focus points, then compositing the in-focus parts of each image to produce a single image with everything in focus. Although some examples show a manual process involving layers in, say, The GIMP or Photoshop, I think an automated process would be better.

    Given that I have a Canon SX230HS camera, the first step is to download the proper version of the Canon Hack Development Kit, unpack it onto a spare SD card, and get used to it.

    As it turns out, the focus bracketing works exactly as intended, but doesn’t do quite what I need: it changes the focus in linear steps by adding a constant bracketing distance. The macro lens adapter drags the “infinity” focus point inward to maybe 15 mm beyond the innermost focus point, but the camera’s focus range still shows 1 m to ∞. Stepping in 1 m increments generates a bazillion pictures that don’t differ by much at all after 5 m, but you still need a few near the far end.

    However, it seems the only way to get a bazillion pictures is by holding the shutter button down with the drive mode set to Continuous, as the camera’s Custom Timer mode has a 10 shot upper limit. If I must do that, I may as well adjust the focus manually: the assumption being that the camera shall be firmly mounted to keep the pix in alignment, which currently isn’t true in any of my setups and certainly won’t be true with my finger on the button.

    The camera already has exposure bracketing, although not to the extreme range available through CHDK. RAW images (or the roughly equivalent DNG format) might come in handy at some point, but right now they’re just a temping digression available only through CHDK.

    If I’m going to keep using CHDK, I must conjure up an artificial NB-5L battery with an external power source. Those cheap eBay batteries work fine for the usual duty cycle, but constant zooming & focusing & suchlike chew them pretty hard…

  • Dropbox vs. Little Bobby Tables

    The Android version of the Dropbox interface (on the Kindle, anyway) lets you create a password like this:

    ab&CDef{gHi

    Come to find out that, although the web-based Dropbox interface doesn’t reject that password, it kvetches that your userid and password don’t match. Yes, even if you cut-and-paste from a text file copied through the USB link.

    Fortunately, the web interface has a password reset mechanism that’s missing from the Kindle app.

    Little Bobby Tables rides again!

  • Kindle Fire: Palm Gnukeyring to KeePass

    My Zire 71 stores all my userids, passwords, credit card numbers, and similar sensitive bits in a convenient offline package, using the obsolete Palm gnukeyring app. The KeePass app on my Kindle Fire can, in principle, do the same thing. Minus the convenient and offline parts, that is.

    So the problem becomes how to export about 150 entries, each containing at least one character string carefully chosen for maximum obscurity and typing difficulty, from the Zire to the Kindle. Ideally, there’s no retyping involved and it’d be nice to not leave the unencrypted contents lying around for very long.

    This has a remarkable number of moving parts…

    It turns out gnukeyring can export each entry into an unencrypted Palm Notepad file through a manual operation, but the cough user experience on the Palm goes something like this:

    • Start gnukeyring, select entry-to-export
    • Select Options, pick export to Notepad
    • Notepad automagically starts up with the new note selected, so close it
    • Exit Notepad, return to Home
    • Iterate

    That gets old very quickly, not to mention that the Notepad file uploads to the PC in the Palm’s slightly binary PDB format that’s basically useless without further hackage and exposes the passwords in unencrypted form.

    Inter-app cutting and pasting on the Kindle is exceedingly tedious, because each app runs more-or-less independently in full-screen mode: it is not feasible to manually transfer the database one field at a time. So we’re looking at a PC-based solution.

    There’s a Java utility that can export the whole encrypted Palm file into spreadsheet CSV format, putting each entry into a separate row and each field into a separate column. OpenOffice / LibreOffice can read the resulting file with no problem. Ctrl-A will select the entire Comment field for each entry, which means I can transfer that in one shot.

    It can also export into an XML file, which seems more useful.

    The utility is a command line thing, with an invocation something like this (use .csv to get that type of output):

    java -jar export.jar Keys-Gtkr.pdb ‘password’ output.xml

    As it turns out, my password contains an exclamation mark and Bash requires fancy escaping (the usual single or double quotes don’t work, because the exclamation mark has higher priority):

    java -jar export.jar Keys-Gtkr.pdb Fancy\!PassWord\! output.xml

    The XML files have this overall structure:

    <U+FEFF><?xml version="1.0" encoding="UTF-8"?>
    <pwlist>
    <pwentry>
      <title>A useful name</title>
      <category>Web</category>
      <notes>The collection of
    notes, asides, and suchlike
    in a multi-line format.
    </notes>
      <lastmodtime>2000-01-01</lastmodtime>
    </pwentry>
    </pwlist>
    

    Notice that each entry contains its category, which makes a certain sense.

    On the KeePass side, there exist Windows and Linux versions of KeePass, although one must be careful to use the 1.x level of the database on Windows, because that’s all the Android and Linux versions know about. It can import XML files and turn them into a database with a unique password; it cannot add entries from an XML file to an existing database.

    KeePassX (the Linux version) XML files looks like this:

    <!DOCTYPE KEEPASSX_DATABASE>
    <database>
     <group>
      <title>Internet</title>
      <icon>1</icon>
      <entry>
       <title>The Entry Title</title>
       <username>your userid goes here</username>
       <password>the mystery password</password>
       <url>example.com</url>
       <comment>Presumably multiline comments work OK</comment>
       <icon>0</icon>
       <creation>2012-05-08T18:48:11</creation>
       <lastaccess>2012-05-08T19:00:49</lastaccess>
       <lastmod>2012-05-08T19:00:49</lastmod>
       <expire>2999-12-28T11:59:59</expire>
      </entry>
     </group>
    </database>
    

    KeePass puts the entries within overall groups, the inverse of the Palm gnukeyring structure. I suppose there’s a way to undo that manually, but … drat!

    So, back to CSV. Although OpenOffice Calc can’t export XML directly, there’s an extension for that. Given a spreadsheet like this:

    asdf dfg
    1 2
    3 4
    5 6

    The XML file looks like this:

    <?xml version="1.0" encoding="UTF-8"?>
    <ooo_calc_export scriptVersion="2.2.0" scriptUpdate="2010-12-19"
                     scriptURL="http://www.digitalimprint.com/misc/oooexport/"
                     scriptAuthor="vjl">
       <ooo_sheet num="1" name="Sheet1">
          <ooo_row><asdf>1</asdf><dfg>2</dfg></ooo_row>
          <ooo_row><asdf>3</asdf><dfg>4</dfg></ooo_row>
          <ooo_row><asdf>5</asdf><dfg>6</dfg></ooo_row>
       </ooo_sheet>
    </ooo_calc_export>
    

    Missing entries generate empty <column-name></column-name> sequences and embedded newlines produce multiline comments.

    So the overall plan:

    • Convert Palm database into CSV
    • Import into OpenOffice Calc
    • Change column names to match KeePass fields
    • Add creation field with current timestamp
    • Add expire field to celebrate the next millennium
    • Sort by Category, delete Category column
    • Export to XML, open in text editor
    • Clean up header & footer tags to suit
    • Insert group tags for category
    • Change ooo_row tags to entry tags
    • Import as new KeePass database
    • Transfer to Kindle Fire
    • Iterate until it works

    While I had the file pinned down in the text editor, I cleaned up some cruft and moved userids / passwords from comments to the appropriate fields.

    I hadn’t used the gnukeyring password field to avoid overwriting an existing password by mistake; that disaster always lay just two screen taps away. It’s a little harder to do on the Kindle, but …

    And then it Just Worked…

  • Kindle Fire Configuration

    Amazon obviously designed the Kindle Fire as an extension of their on-line store: they stripped everything out of Android that could possibly get in the way of buying stuff. Some functions seems obviously necessary, though, so here’s a short list of what I’ve added so far (in addition to the button protector and speaker seals, that is), with all the links collected for reference.

    First and foremost, set a Lock Screen Password. Yes, that means that you must type the password every time the Fire lights up, which is a major nuisance. Remember that the Fire connects directly to your Amazon One-Click account and the browser has the rest of your userids & passwords on tap, so losing it could be a very, very expensive oversight.

    I like 24 hour clocks, but there’s no clock configuration. Dropping a buck on 24 Hours solves that problem.

    My old Zire 71 reminded me of my few appointments & things-to-do, but the Fire lacks the whole Android calendar infrastructure. The Calengoo app syncs with my (previously unused) Google Calendar & Contacts, which means that they now know my social network (such as it is) and what I’m up to. So it goes…

    The system volume control exposes only the Media Volume sound channel. Calengoo produces reminders through the Notification sound channel and, under certain perverse conditions that took me about two days to encounter, can mute that channel and leave it muted forever more. The only way to get audible reminders again is by installing a separate mixer app and resetting the levels: Volume Manager Free.

    Hint: to get dependable audible reminders when the Fire is asleep (which is most of the time), you must enable Pop-up Reminders and disable Pop-up windows, because the dialog box occasionally kills the sound. With that configuration, you’ll get a note in the status bar along the top of the display for each reminder. Set the reminder repeat interval to at least a minute to have enough time for password typing…

    The main reason I got a Fire was to carry all my datasheets & manuals in my pocket, hence the need for color and a backlit screen. Although the Fire can handle PDFs without an app, the native interface leaves quite a bit to be desired. Dropping a few bucks on ezPDF Reader solves most of those problems. Choosing a single file from a collection of several hundred, using a selection browser that ignores the overlaid subdirectory structure, remains challenging.

    File Expert helps a bit by presenting subdirectories and their contents. I think that might be the only way to find a specific PDF.

    Engineering bears need an RPN calculator, of which NeoCal Lite seems to be the best of the bunch.

    Passwords go in KeePassDroid, although it has a clunky clipboard interface to other apps. Of course, any Android app can root the Fire and steal my sensitive bits; that seems to not bother anybody else, so why should I worry? The advantage of using a unique password for each website seems to outweigh the disadvantage of having a single password controlling all of them.

    Turn off the browser’s helpful “remember passwords” function, though…

    Although I’m now using Dropbox to back up the KeePassDroid database file, that whole interface seems overly awkward and I’d rather have encryption applied to every Dropbox file.

    Putting the KeePassDroid database file in the Dropbox folder requires a bit of intervention, as KeePassDroid provides no way to specify the database location. You must find the Dropbox folder using File Expert, slow-click the database file, then drill down through File Expert’s menus in order to specify that KeePassDroid should open the file. After that, KeePassDroid will remember its location. For future reference, it’s at:

    /mnt/sdcard/Android/data/com.dropbox.android/files/scratch/keepass.kdb

    The alert reader may wonder why a Kindle Fire, with a conspicuously missing SD Card slot, has an sdcard subdirectory structure hanging from /mnt: that’s just the way it is. I suppose that’s baked into the Android filesystem; hooray for hardware independence and futureproofing.

    The built-in Silk browser runs slower (certainly, no faster) with Accelerate Page Loading turned on, so there seems no compelling reason to sluice my web content through Amazon’s servers. Not that turning it off improves privacy, of course.

    The Maxthon Mobile Web Browser works reasonably well. The highly regarded Dolphin HD browser isn’t available from the Amazon App Store and sideloading apps from Google Play seems unreasonably difficult. The Firefox Aurora browser isn’t quite ready for prime time, but is the only browser to cover its password database with a master password.

    For the occasional times when I need a stopwatch or timer, the aptly named Stopwatch and Timer app should suffice. It has a breathtakingly awkward UI compared to the Zire app, showing that re-invented wheels sometimes sport square corners…

    ColorNote supports both checklists and text notes for my simple needs.

    Sketch-n-Draw ably demonstrates the Fire’s huge latency between touch-pad input and LCD output; it’s impossible to actually draw anything meaningful. FWIW, the ancient Zire had no trouble doing that, adding Yet Another data point to the curve of software demanding more than the hardware can provide.

    The WordPress blogging app is pretty much useless in comparison with their full web interface, not to mention that typing text on the Fire’s one-finger (or, for those with smaller hands, two thumbs) keyboard is agony, not to mention that the Fire lacks a camera, a microphone, and USB host support. It’s a media consumption device, not a media production device; I knew that when I bought it.

    Although it’s awkward, a conductive-tip Acase stylus helps during extended screen-poking sessions. I have my doubts about the rubbery tip’s durability, though.

    All in all, the Fire seems serviceable…

  • Converting DICOM X-ray Images to Something Useful

    For reasons that aren’t relevant here, we have a CD bearing X-rays of Mary’s shoulder. Of course, they’re in DICOM image format and come with a relentlessly Window-centric viewer that won’t run in Wine and can’t export the files in a more useful format.

    Imagemagick to the rescue:

    
    convert /media/floppy/DICOM/997313/00100000 "Mary Shoulder 2.jpg"
    
    
    Mary Shoulder 2 - detail
    Mary Shoulder 2 – detail

    They tell us she has great bones and everything worked out fine…