Blog Backup

Recent news about Dropbox removing its Public folder feature reminded me to do my every-other-month blog backup. Wordpress provides a method to “export” the blog’s text and metadata in their XML-ish format, so you can (presumably) import your blog into another WordPress instance on the server of your choice. However, the XML file (actually, ten of ’em, all tucked into a paltry 8 MB ZIP file) does not include the media files referenced in the posts, which makes sense.

Now, being that type of guy, I have the original media files (mostly pictures) tucked away in a wide variety of directories on the file server. The problem is that there’s no easy way to match the original file to the WordPress instance; I do not want to produce a table by hand.

Fortunately, the entry for each blog post labels the URL of each media file with a distinct XML tag:

		<wp:attachment_url>https://softsolder.files.wordpress.com/2008/12/cimg2785-blender-bearings.jpg</wp:attachment_url>

Note the two leading tabs: it’s prettyprinted XML. (Also, should you see escaped characters instead of < and >, then WordPress has chewed on the source code again.)

While I could gimmick up a script (likely in Python) to process those files, this is simple enough to succumb to a Bash-style BFH:

grep attachment_url *xml > attach.txt
sed 's/^.*http/http/' attach.txt | sed 's/&lt;\/wp.*//' > download.txt
wget --no-verbose --wait=5 --random-wait --force-directories --directory-prefix=/where/I/put/WordPress/Backups/Media/ -i download.txt

That fetches 6747 media files = 1.3 GB, tucks them into directories corresponding to their WordPress layout, and maintains their original file dates. I rate-limited the download to an average of 5 s/file in the hope of not being banned as a pest, so the whole backup takes the better part of ten hours.

So I wind up blowing an extra gig of disk space on a neatly arranged set of media files that can (presumably) be readily restored to another WordPress instance, should the occasion arise.

Memo to Self: investigate applying the -r option to the base URL, with the -N option to make it incremental, for future updates.

The Thrilling Adventures of Lovelace and Babbage

We’re reading Sydney Padua’s The Thrilling Adventures of Lovelace and Babbage as our evening story, so I gave a Lightning talk at the MHV LUG meeting last week:

MHVLUG – Lovelace and Babbage – Book Report

Earlier versions of the comics graphic novel are on her blog, including several stories that didn’t make the final book cut.

Highly recommended; if you don’t have wet eyes occasionally, you’re entirely too hard-hearted.

You should read Ada’s Analytical Engine Programming Guide; that’s not her title, but that’s what she wrote. If you’ve ever done any assembly language programming, you’ll feel right at home.

Also, get historical documents, commentary, and Analytical Engine emulators (!) at Fourmilab.

Makes me wish I lived in that Pocket Universe, it does:

econ3_005 - Brunel
econ3_005 – Brunel

That picture is ©www.sydneypadua.com, Creative Commons Attribution-NonCommercial 4.0 International License. There exist T-shirts & mugs.

Why Friends Don’t Let Friends Run Windows: Cryptolocker Downloader

Got an email, nominally from one Richard Gilmore of FedEx, concerning a parcel sent as International Next Flight (whatever that is). The Subject line read “We could not deliver your parcel, #00000665103”, although the message didn’t quite match:

Dear Customer,

This is to confirm that one or more of your parcels has been shipped.
Delivery Label is attached to this email.

Kind regards,
Richard Gilmore,
Sr. Delivery Agent.

The email address had nothing to do with FedEx, of course, and my filters tagged it as spam.

The “label” came in a ZIP file: Label_00000665103.zip

Extracting the “label” produced what would look like an MS Word file, if you were so trusting as to hide extensions of “known” filetypes and didn’t worry when you saw a file still sporting a DOC extension: Label_00000665103.doc.wsf

Handing that to VirusTotal produces no surprise at all:

VirusTotal Report
VirusTotal Report

The file contains one very long line, the first chunk of which suggests it’s up to no good:

<job><script language=JScript>var a59253 = '+"HKCU"+cs'; var a59168 = '"); fp.WriteLine(" '; var a5988 = ';} else if('; var a59196 = 'gth;i'; var a59160 = 'fp.W'; var a59261 = 'ion"+c'; var a5999 = 's(f'; var a59254 = '+"SOFTWARE"+';

After a bit of poking, I applied a few minutes of sed reformatting, manual cleanup, and sorting:

sed 's/; var a/;\n/g' Label_00000665103.doc.wsf > lines.txt
... fix a few lines ...
sort -n lines.txt > sort.txt

Which produced a file starting out like this:

<job><script language=JScript>
590 = 'var id="TRIB9RMvAFl04U4Fi7L6RNk9ZowJ2sj_fIrO0WiXGlXd53j6oENCCFDZ9NbVubN-vvJltoR8Wf4_";d';
591 = '="1vcs62wsoYZNc4TdwqgsG5965bDt3mNYW"; var bc="0.52';
592 = '189"; var ld=0;';
593 = ' var cq';
594 = '=S';
595 = 'tri';
596 = 'ng.f';
597 = 'romCharCode(34);';
598 = ' var cs';
599 = '=Strin';
5910 = 'g.fromCh';
5911 = 'ar';
5912 = 'Code(92); var ll';
5913 = '=["32jelen.pl","v';
5914 = 'iktoriascho';
5915 = 'ol.ru","blende';
5916 = 'r.com.br';
5917 = '","pasargad1007.c';
5918 = 'om","www.unit';
5919 = 'ed-systems.it"';
5920 = ']; v';
5921 = 'ar ';
5922 = 'ws=WScript.Cre';
5923 = 'ateObject(';
5924 = '"WScript.Shell';
5925 = '"); v';
5926 = 'ar';
5927 = ' fn=ws';
5928 = '.Expa';
5929 = 'ndEnv';
5930 = 'ironme';
5931 = 'ntString';
... snippage ...

Even without pasting the fragments back together, you can puzzle out the punchline:

59108 = 't",true); fp.Write';
59109 = 'Line("ATTEN';
59110 = 'TION!"); fp.Wr';
59111 = 'ite';
59112 = 'Line(';
59113 = '""); fp.W';
59114 = 'riteLine("All';
59115 = ' your d';
59116 = 'ocuments, p';
59117 = 'hotos';
59118 = ', databases and ot';
59119 = 'her import';
59120 = 'ant ';
59121 = 'pers';
59122 = 'onal fil';
59123 = 'es"); fp.';
59124 = 'Wri';
59125 = 'te';
59126 = 'Line(';
59127 = '"were e';
59128 = 'ncrypted usi';
59129 = 'ng strong RSA-1024';
59130 = ' algorithm with ';
59131 = 'a uniqu';
59132 = 'e key."); fp.Write';
59133 = 'Line(';
59134 = '"To restor';
59135 = 'e your files you h';
59136 = 'ave to pay "+bc+" ';
59137 = 'BTC (bitcoin';
59138 = 's)."); fp.Wri';

Huh. CryptoLocker returns from the dead! Right now, 0.52 BTC = $316.15, so I guess I can drop that into the jar of money saved by running Linux.

If those emails didn’t work so well, they wouldn’t send them…

Monthly Science: Chrysalid Engineer

So then this happened:

Karen - canonical tiger paw graduation picture
Karen – canonical tiger paw graduation picture

Yeah, tanker boots and all; not the weirdest thing we saw during RIT’s graduation ceremonies.

This summer marks her fourth of four co-op semesters with Real Companies Doing Tech Stuff and her final classes end in December; RIT holds one ceremony in the spring and being offset by a semester apparently isn’t all that unusual. She (thinks she) has a job lined up after graduation and doesn’t need her doting father’s help.

But, hey, should you know someone with a way-cool opportunity (*) for a bright, fresh techie who’s increasingly able to build electronic & mechanical gadgets and make them work, drop me a note and I’ll put the two of you in touch. [grin]

(*) If that opportunity should involve 3D printed prosthetics with sensors and motors, she will crawl right out of your monitor…

Hiatus

After devoting the last few months to setting up the Makerspace Starter Kit and extracting / organizing / stashing the stuff I wanted to keep:

New parts cabinets
New parts cabinets

I now have some difficulty accomplishing what needs to be done:

Basement Shop - right
Basement Shop – right

During the rest of May I must write a pair of columns, unpack / arrange / reinstall my remaining tools / parts / toys, endure a road trip to our Larval Engineer’s graduation (*), enjoy bicycling with my Lady, and surely repair a few odds-n-ends along the way.

I’ll generate occasional posts through June, after which things should be returning to what passes for normal around here…

(*) For reasons not relevant here, our Larval Engineer’s schedule includes a final co-op and wind-up semester after “graduation”. Perhaps she’s entering the Chrysalis phase of her development?

Autodesk Privacy Statement: “Do Not Track” and Similar Mechanisms

Autodesk just Borged Netfabb and, in the process of merging their address lists, asked me to update my info and agree to their very detailed Privacy Statement. You should take a look at it; the link will open in a new tab / window / whatever, so you don’t lose your place here.

Have you noticed how those “statements” always have a very long and firmly fixed line width that doesn’t adapt to your window size, use various shades of light-gray-on-white typefaces in the smallest sizes, and continue for pages and pages. I don’t believe in coincidences, either.

Here’s what they think of my Do Not Track browser setting (emphasis added):

“Do not track” and similar mechanisms

Some web browsers may transmit “do-not-track” signals to websites with which the browser communicates. Because of differences in how web browsers incorporate and activate this feature, it is not always clear whether users intend for these signals to be transmitted, or whether they are even aware of them.

Participants in the leading Internet standards-setting organization that is addressing this issue are in the process of determining what, if anything, websites should do when they receive such signals. Autodesk currently does not take action in response to these signals. If and when a final standard is established and accepted, we will reassess how to respond to these signals.

For information about cookies, web beacons and similar technologies, please read our Cookie Policy.

After plowing through much of their “statement”, I decided Autodesk doesn’t do anything I need to know about and, seeing as how Netfabb gradually faded from my attention when their web service joined Microsoft’s Azure cloud, I declined to “confirm my preferences” and didn’t click the big blue button. I doubt such inaction will remove my email address from their list, but it’s the only choice they offer.

Because I can’t tell if a website really wants to track me, I block ads, disable Flash, and destroy all cookies when I leave their site, Just In Case they inadvertently deployed all that crap. I’m sure they never intend to serve malware through an ad slot brokered on their site, but mistakes do happen, and I’m glad to assist them.

If you’re seeing ads on this page right now, they come from WordPress and I get a small cut. You should start using an ad blocker right now; if your browser doesn’t permit you to block ads, change browsers. If you worry that reducing my advertising revenue will compromise the quality and quantity of what you see here, send me a sack of money through Paypal. Fair enough?

Also: Linux, dammit.

Traffic Signal Timing: NYSDOT Responds

On 12 July 2015, I sent a report to NYSDOT about how the traffic signals at Burnett Blvd / Rt 55 greenlighted opposing traffic when our bicycles were still in the intersection:

Can you increase the minimum green and yellow times on the signals from Burnett Blvd to Rt 55?

The current settings are too short for bicycle traffic making a left turn across six traffic lanes.

The pictures show key points from our ride on 2015-07-10, returning from the Balloon Festival in Poughkeepsie. We took the DCRT around Poughkeepsie, went through Arlington to Rt 376 at Collegeview, then took Rt 376 Red Oaks Mill.

The image sequence numbers identify frames extracted from video files. The Front camera runs at 60 fps and the Rear camera at 30 fps.

The red signals are in the process of turning off in Front 0196.

Burnett at Rt 55 Signal - Front 0196
Burnett at Rt 55 Signal – Front 0196

One second later in Front 0260, the car and our bikes are starting to roll. Given the number of drivers blowing through red signals at full speed, devoting one second to watching for oncoming traffic seems prudent.

Burnett at Rt 55 Signal - Front 0260
Burnett at Rt 55 Signal – Front 0260

The yellow signals are turning on in Front 0633, seven seconds after the green. The car has reached the pedestrian ladder across Rt 55, but we’re still crossing the westbound lanes of traffic. We may not be the fastest riders on the road, but we’re not the slowest, either.

Burnett at Rt 55 Signal - Front 0633
Burnett at Rt 55 Signal – Front 0633

We’ve reached the far side of the intersection in Front 1142, just under 16 seconds from the green.

Burnett at Rt 55 Signal - Front 1142
Burnett at Rt 55 Signal – Front 1142

However, Rear 0408 shows that the opposing signals turned green while we’re still crossing the eastbound lanes of Rt 55. That’s about 15 seconds after the Burnett Blvd signals went green.

Burnett at Rt 55 Signal - Rear 0408
Burnett at Rt 55 Signal – Rear 0408

About 2.7 seconds later, Rear 0490 shows cars accelerating across the intersection toward us as we cross the pedestrian ladder. They started rolling immediately after their signal went green; waiting a second isn’t a universal practice.

Burnett at Rt 55 Signal - Rear 0490
Burnett at Rt 55 Signal – Rear 0490

Setting the minimum Burnett green to 12 seconds, the minimum yellow to 10 seconds, and the minimum delay from Burnett green to Rt 55 green to 30 seconds would help cyclists (just barely) reach the far side of the intersection before opposing traffic starts rolling.

Also: can you adjust the sensor amplifiers on Burnett to respond to bicycles and mark the coil locations on the pavement in both lanes? That would help us through the intersection during low-traffic-volume times, as our bikes seem unable to trip the signals.

Thanks…

This reply from the NYSDOT autoresponder was all I ever got from them:

Thank you for your inquiry.  We will respond to your email message as soon as possible.

On 2 August 2015, I sent a report to NYSDOT about how the traffic signals at Old Post Rd – Spring Rd at Rt 9 greenlighted opposing traffic when our bicycles were still in the intersection:

The minimum green-to-opposing-green signal timing from Old Post Road across Rt 9 to Spring road is about 18 seconds: not long enough for bicycles to safely cross an intersection with eight traffic lanes.

The “Green” picture shows our starting position as the signal turned green: behind the first car in line. There’s another car behind us, which ensures the loop sensor will trip; it does not detect bicycles.

Spring Rd - Rt 9 - 2015-08-01 - Green
Spring Rd – Rt 9 – 2015-08-01 – Green

The “Yellow” picture shows the signal changing after 12 seconds, with the car from behind us now in the middle of the northbound lanes. We’re still in the middle of the southbound lanes.

Spring Rd - Rt 9 - 2015-08-01 - Yellow
Spring Rd – Rt 9 – 2015-08-01 – Yellow

The “Opposing Left Green” from the rear camera, 18 seconds from the first picture, shows green left-turn arrows for Spring Road. The opposing cars began rolling with Mary lined up with the northbound right-turn lane and me lined up with the right travel lane.

Spring Rd - Rt 9 - 2015-08-01 - Opposing Left Green
Spring Rd – Rt 9 – 2015-08-01 – Opposing Left Green

The car behind blew through the red signal on Old Post Rd; I think that’s why the opposing left-turning cars didn’t start sooner.

In the other direction, I often use the left turn from Spring Rd to southbound Rt 9 to reach the South Road Square strip mall. Similarly short yellow and overall cycle times apply in that direction.

Can you add (at least!) five seconds to the yellow and perhaps ten seconds to the minimum cycle time for both directions? That would help us clear the intersection before opposing traffic starts moving again.

Can you also mark the sensor loop locations in all those lanes so cyclists can find them and adjust the amplifier sensitivity / dwell to respond to bicycles? We’ve lined up atop the quadrupole loop pavement cuts on Old Post Road to no avail, but there’s not even a hint of the loop positions under the new Spring Rd paving.

Thanks…

This reply from the NYSDOT autoresponder was all I ever got from them:

Thank you for your inquiry.  We will respond to your email message as soon as possible.

On 5 January 2016 I posted a description of our encounter with a car at the Burnett Blvd intersection and sent the link to the NYSDOT Bicycle and Pedestrian Coordinator. If you search for Burnett Blvd, you’ll find a few other mentions of that intersection.

On 6 January 2016, this email message arrived from the same email address that never responded to my reports (emphasis added):

Dear Mr. Nisley:

This is in response to your correspondence regarding your experiences as a bicyclist at the intersections of Route 55 at Burnett Boulevard and Route 9 at Spring Road in the Town of Poughkeepsie, Dutchess County.

The New York State Department of Transportation (NYSDOT) is in the process of investigating alternate detection types and inductance loop patterns that would detect a wider range of vehicles.  As alternate detection types are tested and approved, they will be integrated into the next traffic signal upgrade at both intersections.  The distance varies based on geometry.  The loops are centered in each lane and the front loop is a quadrapole, so there are wires down the middle of the loops.

A new timing program was implemented at Route 9 at Spring Road in August, and the yellow and red clearance times meet the current standards.  The timing at Route 55 at Burnett Boulevard is in the process of being updated, and the clearance times will be updated as necessary to meet the current standards.  Clearance times are determined based on speed, intersection dimensions, grade, and reaction time and cannot be adjusted.  The sensitivity on all loops will also be adjusted, so they are as sensitive as possible without causing cross talk between the loops.

We appreciate and share your interest in making our highway systems safe and functional for all users.

If you have any questions or need additional information, please feel free to contact our Regional Traffic Safety & Mobility Group at (845) 437-3396.

NYSDOT Hudson Valley Region

I don’t regard that date a coincidence; NYSDOT was not responding to my reports. I sent a further note to clarify a few points:

On 01/05/2016 02:18 PM, dot.sm.r08.nysdot wrote:
Clearance times are determined based on speed,intersection dimensions, grade, and reaction time and cannot be adjusted.

That seems to mean the times can be adjusted, but you won’t adjust them to allow cyclists enough time to clear the intersection.

We appreciate and share your interest in making our highway systems safe and functional for all users.

So, giving opposing traffic a green light while we’re still in the intersection NYSDOT’s way of “making our highway systems safe and functional for all users”.

Do I understand your statements correctly?

No reply, as I’ve come to expect by now.

I think the emphasis on “meet(ing) the current standards” is how NYSDOT will attempt to defend against claims that road conditions caused or contributed to a car-on-bike collision. I find it surprising that contemporary “standards” would allow greenlighting opposing traffic against bicycles, but perhaps they simply choose a standard that excludes bicycles.