Rant: Software Testing

I admit to having expectations, perhaps unreasonable expectations, about what should transpire after filing a bug report with a software project large enough to have a bug tracking system.

• Acknowledge the report right away, lest it appear nobody cares.
• Figure out what else you need to know; I give good bug report, but if you need more, ask now.
• Triage, set, and meet a due date, lest your development process appear shambolic.
• When we outsiders become the software testers, keep us in the loop.

My experience shows that the larger and better-funded the organization, the less productive will be any given report: individual problems get lost in the noise. Firefox, Ubuntu, and the late OpenOffice serve as cautionary tales; their forums may help, but submitting a problem report doesn’t increase the likelihood of getting a timely fix.

I cut one-horse open-source operations considerable slack, although they rarely need any. For example, a recent OpenSCAD problem produced turnaround time measured in hours, including a completely new source file cooked up by a bystander and the lead developer polishing it off a sleep cycle later. That was on Christmas Eve, from on vacation, in a low-bandwidth zone, evidently through an iDingus.

Perhaps I’m more sensitive to software quality assurance than most folks, but for good reason.

Quite some years ago, my esteemed wife earned an IBM Outstanding Technical Achievement Award for testing a major component of a major OS. They don’t hand those tchotchkes out lightly and rarely for anything other than development. She had skip-leveled(*) to her umpteenth-level manager: “IBM must not ship this product. It does not work. It is an embarrassment. I will not concur with any decision to ship.”

The thing eventually shipped, half a year behind schedule, after the developers produced code that passed her test suite and she signed off on the results. Word has it that blood ran ankle-deep in the corridors toward the end.

If you should ever encounter someone in need of a software-testing team leader who doesn’t take crap from anybody, let me know. She’ll definitely require considerable inducement to drag her back into the field, away from her gardens and quilting…

So, anyway, we know a bit about software testing and verification and QC in these parts.

Selah.

(*) Old-school IBM jargon for walking around several levels of obstructive management to meet with a Big Shot in the corner office. Might not happen in the New IBM, alas.