The Smell of Molten Projects in the Morning

Ed Nisley's Blog: Shop notes, electronics, firmware, machinery, 3D printing, laser cuttery, and curiosities. Contents: 100% human thinking, 0% AI slop.

Category: Software

General-purpose computers doing something specific

Kmail: FAIL

In the unlikely event you’re keeping track of this, slashing the total volume of email made Kmail much more competent: it hadn’t trashed an index file in, oh, weeks…

Until it happened again.

I don’t know that 30 days of email is magic, but 64 MB worked much better than 3 GB. The offending folder has all of 6 MB and 280 files, which puts a pathetic upper bound on Kmail’s good behavior.

Anyhow, Kmail still screws up its indexes, but … it’s better than it was.

You’d think this would be an important thing to get right, but the KDE apparat has far more important things to worry about. Eye candy, as nearly as I can tell.

I use Kmail because it’s one of the few email readers that stores messages in maildir format. That’s important with large email collections, because mbox, the other choice, tucks all the messages into a single honkin’ big file (perhaps one file per folder). That doesn’t work well with a daily backup strategy, because each message changes the file and triggers a backup of the whole thing. Maildir format means backing up only the new messages, which makes far more sense.

But, if this blank email thing continues, it’s time to move on…

2010-05-22
OpenOffice: Not Spell-Checking URLs
For some reason, the default character style automagically applied to URLs uses English… which means the spellchecker doggedly attempts to make sense out of that gibberish.

Solution (at least for OOo 3.2):
- F11 to get the Style and Formatting dialog up
- Click Character Styles
- Click Hierarchical in the lower drop-down list to see ’em all
- Right-click Internet Link, select Modify
- Select [None] in the Language drop-down list
- OK your way back out
There, now, wasn’t that easy?

Oh, yeah, you’ll want to do that to all the templates you use to create blank documents, too.
2010-05-03
Kmail: Copying Directory Structures and Some Files Therein
As part of the Kmail blank email problem, I conjured up a new, shrunken maildir structure with just the most recent 30 days of email. However, I want to keep all the same folders, even if they have no current email messages, so my filters can sort the incoming mail properly.

The sequence of events:
- shut down Kmail!
- move existing email directory out of the way
- set up a new directory
- copy the directory structure
- copy the most recent 30 days of email
- delete the existing index files
That’s straightforward, at least after you figure it out. Took me a while to get it right, but here ya go…
```
cd to the directory holding the Mail folder
mv Mail Mail.base
mkdir Mail
cd Mail.base
find . -type d -print0 | cpio --null -dmpv ../Mail
find . -type f -mtime -30 -print0 | cpio --null -apdv ../Mail
cd ../Mail
find . -type f -name ".*index" -print0 | xargs -0 rm
find . -type f -name ".*index.ids" -print0 | xargs -0 rm
```
Then fire up Kmail and let it rebuild all the index files.

You ought to try that out on a dummy version of your email, as something may have gotten clobbered in the transition from my terminal to your fingertips, OK?
2010-05-02
KMail: The Blank Email Problem
Of late, Kmail has been turning email messages into complete blanks: the Subject, From, Date, and body are all completely blank. This is evidently a problem of long standing with Kmail and has something to do with fumbling the indexes that point to the emails within its maildir directory structure.

The FAQ blandly notes:

You have empty ‘ghost-mails’ in your inbox (or other folder)

Symptom: For some reason, certain messages aren’t accessible in KMail. They show up in the message list window but selecting them there results in a blank message window. I can’t open them or reply to them, etc.

Solution: This problem ist most likely due to corrupted index files, see issue ‘You are loosing mail’ above. So just follow the advice given there.

Well, yeah, except that rebuilding the indexes more than once a day seems excessive… and the problem is, intermittently, much worse than that.

I’m running KMail in XFCE, which introduces some complexity, but other folks with the same problem are running it in bone-stock KDE. Surprisingly the recent 4.x upheavals haven’t changed the problem in the least.

I’ve been keeping the maildir structure on the file server, rather than my local drive, and symlinking to it from my home directory through NFS. That also doesn’t seem to change the symptoms, although putting a heavy load on either the network or the server sometimes increases the number of blank emails.

Over the last few months I’ve tried a number of things, like tweaking NFS buffer sizes & timings, to no avail. Time to start writing this stuff down…

With that as prologue, here’s how to recover those blank emails.

Most important: when you see a blank email, get out of Kmail. Nothing you do within Kmail will help and many things will hurt, so just bail out.

Fire up a terminal window and cd to the directory representing that email folder. First-level folders have the obvious name, but all the second-level folders are in hidden directories. For example, I have a top-level folder called Bulk Stuff, with one sub-folder (among many) being EMC.

The directory structure:
```
Mail/Bulk Stuff/
Mail/.Bulk Stuff.directory/EMC/
```
Yeah, embedded blanks. Sue me.

Each of those directories has three subdirectories: cur, new, and tmp.

The problem seems to arise when a new message gets transferred from new to cur, although sometimes existing messages in cur go bad. The index entry seems to point to the wrong place; the actual mail message file is in cur, but the index points off into the bushes somewhere.

The solution is to manually move the file from cur back to new, then rebuild the offending index. Leaving it in cur and just rebuilding the index does not always work, for reasons I do not understand.

The easiest way to find the newest messages:
```
cd "Mail/.Bulk Stuff.directory/EMC"
ll cur | tail
```
This will show the most recent few entries, which will look something like this:
```
-rw-r--r-- 1 ed ed 22256 2010-04-19 20:48 1271724492.2194.DbdZD:2,S
-rw-r--r-- 1 ed ed 23513 2010-04-20 13:09 1271783386.2232.jxmG6:2,S
-rw-r--r-- 1 ed ed 20901 2010-04-20 17:10 1271797805.2232.i6fP3
```
The last line shows the most recent files hasn’t been read yet, which is a tipoff that something’s wrong. If you have an older message with a rotten index entry, use grep (or some such) to find it.

Move the file back to new and delete the corresponding index files:
```
mv cur/1271797805.2232.i6fP3 new
cd ..
rm ".Bulk Stuff.index*"
```
Then fire up Kmail again and it’ll automagically rebuild the indexes. That’ll work fine for a while, then it’ll screw up again.

I suspect that the problem is related to either the number of messages or the index file size for each maildir directory. I have, in round numbers, 3 GB of mail stashed away. As with anything, most of it is useless , but I occasionally need one of those messages ever so urgently.

I set up a new maildir structure with only the last 30 days of email transactions, which should be enough to either eliminate the problem or show that Too Many Messages is just another dead end.

More details on that tomorrow…
2010-05-01
On Schematic Capture and PCB Layout Programs

This useful comment thread showed up in relation to a post about a chainsaw repair, which would hide it from any rational collection of search terms. Here’s the thread in all its glory, as there doesn’t seem to be a way to move comments from one post to another.

Feel free to continue the topic in the comments to this post…

randomdreams

Offtopic: have you ever used gEDA for schematic or pcb? I’m looking for something with reasonable abilities, and the crippled demo versions of orcad, eagle, and winqcad all look fairly crippled. I’ve zero use for autorouters and autoplacers (because they suck for analog design) but it’d be nice to have something that’s fairly usable for schematic and layout.

Ed

used gEDA for schematic or pcb?

Nope. Every time I’ve looked at it, the status seems to be heartbreakingly close to being useful by someone who really doesn’t want to work around a morass of limitations. That’s becoming less true and maybe by now it’s practical… but I haven’t done a serious examination for maybe a year.

I actually coughed up half a kilobuck for the Standard version of Eagle schematic & layout, as an autorouter doesn’t do much for the little bitty boards I build. Works fine, no complaints, but if I weren’t doing columns and suchlike, it’d be hard to justify.

Neal H.

andomdreams,

I have used it for both a small project and a slightly bigger project,
http://www.instructables.com/file/FPCHBLIG1M2BQTH (schematic)
http://www.instructables.com/file/F6SYDYYG1M2KI8M (render of layout)

It is quite usable, but the version that ships with most linux distros is pretty old, I had much better luck building it from source following the instructions on the gEDA homepage. The hardest part is creating symbols for the PCB tool, that is a little tricky to learn, but there are a lot of them pre-created for you on gedasymbols.org.

randomdreams

Thanks to both of you. I’ll probably give it a run. I currently spend much of my day creating symbols for Cadence, and I consider it impossible for any other part-creation process to be as painful or difficult as that. I’m more worried about general usability. Eagle’s the back-up plan.

John Rehwinkel

I still haven’t found a PCB layout program I like (and I’ve gotten tired of the truly primitive one I wrote 20 years ago). For schematics, I use DesignWorks Lite, which is apparently no longer offered (though DesignWorks Professional is still available).

John Rehwinkel

Ah, it is still offered (only $40), just not at the main Capilano site. The companion PCB layout program is Osmond ($200), which I keep meaning to try out. You can download the trial version at designworks4.com.

2010-04-30
Ancient CM11A X10 Controller Pinout
I have an X10 CM11A “Two Way Computer Interface” handling the very very very few scheduled events for our house. Basically, it turns the living room lights on in the evening and everything off much later.

As a result, I tend to ignore it for years at a time. A recent power outage killed the regularly scheduled events, which suggested that the backup batteries needed changing… and, yes, they were pretty well corroded.

With that out of the way, I discovered that the last time I’d loaded a program into the thing was so long ago that the heyu config files had either gone missing or were on a system not near the top of my heap. It’s easy enough to configure, so I installed heyu and spun up a new set of config files.

All the doc I can find says the CM11A has an RJ11 modular phone jack, which mates with the standard 6-position 4-conductor dingus found on the end of every phone in this part of the world. My CM11A, however, has a 4P4C jack, the narrower dingus found on phone handsets. Given that heyu reports
```
Firmware revision Level = 1
```
I suspect that this thing is slightly older than some of the folks reading this post and the X10 factory switched to a somewhat less bizarre connector in mid-stream.

Anyhow, the DB9 (yeah, it’s a DE9, but nobody calls it that) connector has “X10 Active Home” printed on it in my very own handwriting, with a standard RJ11 plug on the end. A double-jack adapter connects a hank of cable with an RJ11 plug on one end and a 4P4C connector on the other. I have no idea where that cable came from; perhaps I replaced the 4P4C plug with something less bizarre to add that extension so the cable would stretch from PC to wall outlet?

I plugged the thing into a USB-RS232 adapter and heyu had no trouble talking to the CM11A. However, trying to execute
```
heyu dim n13 10
```
produced the discouraging report
```
RI serial line may be stuck.
```
A bit of deft multimeter work produced this pinout list, which agrees with most of the doc you’ll find elsewhere. Hold the 4P4C connector with the tab down and the cable away from you: the pin numbers are 4 3 2 1 from left to right. The RS-232 pins are printed right on the DB-9 connector.
```
4P4C   DB9
 1      2 RxD
 2      9 RI
 3      3 TxD
 4      5 Gnd
```
It’s entirely possible the USB converter doesn’t support RI or it doesn’t do a good job of it. I jammed the cable into the serial port on the back of the PC and shazam it works perfectly.

The x10.conf file, for the next time around
```
TTY /dev/ttyS0

HOUSECODE N

LATITUDE 	41:40N
LONGITUDE	73:53W

ALIAS MBR_Dresser	N1
ALIAS Front_Hall	N5
ALIAS RV_XCVR		N9
ALIAS Couch		N10
ALIAS Mary_Reading	N11
ALIAS LR_Ceiling	N12
ALIAS Fireplace		N13
ALIAS Kitchen		N14
ALIAS Patio		N15
ALIAS Garage_Spots	N16

START_ENGINE	AUTO

LOG_DIR		/var/log/heyu/

DATE_FORMAT	YMD '-'
```
2010-03-30
EMC2 Gamepad Pendant: Joystick Axis Lockout
Nothing like sleeping on a problem. It turns out that a chunk of HAL code can do a nice job of locking out an inactive joystick axis.

The general idea:
- A priority encoder selects one axis when both go active simultaneously
- The prioritized outputs set flipflops that remember the active axis
- The active axis locks out the other one until they’re both inactive
That way, you can start to jog either axis on a knob without worrying about accidentally jogging the other axis by moving the knob at a slight diagonal. I hate it when that happens.

The other tweak is that the quartet of buttons on the right act as a “hat” for the Z and A axes, jogging them at the current maximum speed.

Because it’s tough to accidentally push two buttons at once, there’s no need to lock them out. So you can jog diagonally by deliberately pushing adjoining buttons, but you must want to do that.

Rather than dumping the whole program again, here are the key parts…

Figuring out if a joystick axis is active uses the window comparators. It seems the idle counts value varies slightly around 127, so I relaxed the window limits. Should the window comparator go active with the knob centered, the buttons for that axis won’t produce any motion.
```
net		x-jog-count-int	input.0.abs-x-counts	conv-s32-float.0.in
net		x-jog-count-raw	conv-s32-float.0.out	wcomp.0.in
setp	wcomp.0.min		125
setp	wcomp.0.max		130
net		X-inactive		wcomp.0.out				not.0.in
net		X-active		not.0.out
```
The priority encoder is just a gate that prevents Y (or A) from being selected if X (or Z) is simultaneously active. Here’s a sketch for the ZA knob:

Axis priority encoder

The active and inactive signals come from the window detectors. The sketch gives the K-map layout, although there’s not a whole lot of optimization required.

The corresponding code:
```
net		Z-inactive		and2.5.in0
net		A-active		and2.5.in1
net		A-select		and2.5.out				# select A only when Z inactive

net		Z-inactive		and2.6.in0
net		A-inactive		and2.6.in1
net		ZA-Deselect		and2.6.out				# reset flipflops when both inactive

net		Z-active		and2.7.in0				# set Z gate when knob is active
net		A-gate-not		and2.7.in1				# and A is not already gated
net		Z-set			and2.7.out					flipflop.2.set

net		ZA-Deselect		flipflop.2.reset		# reset when neither is active
net		Z-gate			flipflop.2.out				not.6.in
net		Z-gate-not		not.6.out

net		A-select		and2.8.in0				# set A gate when knob is active
net		Z-gate-not		and2.8.in1				# and Z is not already gated
net		A-set			and2.8.out					flipflop.3.set

net		ZA-Deselect		flipflop.3.reset		# reset flipflop when both inactive
net		A-gate			flipflop.3.out				not.7.in
net		A-gate-not		not.7.out
```
The flipflops remember which axis went active first and lock out the other one. When both axes on a knob return to center, the flipflops reset.

The quartet of buttons produce binary outputs, rather than the floats from the Hat, so a pair of multiplexers emit -1.0, 0.0, or +1.0, depending on the state of the buttons, for each axis.
```
setp	mux2.6.in0	0.0
setp	mux2.6.in1	-1.0
net		A-btn-neg		input.0.btn-trigger		mux2.6.sel
net		A-btn-neg-value	mux2.6.out				sum2.1.in0

setp	mux2.7.in0	0.0
setp	mux2.7.in1	1.0
net		A-btn-pos		input.0.btn-thumb2		mux2.7.sel
net		A-btn-pos-value	mux2.7.out				sum2.1.in1

net		A-jog-button	sum2.1.out

net		A-btn-neg		or2.1.in0
net		A-btn-pos		or2.1.in1

net		A-btn-any		or2.1.out				or2.2.in0
net		A-gate			or2.2.in1
net		A-motion		or2.2.out
```
The A-motion signal is true when either of the A jog buttons or the A joystick axis is active. That gates the MAX_ANGULAR_VELOCITY value to halui.jog-speed, rather than the default MAX_LINEAR_VELOCITY. Or, depending on the state of the toggle from the two joystick push switches, 5% of that maximum. A mere 5% may be too slow for the A axis, but it’ll take some experience to determine that.

With that in hand, the final step is gating either the knob or the button values to halui.jog.*.analog.
```
net		Z-jog-button	mux2.8.in0
net		Z-jog-knob-inv	mux2.8.in1
net		Z-gate			mux2.8.sel
net		Z-jog			mux2.8.out				halui.jog.2.analog

net		A-jog-button	mux2.9.in0
net		A-jog-knob		input.0.abs-z-position	mux2.9.in1
net		A-gate			mux2.9.sel
net		A-jog			mux2.9.out				halui.jog.3.analog
```
The complete source file (Logitech Dual Action Gamepad – joystick axis lockout – custom_postgui-hal.odt) is over on the G-code and Suchlike page, so you can download it as one lump. It’s an OpenOffice document because WordPress doesn’t allow plain text files.

I loves me my new joggy thing!
2010-03-24