Finding & Copying Only The New Music Files

Given a collection of music files in various subdirectories, find all the mp3 files that aren’t in the target directory and copy them. The only catch: don’t use rsync, because the target directory is on a Google Pixel phone filesystem which doesn’t support various attributes required by rsync.

The solution goes like this:

cd /mnt/music/Netlabel Mixes
sudo jmtpfs /mnt/pixel -o allow_other,fsname="Pixel"
find . -name \*mp3 -execdir test ! -e  /mnt/pixel/Internal\ shared\ storage/Music/Netlabel/\{\} \; -execdir cp -v -t /mnt/pixel/Internal\ shared\ storage/Music/Netlabel/ \{\} \;
sudo umount /mnt/pixel

The trick is remembering the second execdir operation in find happens only if the first succeeds, so the cp runs when the target file doesn’t exist.

All the backslash escaping gets tedious, but it’s the least awful way to get the job done when the directories contain blanks, which is true for the default directory structure inside the Pixel.

Your choice in music will surely be different …

Closing the Dmesg Audit Firehose

I’m not entirely clear what’s being audited in the Manjaro Linux boxes I’ve recently set up, nor what the difference between res=success and res=failed might mean for the x11vnc unit:

[   98.632347] audit: type=1131 audit(1594859418.419:110): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=x11vnc comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[   98.632348] audit: audit_lost=46 audit_rate_limit=0 audit_backlog_limit=64
[   98.632349] audit: kauditd hold queue overflow
[   98.649743] audit: type=1130 audit(1594859418.433:111): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=x11vnc comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
[   98.649745] audit: audit_lost=47 audit_rate_limit=0 audit_backlog_limit=64
[   98.649746] audit: kauditd hold queue overflow
[  100.515527] audit: type=1101 audit(1594859420.299:112): pid=843 uid=1000 auid=1000 ses=2 msg='op=PAM:accounting grantors=pam_unix,pam_permit,pam_time acct="ed" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'

That’s the better part of two seconds in the life of the box and, later on, the pace picks up. Casual searching suggests nobody else knows what’s going on, either, apart from the fact than that it obviously has something to do with systemd and, thus, is just the way things are these days.

Add audit=off to the default kernel command-line parameters by editing /etc/default/grub thusly:

GRUB_CMDLINE_LINUX_DEFAULT="quiet resume=UUID=whatever udev.log_priority=3 audit=off"

Then update the bootloader:

sudo grub-mkconfig -o /boot/grub/grub.cfg 

Whereupon dmesg becomes all quiet and (mostly) meaningful to this civilian.

Copying All! The! Files! (Except Some)

All our data files spin around on a nearly full 1 TB drive in a “file server”, a grandiosely overqualified and dirt-cheap off-lease Dell Optiplex desktop sitting in the basement. It’s been running headless and unattended for the last half-dozen years and is badly in need of replacement, so I must copy all its files to a newer, even more overqualified, and equally cheap off-lease Optiplex.

Copying the files from the /mnt/music collection on the existing server to the identically named directory on the new server proceeds thusly:

sudo mount -o ro fileserver.local:/mnt/music /mnt/nfs
sudo rsync -ahu --progress --log-file=/tmp/music.log \
 --exclude="/lost+found" \
 --exclude=".Trash*" \
 --exclude=".dtrash*" \
 --delete \
 /mnt/nfs/ /mnt/music

Mount the existing collection (from the old server) in read-only mode to avoid heartache subsequent to confusion. It could happen.

The first time through, add a -n option for a dry run, then inspect the log file for surprises.

The various --exclude options avoid copying trashed-but-not-yet-deleted files from the various trash directories maintained by various file handlers. In the process of sorting this out, I learned the DigiKam photo manager creates a .dtrash directory holding deleted files for each of its Album listings, appearing down near the bottom of the top-level album wherein you’ve quasi-deleted photos via “Move to Trash”.

The --delete option removes files on the destination (new disk) if they’re not on the source (old disk). I started this migration earlier this year, before the world fell apart, and have moved / consolidated / renamed various directories & files in the interim, so deleting the previous copies from their old locations makes the destination match the source.

So far, so good …

Manjaro Linux vs. Dell Latitude E7250 Bluetooth

Although the Dell Latitude E7250 allegedly had Bluetooth capability and the Blueman applet tried connecting to my Bluetooth headsets, the connection aways failed and nothing worked. There’s a WLAN module stuck in an M.2 socket inside the laptop providing both WiFi and Bluetooth:

Dell E7250 - DW1560 card in place
Dell E7250 – DW1560 card in place

A bit of searching suggested the driver wasn’t loading properly, which became obvious after I knew where to look:

dmesg | grep -i blue
… snippage …
[    5.678610] Bluetooth: hci0: BCM20702A1 (001.002.014) build 1572
[    5.678851] bluetooth hci0: Direct firmware load for brcm/BCM20702A1-0a5c-216f.hcd failed with error -2
[    5.678853] Bluetooth: hci0: BCM: Patch brcm/BCM20702A1-0a5c-216f.hcd not found
[   10.854607] Bluetooth: RFCOMM TTY layer initialized
[   10.854613] Bluetooth: RFCOMM socket layer initialized
[   10.854619] Bluetooth: RFCOMM ver 1.11

Without having the proper firmware / patch loaded, the module won’t work, even though the TTY / socket layers know it’s present, which explains why Blueman did everything except actually connect to the headsets.

More searching suggested you must extract the firmware HEX file from the Windows driver. Feeding the Service Tag into the Dell support site, then feeding “Bluetooth” and “Windows 8.1, 64-bit” (preinstalled on the laptop) into the Drivers & Downloads tab gets you the relevant EXE file: Dell Wireless 1550/1560 Wi-Fi and Bluetooth Driver. It turns out to be a self-extracting ZIP file (in Windows, anyway), so unzip it all by yourself:

unzip Network_Driver_5DFVH_WN32_6.30.223.262_A03.EXE

This produces a blizzard of HEX files in the newly created Drivers/production/Windows8.1-x64 directory. Each firmware HEX file is keyed to the USB Product Code identifying the unique USB gadget, found with lsusb:

lsusb
… snippage …
Bus 002 Device 003: ID 0a5c:216f Broadcom Corp. BCM20702A0 Bluetooth
… snippage …

The DW1560 apparently has a USB RAM interface, with the specific HEX file identified in the CopyList stanza of the INF file corresponding to that USB Product Code:

grep -i -A 5  ramusb216f.copylist Drivers/production/Windows8.1-x64/bcbtums-win8x64-brcm.inf
[RAMUSB216F.CopyList]
bcbtums.sys
btwampfl.sys
BCM20702A1_001.002.014.1443.1572.hex
… snippage …

However, the Linux firmware loader needs a different file format with a different name, mashed together from the HEX file, USB Vendor, and USB Product codes:

hex2hcd -o BCM20702A1-0a5c-216f.hcd BCM20702A1_001.002.014.1443.1572.hex

The converted firmware file goes where the loader expected to find it:

sudo cp BCM20702A1-0a5c-216f.hcd /lib/firmware/brcm/

Whereupon next reboot sorted things out:

dmesg | grep -i blue
[    6.024838] Bluetooth: Core ver 2.22
[    6.024868] Bluetooth: HCI device and connection manager initialized
[    6.024872] Bluetooth: HCI socket layer initialized
[    6.024874] Bluetooth: L2CAP socket layer initialized
[    6.024881] Bluetooth: SCO socket layer initialized
[    6.100796] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[    6.100800] Bluetooth: BNEP filters: protocol multicast
[    6.100804] Bluetooth: BNEP socket layer initialized
[    6.157114] Bluetooth: hci0: BCM: chip id 63
[    6.158125] Bluetooth: hci0: BCM: features 0x07
[    6.176119] Bluetooth: hci0: BCM20702A
[    6.177114] Bluetooth: hci0: BCM20702A1 (001.002.014) build 0000
[    7.031228] Bluetooth: hci0: BCM20702A1 (001.002.014) build 1572
[    7.047177] Bluetooth: hci0: DW1560 Bluetooth 4.0 LE
[   13.141854] Bluetooth: RFCOMM TTY layer initialized
[   13.141865] Bluetooth: RFCOMM socket layer initialized
[   13.141872] Bluetooth: RFCOMM ver 1.11

The firmware may be in one of the myriad Bluetooth packages not installed by default, so perhaps identifying & installing the proper package would sidestep the hocus-pocus.

Maybe next time?

Now I can wear my Bose Hearphones in Zoom sessions with the E7250, because my Pixel 3a phone heats up almost to the gets-bendy level while thrashing its battery to death.

USB Media Card Reader: Contrast Improvement

Consumer electronics designers seem to favor low- or no-contrast markings, with this USB reader falling on the vanishing end of the spectrum:

USB card reader - low-contrast slots
USB card reader – low-contrast slots

I poke the MicroSD card from the AS30V helmet camera into the smaller slot on the top surface, but, contrary to what’s revealed by the camera’s flash, the slot is a black-on-black target.

Well, I finally fixed that:

USB card reader - high-contrast slots
USB card reader – high-contrast slots

Although white tape surely would have sufficed, the roll of fluorescent red came to hand and that’s what it’ll be. The CompactFlash and Memory Stick slots on the front don’t see much traffic and have better access.

I slapped tape on case, trimmed the slots with a razor knife, and declared victory.

Much better!

PiHole with DNS-over-HTTP: Revised

More than a year later, the PiHole continues to work fine, but the process for installing the Cloudflare DoH machinery has evolved.

(And, yes, it’s supposed to be DNS-over-HTTPS. So it goes.)

To forestall link rot, the key points:

cd /tmp ;  wget https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-arm.tgz
tar -xvzf cloudflared-stable-linux-arm.tgz 
sudo cp cloudflared /usr/local/bin
sudo chmod +x /usr/local/bin/cloudflared
sudo cloudflared -v
sudo useradd -s /usr/sbin/nologin -r -M cloudflared
sudo nano /etc/default/cloudflared
----
CLOUDFLARED_OPTS=--port 5053 --upstream https://1.1.1.1/dns-query --upstream https://1.0.0.1/dns-query 
----
sudo chown cloudflared:cloudflared /etc/default/cloudflared
sudo chown cloudflared:cloudflared /usr/local/bin/cloudflared
sudo nano /etc/systemd/system/cloudflared.service
----
[Unit]
Description=cloudflared DNS over HTTPS proxy
After=syslog.target network-online.target

[Service]
Type=simple
User=cloudflared
EnvironmentFile=/etc/default/cloudflared
ExecStart=/usr/local/bin/cloudflared proxy-dns $CLOUDFLARED_OPTS
Restart=on-failure
RestartSec=10
KillMode=process

[Install]
WantedBy=multi-user.target
----
sudo systemctl enable cloudflared
sudo systemctl start cloudflared
sudo systemctl status cloudflared

Then aim PiHole’s DNS at 127.0.0.1#5053. It used to be on port #54, for whatever that’s worth.

Verify it at https://1.1.1.1/help, which should tell you DoH is in full effect.

To update the daemon, which I probably won’t remember:

wget https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-arm.tgz
tar -xvzf cloudflared-stable-linux-arm.tgz
sudo systemctl stop cloudflared
sudo cp ./cloudflared /usr/local/bin
sudo chmod +x /usr/local/bin/cloudflared
sudo systemctl start cloudflared
cloudflared -v
sudo systemctl status cloudflared

And then It Just Works … again!

Clearing The Noto Font Clutter

The Noto (“No Tofu”) font family includes nearly All. The. Languages., which is certainly a noble goal, but I’m just not ever going to need fonts like these:

./NotoSerifTelugu-Regular.ttf
./NotoSansBengali-Bold.ttf
./NotoSansGurmukhiUI-Bold.ttf
./NotoSansGurmukhi-Bold.ttf
./NotoSerifTamil-Regular.ttf
./NotoSansOriyaUI-Bold.ttf
./NotoSerifSinhala-Regular.ttf
./NotoSerifSinhala-Bold.ttf
./NotoSerifMalayalam-Bold.ttf
./NotoSansTelugu-Bold.ttf
./NotoSansAvestan-Regular.ttf
… and so forth and so on …

A bit of searching & listing identified the few I might ever use, so armor those against the coming catastrophe:

cd /usr/share/fonts/truetype/noto/
sudo chmod a-w NotoMono-Regular.ttf
sudo chmod a-w NotoSans-Bold*
sudo chmod a-w NotoSansDisplay-*
sudo chmod a-w NotoSans-Italic.ttf
sudo chmod a-w NotoSansGothic-Regular.ttf
sudo chmod a-w NotoSansMono-*
sudo chmod a-w NotoSans-Regular.ttf 
sudo chmod a-w NotoSansSymbols-*
sudo chmod a-w NotoSerif-Bold*
sudo chmod a-w NotoSerifDisplay-*
sudo chmod a-w NotoSerif-Italic.ttf 
sudo chmod a-w NotoSerif-Regular.ttf

There seems no regex-ish way of picking those out; next time, I’ll recycle the list as a script.

With armor in place, remove the rest:

find . -perm -u=w -type f -exec sudo rm '{}' \;

Rebuild the font caches:

sudo fc-cache -v -f

Maybe do such things near the end of the day, when you’re going to shut down anyway, because you’ll want to restart any programs using fonts in any nontrivial way.

Making the desired fonts read-only may confuse the next update involving the Noto fonts, but this setup (Xubuntu 18.04 LTS) is getting old and maybe something else will happen when I get around to installing a whole new release.