Hugo Migration
I converted the site from XSLT to Hugo. If you notice something awry, please let me know. I’m hoping the change makes posting easy enough for me to do it more frequently.
Hugo felt reasonably natural to me. It isn’t that far away in concept from phppen used by the main ersoft.org, except it isn’t using PHP nor MySQL for its backing store. There were some things that needed figuring out, but that was partly because I wanted to match the existing page structure. The only strong oddity was it not having built-in Atom support.
Surprisingly it had trouble computing URLs for attachments to posts. That same
problem was a pain when implementing the XSLT-powered code. I’m happy for
layouts/_markup/, as it allows fixing the problem without shortcode, which
would have made the posts depend on Hugo. Looks like the next Hugo version will
improve the deault and I might be able to remove my custom code.
Youtube Music Playlist Dumping
Via Google Play Music, I have some old Youtube Music playlists. But there does not seem to be a way to export them into a useful form. Takeout lets you export them, but you just get video IDs and a timestamp which is not immediately helpful and for uploaded music prohibitive to make sense of.
Here’s a javascript snippet that can be pasted into the web inspector while viewing a playlist on music.youtube.com, to dump the track metadata. It exports tab-separated values.
// Scroll to bottom of playlist.
// To save results, right click and "Copy Object".
// Double-check the number of results; if entries are missing, you might need to scroll further.
// (Sometimes the number of entries simply doesn't match the playlist's count. May be limited to auto playlists.)
{
const playlistName = document.getElementsByTagName('ytmusic-detail-header-renderer')[0].__data.data.title.runs[0].text;
const playlist = document.getElementsByTagName('ytmusic-playlist-shelf-renderer')[0];
const entries = ["Video Id\tTitle\tArtist\tAlbum"];
function textFromColumnIfPresent(col) {
return col.musicResponsiveListItemFlexColumnRenderer.text.runs ? col.musicResponsiveListItemFlexColumnRenderer.text.runs[0].text : "";
}
for (const item of playlist.__data.data.contents) {
const renderer = item.musicResponsiveListItemRenderer;q
const id = renderer.playlistItemData ? renderer.playlistItemData.videoId : ""; // Track may be unavailable
const title = textFromColumnIfPresent(renderer.flexColumns[0]);
const artist = textFromColumnIfPresent(renderer.flexColumns[1]);
const album = textFromColumnIfPresent(renderer.flexColumns[2]);
entries.push(id + "\t" + title + "\t" + artist + "\t" + album);
}
console.log("Found " + (entries.length-1) + " tracks");
console.log("Playlist: " + playlistName);
console.log(entries.join('\n'));
}
Vector 4 Technical Information
I recently came into possession of the Vector 4 Technical Information (P/N 7200-0001) by Vector Graphic Inc, and it is a gold mine. The Vector 4 User’s Manual and Vector 4 Programmer’s Guide seemed to have been standard issue so my family had multiple copies (one per computer), and they are already digitized by bitsavers. But the Technical Information seems to be much rarer, and seems to be oriented toward maintenance shops. But that means it has very detailed information including jumpers, clock timing, and schematics, with lots of text explaining it all. I’ve found it to be extremely helpful for working on my Vector 4.
I’ve happily digitized my copy. The schematics at the end are ledger-size pages (17" x 11") which made them difficult to scan, and unfortunately I was only able to scan them at 400 dpi. They are usable, but the originals are easier to read.
Vector 4 Keyboard Adapter
I was working on an MFM decoder for PulseView to help find bugs I introduced in FlashFloppy. That was working pretty well, but I needed some help keeping count of the current track. Using the Counter decoder on STP signal (step pulse) was pretty close to what I needed, but it wasn’t quite good enough. So I read through all the decoders. And I came across UART.
UART. Wait… Earlier when I was investigating emulating a Vector 4 keyboard I knew from past experience the signal looked pretty similar to serial (RS-232), but it wasn’t quite the same so I thought it’d be a pain to figure out how to configure some hardware appropriately. “Bit banging will be so easy!” (And then it wasn’t; but it was fun.) I hadn’t considered whether it might just be a plain UART. A quick look on Wikipedia confirms, it is totally a plain, ordinary UART.
I plug a USB to UART TTL adapter into my computer and attach the UART to the
keyboard port, fire up miniterm.py /dev/ttyUSB0 300 and… it just
works. Dang, that means it is also trivial to use the UART on the Raspberry Pi
or the
UART on Raspberry Pi Pico with Micropython
or a stm32f1 Blue Pill.
In retrospect, it should have been obvious. The Vector 4 User’s Manual even says as much:
Let’s make an adapter
My real goal is to use a modern keyboard with the Vector 4. I can already do that with USB UART and miniterm, but that requires my computer. It’d be easy to use a Raspberry Pi Zero instead, but it is an overkill and would need to boot each power-on. A microcontroller is ideal if it is easy.
There’s two options for reading from a “modern” keyboard: PS/2 and USB. PS/2 has separate data and clock and doesn’t look too bad, although it is 5V. USB requires USB host support, so the Blue Pill wouldn’t work but the Pico would work with USB On-The-Go. And I already have an adapter cable.
Some searching and I find TinyUSB has a keyboard example. And the example outputs to a terminal via a UART! All I need to do is disable some debug printfs and swap to 300 baud, and build for the Raspberry Pi Pico.
tar xf tinyusb-0.11.0.tar.gz
vim hw/bsp/board.h # change CFG_BOARD_UART_BAUDRATE to 300
cd examples/host/cdc_msc_hid
vim src/hid_app.c src/main.c # disable printfs
mkdir build
cmake -DFAMILY=rp2040 -DBOARD=raspberry_pi_pico .. # Seems fine without -DBOARD
make -j
# copy cdc_msc_hid.uf2 to Pico
The logic analyzer confirms it is working, so now I need a physical adapter. I grab a 6P6C mount I happen to have, solder wires directly to it, and attach it to the Pico. The Vector provides 5V, which is perfect for using USB OTG and powering the Pico. (Power+ground on the right. TX on the left, with RX floating because it is 5V.)
Testing shows it works with the Vector, so then I enhance it to behave more like the original keyboard. Since the last post, I have now obtained the Vector 4 Technical Information (P/N 7200-0001) which provides all the keycodes. I also get rid of the blinking LED and have the LED signal key presses instead. At some point I’ll need to implement auto-repeat when holding down a key, but the patch is already usable.
Vector 4 Keyboard
I have a Vector 4 manufactured by Vector Graphic, Inc. It’s an old machine from the 80s that happily still runs. Or at least one out of two I have still runs.
I’ve been on a long escapade trying to preserve the device as there’s very little information archived for it online. This has included adding hard sector and Micropolis support to FluxEngine and hard sector support and async I/O handling to FlashFloppy.
In 2020 I was trying to use the machine and the keyboard wouldn’t work. The keyboard has foam pads with one conductive side and the foam had disintegrated. Thankfully other keyboards from the era, much more common than mine, have suffered the same fate and TexElec sells replacements. They are a bit finiky so some keys won’t work and I have to reopen the keyboard (with over 20 screws) to fiddle with the pads. But all the keys are currently working. Great.
But the keyboard isn’t that great to type on. It works, but pales in comparison to cheap keyboards today. How hard would it be to make an adapter to use a “modern” USB or PS/2 keyboard? The keyboard’s operation isn’t documented anywhere to my knowledge, but I figured it’d be quick to reverse engineer.
The keyboard connects to the computer with a 6P6C connector, and a board in the machine has helpful labels. The 8P8C connector is on the bottom left; ignore the pin numbers, except pin one.
Following the traces, we get:
| Pin | Assignment |
|---|---|
| 1 | GND |
| 2 | GND |
| 3 | GND |
| 4 | +5V |
| 5 | S OUT |
| 6 | S IN |
“S” would clearly mean serial, and there’s no separate clock. Maybe it’ll be like RS-232. Now let’s take a look with a logic analyzer.
Typing is at least mostly a one-way protocol. Pin is high during idle. Comparing for more button presses made the encoding clear. 0 is low, 1 is high. The bit rate is 300 Hz. It starts with a 0 bit, has 8 data bits with LSB first, and stops with a 1 bit. Data bits are ASCII characters. When holding down a key, presses are 71 ms apart. There’s no obvious source of queuing, so it’d be hard to have two letters nearer each other. My key presses are buffered, so I guess it is queued on the computer and not the keyboard. I’m not aware of an easy way to overflow that buffer as it seems to be greater than ~30 characters large. Wow. That was easy.
Now can I make a keyboard emulator? I figured I’d start with a Raspberry Pi Zero W to make entering text easy over wireless. Python should be easy, and 3ms timings seem like something it could probably handle. A quick script later and… it doesn’t work. Looking at the logic analyzer shows really bad timing. Oh well… let’s port it to Go. Surely a quick go program will work. Nope. Still poor timing. Much better. But not good enough. Busy looping and nice levels didn’t help. Shucks.
Well, I was wanting a project to use a Raspberry Pi Pico. It has micropython; I wonder what that’s like. A quick port to micropython and… it works! The computer displays “KBD error”, but the keyboard works regardless.
What’s the point of S OUT? Just after cold boot starts the computer sends 0x08 (BS?). But there’s no response. Later the computer sends 0x10 (DLE? surely not), the keyboard replies 0xAA, and the computer sends 0x02 (STX?). That exchange seems to be the KBD test and can be triggered separately with a ‘T’ Test mode instead of the normal HDD or FDD bootup. The keyboard beeps during this exchange. After pressing ‘W’ to choose to boot from the “Winchester” HDD the computer sends 0x02 (STX?). Also no response. Seems S OUT data is some custom meaning and just uses powers of two. The 0xAA response has alternating bits so is good for testing.
The Raspberry Pi Pico isn’t 5V tolerant, so I won’t bother any more with S OUT. USB or PS/2 can be for some other day, but it is quite feasible.
In a follow-up post I make the adapter.