deskthority

Which firmware to use?

In this thread I begin take you through a guided tour of the prevailing keyboard firmwares on the market today. I focus on open source efforts with some exceptions. I discuss pros and cons, uses, supported targets, architectural decisions, and so on. These are vaguely sorted in order of oldest to youngest and simplest to most complex. Let's jump right in!

For starters it's helpful to lay out some definitions. Feel free to skip this, I'm laying it out for lack of ambiguity and for newcomers.
First a brief section on microcontroller choices. Maybe skip this section if you're already familiar with AVR 8 bit and ARM Cortex M0 MCUs.
A little bit of controller theory follows. Skip this if you're familiar with the software architecture of a controller.
What are the things that I look for when evaluating a keyboard firmware?

Well, I'm a hobbyist first and foremost, but I always have mass market production on the back of my mind. So I'm looking for a few things:

- How hard is it to define a custom layout?
- How active is the community for this firmware - and how likely am I to get a response if I have a problem?
- How much does it cost? (Ideally: free.)
- How annoying or expensive is it to procure hardware that can run this firmware?
- How is ghost key blocking supported? (Watch for a future post on ghosting based on some notes I have.)
- How are macros, layers, and remaps supported?
- How much configuration can be done on the keyboard, versus "interactively" in a text editor or using a formal GUI or CLI tool?
- Are exotic hardware features like backlighting, a built-in screen, a rotary encoder, a split, or a solenoid supported?
- Are exotic software features like complex layers (e.g. SpaceFn, http://geekhack.org/index.php?topic=51069.0), settings banks, and (when applicable) per-switch actuation supported?
- How "familiar" is the software in terms of architecture and layout? How tough is it to tease out something like the psuodocode I wrote above?
- What does the workflow look like for a Linux or Mac user?
- Intangibly, how much do I like it?

Empirically, my goals are to support two keyboards. One is the Northgate Omnikey with NKRO, my absolute favorite keyboard hacker board. IT's easy to find, built like a tank, has full NKRO, the matrix is actually documented, it has fun hidden DIP switches, and the solder pads are huge and an absolute delight to work with. The other is the XMIT Hall Effect keyboard, which I'm working to make modern with a newer firmware for previous customers.

Let's begin with the classics: Soarer's Converter and Soarer's Controller.

Converter: viewtopic.php?t=2510
Controller: viewtopic.php?t=6767
WYSE Converter: viewtopic.php?t=7424

These are the controllers and converters I cut my teeth on. They're easy to use and they work. They're also, sadly, closed source. I like the tools that come with them but they're pretty basic. Soarer's Converter is great for converting AT to USB in a pinch.

I ran out of time, so, that's all for now! In the coming little while I'll follow up with investigations for:

xwhatsit
CommonSense
Kaleidoscope by keyboard.io
Kiibohd by Input.Club
TMK by Hasu
QMK
Das Keyboard Q

Reserved #1.

Reserved #2.

Go look at the original post! Reserved #3.

Nice idea to write these up as a guide. For newcomers sake, I’d highlight how these actually appear in use, too. Even as a veteran keyboard remapper, I happen to prefer a GUI to a text tool: so my faves are Unimap (formerly gnarly TMK now neatly hidden behind the scenes) and the very visual friendly Xwhatsit.

Speaking of Soarer specifically, his controller / converter is a hybrid of open and closed source. He said he would likely open source the whole thing before he disappeared (and there is a reverse engineered source that someone made and gave me, but refused to open source without Soarer’s still absent personal permission). The current state of affairs is the converter hex file itself is (assumed) closed source while the tools users use (scas and scwr etc.) have always been open source, and I even compiled them for another architecture once myself.

You mention the user experience on the Mac. Now that Apple’s killed 32 bit, Soarer’s tools are dead. I need to recompile them for Intel 64 bit sometime, as I did for PowerPC back in the day. Depending on dependencies, I don’t know if this will work, though. PowerPC was a bit of a fight, and never worked on anything but Leopard.

Also broken in the 32-bitpocalypse: PJRC's Teensy Loader. Fortunately, there's an altogether better tool nowadays for all that: the very helpful QMK Toolbox. My TMK workflow is entirely source code and indeed terminal free thanks to Unimap giving me a GUI front end and the resulting .hex file, which then goes through QMK Toolbox to the Teensy.

Your explanation and code snippet of the scan logic is so succinct and easily digested.

Looking at the TMK code initially intimidated me but I had a moment of clarity once I realized that it's just been architected to do a lot of different things. Once I got past that, working with it and tweaking to my own needs suddenly became a doable thing!

Kiibohd always interested me as well but I never dipped my toe into that pool.

I look forward to seeing where you go with this post. You, in particular, have a unique perspective on the topic having worked on a commercially available product.

Now that Apple’s killed 32 bit, Soarer’s tools are dead.

Ugh. This further validates my refusal to "upgrade" from 10.14 while I have important projects afoot.

Smart move. Catalina's a right pain in the jacksie in several respects. But you can expect no remorse from Cupertino! 32 bit is as dead as Classic.

So, I tried to build Soarer's tools from source. Their source code is in the tools folder within Soarer's zip file. After figuring out just where exactly Apple moved the SDKs…

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include

…and replacing i386 with x86_64 throughout, make is still barfing volumes. Apparently there is more to this 32 to 64 bit lark than I understand! See if a bona fide coder like yourself can pull it off. We could all use scas and scwr and friends in 64 bit.

Very interested in seeing more! Thanks for working on this, very helpful idea to put a run down on things in one place.

Apparently there is more to this 32 to 64 bit lark than I understand!

I'll see if I can take a look though this is pretty low on my list. Arch changes tend to bring out integer size bugs. Anyone tried building these on Linux x86_64? Getting it up there would be easier.

I am surprised nobody has mentioned Jorn and his wonderful viewtopic.php?t=17388 BL-USB Bluetooth Model M Controller Conversions. This solves the issue right at the root instead of trying to adapt.

Those are just a product, though, rather than a project. And only work for one model line of keyboards: IBM Model M.

The controllers (yes we are also talking “root” here) worth discussing are general purpose and ideally open hardware and open source.

XMIT wrote: 23 Nov 2019, 04:02 Which firmware to use?

Chris' firmware: https://github.com/chrisandreae/keyboard-firmware
or my branch of it: https://github.com/hercek/keyboard-firmware

Features:

• Almost no community. Chris did respond to my questions when I had any. I can respond to questions about my branch.
• Uses LUFA or vusb (so MCUs which are supoorted by it, e.g. atmega, atxmega). You need to add your backend. Provided backends are for ergodox, Chris' kinesis advantage custom controller, Katy custom controller. You need to port it for other HW.
• Custom layout can be modified by an GUI app or on the fly using Program key (as Kinesiss Advantage).
• Open source.
• HW is hard to produce (if you consider Ergodox building hard).
• Remaps over GUI or through Program key. Macros over GUI or through Program key. Programs over GUI only. Programs are a small interpreted pieces of code in a C-like language which you compile to a p-code and load to firmware and associated with a trigger using an GUI application. All these things do not need firmware re-flash.
• Mouse keys support. No unlimited sticky keys or dual function key support. Two layers (normal and keypad) or three layers in my branch for K84CS (normal, keypad, function).
• Maximum number of macro triggers and macro storage (taken by all macro definitions together), and program storage limited by a compile time constant. At most 6 programs are possible. Storage locations are eeprom, spi-eeprom, i2c-eeprom, flash (not sure this one works well on the fly).
• GUI can do everything except definition of macro sequences themselves. Program key (on the fly) definition can do everything except program definitions and trigger assignment to programs.
• No backlight support. LCD support but only for Katy.
• No pseudocode compiler. There is compiler for custom very slashed down C-like program definition language (e.g. it support emitting keys, checking key presses, precise delay, mouse keys). A new HW suport must be added by implementing an interface for the HW control in C.
• Workflow: mostly as Kinesis Advantage with perks (macros, GUI, ...).
• The GUI application is in QT and compiles at least for Windows and Linux. It probably will compile on Mac ... with some tweaking. One needs to instal also libusb driver for Windows.
• 6KRO only. I did not see the allure of NKRO but it can be added by a C coder. The matrix scanning is more or less NKRO. Only the protocol is the bios keyboard (6KRO).
• Some special features like LCD, on-the-fly debounce length configuration and mouse keys speed configuration in my branch.
• Probably not useful for people who are clueless about C if they are not going to use it with the exactly supported keyboards.
• You probably never heard about it so you do not have an opinion .... yet

Oh cool, I'll definitely take a look!

SDS604 wrote: 27 Nov 2019, 09:53 I am surprised nobody has mentioned Jorn and his wonderful viewtopic.php?t=17388 BL-USB Bluetooth Model M Controller Conversions. This solves the issue right at the root instead of trying to adapt.

Oh, this project happened while I took some time away from keyboards. I'll have a look at this one too.

If anyone else has firmwares to recommend please continue to update the list. Though the Omnikey and the XMIT Hall Effect are my top two development target boards, it's fair to say that the venerable Model M is #3.

For the future, I think Circuit Python may be an interesting place to put bets. Adafruit is making boards close to the right form factor and price. And she has people helping to make it a usable ecosystem.

For keyboards, there is someone on Tindie selling a macropad. The scan loop and other software beyond the standard libraries looks to still be pretty primitive. But, in the end, I think this may all prove more approachable.

PiKeeb, though, seems like it's too much and so ends up being unsure what it's supposed to be.