Introduction
I have typing-induced RSI aggravated by a fall off a bicycle. I've struggled for months with available voice-input software. I've been forced to place nearly all of my personal projects on hold, and I'm uncertain how I will perform a normal job without further degrading my hands.
I will not stand for this. Computers can do anything. My mission in 2021 is to build the best possible no-hands interface for textual work at the computer.
I'm currently seeking funding which would allow my mission to become my job. Failing that, I will at least find mentors to guide me as I create this interface myself.
Hundreds of Thousands of Injuries
It's hard to get a straight answer on how many people suffer injuries that would impair normal use of a keyboard and mouse.
For my purposes here, I'll offer two approximations of a lower bound on potential customers.
Injury Prevention (academic journal)
Global rates of bony hand trauma have decreased slightly over the last 27 years (table 1). In 2017, an estimated 178.9 (95% UI 145.8 to 216.8) age-standardised hand and wrist fractures per 100,000 individuals occurred worldwide, representing a 2.6% decrease from 1990.
(Source: https://injuryprevention.bmj.com/content/26/Supp_1/i115. "UI" here means "uncertainty interval.")
This figure, 178.9/100k people, is the age-standardized incident rate (ASIR) for the whole world. The only North America figure (from 2017) is the ASIR for bony hand trauma in "High Income North America." That ASIR is 210.7 (95% UI 167.4 to 262.8).
The US Census currently estimates US population at ~330,786,186.
Absent other information, I'll assume that the "High Income" subpopulation has bony hand trauma at the same rate as other income brackets. Multiply US population by the ASIR of 210.7/100,000 ...
(* 330786186 (/ 210.7 100000))
... and we get 696,966 -- just shy of 700,000 US citizens -- with bony hand trauma.
Bureau of Labor Statistics
In 2019, the BLS reported 339,800 cases of workplace injury to the upper extremities (shoulder, arm, wrist, and hand.) Here's an abridged version of a table from their data reporting tool. Workplace injuries alone total ~340,000 cases. How many more happen outside work?
| Characteristic | All parts of body | UPPER EXTREMITIES (code 4XXXXX) |
|---|---|---|
| 2019 | 2019 | |
Total: |
1,108,300 | 339,800 |
Sex: |
||
Men |
662,740 | 210,230 |
Women |
441,500 | 128,520 |
Nature of injury, illness: |
||
Fractures |
100,480 | 42,050 |
Sprains, strains, tears |
372,240 | 100,500 |
Amputations |
6,250 | 6,100 |
Cuts, lacerations, punctures |
100,400 | 69,120 |
Cuts, lacerations |
78,960 | 56,560 |
Punctures (except gunshot wounds) |
21,450 | 12,560 |
Bruises, contusions |
99,070 | 21,260 |
Chemical burns and corrosions |
4,740 | 840 |
Heat (thermal) burns |
16,650 | 8,920 |
Multiple traumatic injuries |
32,380 | 3,920 |
With sprains and other injuries |
17,710 | 1,710 |
With fractures and other injuries |
4,350 | 1,090 |
Soreness, pain |
212,190 | 49,210 |
Carpal tunnel syndrome |
4,630 | 4,630 |
Tendonitis |
1,670 | 1,360 |
All other |
157,600 | 31,900 |
Not Just Carpal Tunnel, Not Just RSI
Carpal Tunnel Syndrome is the most well-known repetitive motion/strain injury (RMI/RSI), but there are others, like Cubital Elbow Syndrome, or De Quervain’s Tendosynovitis (inflammation of the thumb tendons, aka "texting thumb.")
However, while RSI is frequently caused by keyboarding and mousing, many other kinds of injury can impair use of those devices.
Look again at the "Upper Extremities" column in the table from the Bureau of Labor Statistics.
9,760 heat and chemical burns. Over 6,000 amputations. Could you type with a chemical burn?
Current Software "Solutions" Aren't
Suppose you're injured to the point of being unable to use a keyboard and mouse.
"There's an app for that," right? Not so fast!
I merely have RSI -- no amputation here -- yet I get piss-poor service from existing voice-input and dictation software. Here are the problems that make existing software feel infuriatingly backward.
- voice input, but zero ability to voice edit
- no structured prose manipulation at the word/sentence/paragraph level
- lack of feedback during and after user commands
- long startup time (looking at you, macOS dictation)
- high latency between end of speech and execution of command (or appearance of dictated text)
- failure to adapt to user goals by squashing together "command" and "dictate" paradigms
- no alternate input modalities to ease strain on user's voice
My Goal
Produce the best possible no-hands interface for desktop computing, with a focus on text/document work, and targeted at non-programmers.
The UI must leap out of the screen, guiding the user at every step with context menus. Its behavior will made visible with a display for "currently hearing" and "last recognized" commands.
User will be able to choose from multiple input modalities to accommodate their impairments. This goes beyond speech: there will be gesture commands, as well as low-cost physical tokens (stickers) recognizable by the same video system that recognizes gestures.
There will always be some chance of inaccurate speech-to-text recognition, but with smart design, the frustration of inaccurate recognition can be minimized. Dictation and command modes will be separate, with their own UI affordances. Gestures form a much smaller class than the enormous space of command words, so they can be recognized with higher accuracy. Finally, constant visual feedback will keep the user aware of how the app is interpreting their actions.
One of Many Mockups
Here's a single oh-so-early mockup of some intended UI flows for prose editing.
Comparisons to Other Apps
Coming soonish. Let's just say macOS dictation is pitiful.