This sounds promising -- would love to see this become more practical.
With wearable devices becoming more common - I am anticipating a wave of "sensors" that can be as simple as small band-aid patches that wirelessly send data to your smart device. Those sensors could also open up human-coputer-interface innovations like these.
---
In similar space ...
There was a post about a thought-to-text project from MIT no less --
When I saw the demo posted last year it left me with an uneasy feeling -- gut feel said it was more marketing than a real working technology demo. Nothing seems to have come out of that lab since then -- strengthemning my suspicions.
I remember reading about something like this tech in one of Asimov's Robot novels. Thought about trying to build something like this, but don't really have the skills. Awesome they did build this! I guess if they make it easy to train on your speech (just combine it with speech to text), it's a product waiting to happen!
The use of ultrasound is interesting! The first step (identifying speech) is called subvocalization recognition, and early research (Chuck Jorgensen, et. al (2023) from NASA) use EMG[1].
You move your mouth and tongue as if you are gonna make the sounds, just don't exhale the air. They have a demonstration video in the blog post. As somebody else have already pointed out, it's a great tool for interacting with always-on camera devices like smart assistants, makes it a bit less weird.
Really impressive. Seems like this could help people with health problems like ALS or Parkinson's, if you can discern clear intent from their tongue muscles. Or just help people who are intubated in the hospital.
Very cool! While this is a different type of inverse problem, it reminds me of a radiolab story [0] about a device to help blind people “see” using electrical impulses on a metal strip under the tongue.
I'm very eager for subvocalization tech to take off - this, AlterEgo, etc.
I have never liked talking aloud to Siri or similar, but I could see using "voice" as an interface for so much more if I could speak silently to my device.
> The two biggest hardware challenges are reducing the size and weight of the ultrasound probe and replacing ultrasound gel with a more practical coupling material, such as hydrogel. We think both are solvable, making it possible for the probe to eventually become a lightweight wearable or adhesive patch.
Not sure I'd want to put an adhesive patch on my neck every morning so I can silently talk to an LLM in the cubicle farm. I hope this is not our future.
Very cool tech though and surprisingly good results for so little training.
I think time might be better spent improving a lip reading model (no adhesive required), assuming we're unable to read brainwaves directly.
Sony presented a simpler solution to this at CHI https://dl.acm.org/doi/10.1145/3772318.3791397 - check video in that. Also it seems every Cornell student is working on this. Just search on YouTube Cornell silent speech
I wonder if, like with lip reading, they switch from American English to a different language that's not so peculiar they would have and much less error rate
I suppose Spanish or German would be easier to recognize than English, due to the more distinct sounds, but Chinese would be much harder, due to the tones.
Wonderful tech, and video example. I think there may also be a special forces application, but I don't know enough about how well their current solution works.
In the office, a non-contact video solution (lip reading) is likely to be far more popular, but a lot depends on which is more accurate.
This sounds promising -- would love to see this become more practical.
With wearable devices becoming more common - I am anticipating a wave of "sensors" that can be as simple as small band-aid patches that wirelessly send data to your smart device. Those sensors could also open up human-coputer-interface innovations like these.
---
In similar space ...
There was a post about a thought-to-text project from MIT no less --
"AlterEgo"
10 months ago: https://news.ycombinator.com/item?id=45174125 8 years ago: https://news.ycombinator.com/item?id=16780357
When I saw the demo posted last year it left me with an uneasy feeling -- gut feel said it was more marketing than a real working technology demo. Nothing seems to have come out of that lab since then -- strengthemning my suspicions.
I remember reading about something like this tech in one of Asimov's Robot novels. Thought about trying to build something like this, but don't really have the skills. Awesome they did build this! I guess if they make it easy to train on your speech (just combine it with speech to text), it's a product waiting to happen!
The use of ultrasound is interesting! The first step (identifying speech) is called subvocalization recognition, and early research (Chuck Jorgensen, et. al (2023) from NASA) use EMG[1].
[1]: https://scispace.com/pdf/sub-auditory-speech-recognition-bas...
s/2023/2003/
I'm having trouble imagining how one would use this. How do you speak without making sounds?
You move your mouth and tongue as if you are gonna make the sounds, just don't exhale the air. They have a demonstration video in the blog post. As somebody else have already pointed out, it's a great tool for interacting with always-on camera devices like smart assistants, makes it a bit less weird.
Really impressive. Seems like this could help people with health problems like ALS or Parkinson's, if you can discern clear intent from their tongue muscles. Or just help people who are intubated in the hospital.
Very cool! While this is a different type of inverse problem, it reminds me of a radiolab story [0] about a device to help blind people “see” using electrical impulses on a metal strip under the tongue.
[0] https://radiolab.org/podcast/seeing-tongues
I'm very eager for subvocalization tech to take off - this, AlterEgo, etc.
I have never liked talking aloud to Siri or similar, but I could see using "voice" as an interface for so much more if I could speak silently to my device.
I remember a story decades ago about "subvocal" speech, similar to this: https://spacenews.com/nasa-develops-system-to-computerize-si...
> The two biggest hardware challenges are reducing the size and weight of the ultrasound probe and replacing ultrasound gel with a more practical coupling material, such as hydrogel. We think both are solvable, making it possible for the probe to eventually become a lightweight wearable or adhesive patch.
Not sure I'd want to put an adhesive patch on my neck every morning so I can silently talk to an LLM in the cubicle farm. I hope this is not our future.
Very cool tech though and surprisingly good results for so little training.
I think time might be better spent improving a lip reading model (no adhesive required), assuming we're unable to read brainwaves directly.
Keeping a camera focused on the lips could get awkward. A hands free throat patch may be more ergonomic.
That could be immensely helpful for people who cannot speak due to vocal chord problems.
It could also be the ultimate, always-on remote control for everything around, with a near-zero error rate.
Sony presented a simpler solution to this at CHI https://dl.acm.org/doi/10.1145/3772318.3791397 - check video in that. Also it seems every Cornell student is working on this. Just search on YouTube Cornell silent speech
Interesting. This seems similar to (though using different methods/techniques) what q.ai (that was sold to Apple for $2b) is developing.
See this hn thread about it: https://news.ycombinator.com/item?id=46816228
I had thought of this, so nice to see it done!
Straight out of Life Artificial...
These guys are on a roll!
I find it annoying to try and talk completely silently. I'm always whispering at least a bit. I can't imagine doing this for the entirety of a call.
I wonder if, like with lip reading, they switch from American English to a different language that's not so peculiar they would have and much less error rate
I suppose Spanish or German would be easier to recognize than English, due to the more distinct sounds, but Chinese would be much harder, due to the tones.
reminds me of the handwriting recognition language Graffiti from palm/handspring days.
It ended up altering my handwriting even after I stopped using it.
Wonderful tech, and video example. I think there may also be a special forces application, but I don't know enough about how well their current solution works.
In the office, a non-contact video solution (lip reading) is likely to be far more popular, but a lot depends on which is more accurate.