In a recent report in the New Scientist it emerged that Google’s DeepMind and the University of Oxford are working on a lip-reading system powered by artificial intelligence. It was reported that the AI system already outperforms professional lip-readers by a huge margin, opening doors to exciting new opportunities in consumer technology for both the hard of hearing and wider consumers.
The two organizations applied a deep learning system to a large dataset of BBC TV programs. The AI was trained using 5,000 hours of video from six TV shows that aired between January 2010 and December 2015. After this initial learning period the system was then tested on live broadcasts between March and September 2016.
In a controlled test, the AI hugely outperformed professional lip readers. The professional correctly annotated just 12.4% of the words, compared to the new system which got 46.8% of all words correct. While that may seem low it is said to be more accurate than all other automated lip-reading systems. As the system continues to learn and evolve it is hoped that the per centage will increase.
Amazing use possibilities for the hard of hearing
This system could have amazing use cases for the hard of hearing. if the system can be designed and fit into small electronic devices it could easily be amazingly useful.
Hearing aids with speech recognition
This type of lip reading system theoretically could be used to improve hearing aids. They could make a big difference to the understanding of speech in noise by dubbing conversations in real-time. At some stage in the future and with the right research and development the system could be both learning and adaptive, only working when it was needed.
Lip reading glasses
Even if the system was not right for hearing aids, the introduction of the system to glasses with a small onboard display could be a god send for people with hearing loss. It could display a transcription of the ongoing conversation in real-time that would act as a prompt to accompany the amplification of speech by hearing aids.
As this technology grows, use cases will also grow. The evolution of this and other technology is really only limited by our imagination. The integration of such systems into hearing aid technology is something that will have to be considered by the manufacturers. Alternatively, don't be surprised if we see a re-birth of a Google Glass type device delivered with this and other use cases in mind.
