New telecommunications format in mixed reality with gesture recognition and translation of sign languages
Why?
Over 5% of the world’s population – or 466 million people – has disabling hearing loss (432 million adults and 34 million children). It is estimated that by 2050 over 900 million people – or one in every ten people – will have disabling hearing loss.
Toogy tries to solve isolation problems for people with hearing disabilities.
Toogy tries to solve isolation problems for people with hearing disabilities.
What?
Toogy - is a space for communication. A person can either control with gestures and communicate with voice, or control and communicate only with gestures. Toogy translates spoken words into gestures and synthesizes the voice of the gestures. This way you always speak the same language.
How?
The Toogy glasses are equipped with kinect cameras. Using gesture recognition technology Toogy translates the gestures of person with hearing disorder into voice in real time, and vice versa, gestures are synthesized from speech.
Interact (only gestures)
Your dialog partner says
This is how the person with hearing loss sees this speech
("Hello" American Sign Language )
("Hello" American Sign Language )
Interact (voice and gestures)
Your dialog partner says in sign language
("How are you?" American Sign Language )
("How are you?" American Sign Language )
This way you can hear what your
dialog partner said in sign language
dialog partner said in sign language
More about the Interface
1. Choose your location. It could be your living room, a restaurant, or on Mars. Also Toogy offers users to create their own location.
2. Upload your photos to create your virtual avatar or choose one of Toogy character.
3. Invite another person (or several) to join the meeting. Or accept the invitation if you are not the organizer.
2. Upload your photos to create your virtual avatar or choose one of Toogy character.
3. Invite another person (or several) to join the meeting. Or accept the invitation if you are not the organizer.
Basic gestures
Yes - as a nod of the head.
Examples of actions:
Yes
Confirm
Accept
Examples of actions:
Yes
Confirm
Accept
No - like closing a page.
Examples of actions:
No
Reject
Close
Examples of actions:
No
Reject
Close
Undo
Examples of actions:
Back to previous screen
Cancel
Examples of actions:
Back to previous screen
Cancel
Project for the Interface Design course, Merz Akademie.
Author: Marina Selivanova. Academic Adviser: Professor Olia Lialina.
Stuttgart, Germany, 2020
mselivanova@yahoo.com
