Improving speaking ability of hearing loss disablities with bone conduction and diaphargm voice training techniques

Resumo

Hearing loss is one of the major health problems with 422 million people affected worldwide. This project dealt with inventions of a hearing aid device and voice training technique to help deaf people to speak with more power. The project was based on two hypotheses: firstly, that deaf people can hear  sound using a bone conduction hearing aid device named EarZ and secondly, that the Diaphragm Voice Training Exercise (DVTE) can help improve speaking ability (power) of deaf people.


In the project, I created a homemade bone conduction hearing aid device, called EarZ, and developed the sounds to test EarZ's feedback on deaf volunteers. Then I applied the diaphragm voice training exercise (DVTE) to test seven deaf volunteers and rated for their speaking ability. Results show that the minimal hearing loss (40-75dB) volunteers heard the low-mid frequency sound using EarZ with the average hearing score of 9.5 from 10. Meanwhile, the volunteers with severe (80-90 dB) and profound (90-120dB) hearing loss heard the same sounds with less score of 5.5 and 4.8. When we applied DVTE training, the volunteers were able to increase the vocal power from 65.5% to 90.0%, which was a significant improvement  by 24.5%. The volunteers could now pronounce the words with more clearer and powerful voice.


The valuable findings of this project are that the hearing ability of deaf people can use the EarZ device and DVTE technique to improve their vocal power when speaking to lead a better life by having improved voice communication.

Pergunta / proposta

The main problem that I am investigating is how to improve speaking ability of deaf people. I wonder if I can use science, technology and art combined together to solve the problem. I accidentally discovered the sound conduction through a jaw-bone while playing electric guitar without an amplifier connected and noting the vibrations produced. My first question is, can deaf people hear a sound through a jaw-bone directly to the cochlea? If so, can I create a bone conduction device that could help deaf people to hear?

There are 466 million people worldwide who suffer from hearing loss (WHO, 2018). Thus, the success on innovating a low-cost bone conduction hearing aid device can make a large impact on helping the world hearing loss society to live a better life.  After a visit to the deaf training school in Chiang mai, Thailand, I noticed that hearing loss students had problems with their pronunciation and lacked power when speaking. Their short inhale and exhale breathing is different to that from normal people. Thus second question is that, can I use my singing skills to help them speak better? Is it possible to use my diaphragm voice training method to help correct their breathing cycle and improve pronunciation and vocal power?

Thus my proposal is based on two hypotheses; firstly, deaf people can improve hearing sensing the sound using bone conduction hearing aids device, and secondly, the diaphragm voice training exercise can help improve speaking ability of deaf people, in particular their power.

Pesquisa

Bone Conduction

After researching, I found that bone conduction effect was first documented to be discovered by Beethoven around the 18th century while he put his teeth on the piano and was able to hear sound from the piano. More recently,  Jonah Kohn (2013) entered the Google Science Fair with his discovery on bone conduction through his guitar practice, similar to my discovery. Kohn (2013) developed his experiment on hearing-aids device by using the tactile sound through the fingers of the volunteers. However, Kohn’s experiment did not explain the application on how to help the deaf people. Even though my work is based on a similar discovery using the bone conduction effect, my proposal focuses on different applications and experimental approaches than that of Kohn’s. My homemade bone conduction earphone, EarZ, was designed using different technology from the commercial ones such as AfterShock earphone as well.  


Diaphragm Method

Diaphragm voice training method is a basic voice training for beginning singer to help gain more vocal power. After a visit to deaf training school in Chiang mai, Thailand, I found that the school owns a 15 year old speaking training software on a PC use for hearing disabilities. My software only measures the loudness but not on the quality of voice or pronunciation. There is no evidence of other commercially available products that incorporates the diaphragm voice training to help improve the ability of the people who are deaf to speak.

The above research help me explore the previous works, leading to better formation of my project on EarZ hearing aid device. Also the research on the DVTE and current deaf training software lead me to think further how to help deaf people speak with more vocal power because when deaf people speak, they don’t have much vocal power due to an incorrect breathing cycle.

Método / Testes

My experiment was divided into two parts. The first part was to prove that EarZ, a hearing-aids prototype, can help deaf people hear sound. The second part  to prove that voice training using the diaphragm method with training software and EarZ device can help hearing disabled people to improve their speaking and pronunciation.

1. Testing on the EarZ hearing aids device

1.1) Together, with the help and support of my father as a coach, I developed the EarZ prototype which applied the bone conduction concept.

Figure 1 shows the timeline of EarZ development. The EarZ prototype consists of two main components, a homemade bone conduction speaker and bluetooth circuit receiver. The home-made bone conduction speaker is wired to the receiver. The receiver is connected to an electrical circuit to allow sound input from a microphone or from mobile devices by bluetooth such as music or a test MIDI voice.

1.2) I tested the EarZ prototype on 7 deaf volunteers at Sodsuksa Anusansunthorn Deaf Training School, Chiang Mai, Thailand. Volunteers were divided into groups depending on their degree of hearing loss ranging from 25-40, 40-75, 80-90, and 90-120 dB. For each group, they listened to bass notes, treble notes, drum beats, and environmental sounds. To cover all ranges of human hearing frequency, the bass notes were created in a range of low-mid frequency from 24.5 to 196 Hz while the treble note was created in a range of mid-high frequency from 392 to 3135.96 Hz. The hearing test was repeated three times.

1.3) Data were collected from self-report of volunteer's ability to hear the different sounds using a scale from 0-10 (Figure2, Figure3, Figure4).

2. Testing on the Diapgragm voice training exercise

2.1) Using my knowledge of the diaphragm voice training exercise (DVTE) from my singing lessons, I developed the DVTE videos shown in  Clip1 and Clip2.

With the help of a software developer, Nont from Maejo University, DVTE software based on Google Chrome and speech recognition was developed to help participants use this technique to improve their own vocal power. The first version of DVTE with EarZ device was then pre-tested with a deaf volunteer.  

2.2) After pre-test, the team discussed and refined the first DVTE software to produce a second version renamed, EZ-Speak as shown in Figure5.

 

2.3) I tested seven deaf volunteers on speaking ability using the EZ-Speak software based on the DVTE. I recorded the data on my datasheet, before and after the diaphragm voice training to show whether or not this technique helped them to gain vocal power on their speech. An example of the datasheet is shown in Figure6.

In data collection sheet, each tick mark representing each speech on a distict word. Each tick mark was incrementally rated as Good=10, Medium=6, and Low=2. For each word, I averaged the 3 speeches of each volunteer on their ability to produce six distinct words. The average of each rating was multiplied by 10 to obtain the percentage of speaking ability.

Resultados

1. Testing on the EarZ hearing-aids device

I started the EarZ pre-test with a few deaf volunteers. One of the volunteers was almost fully deaf at a level of 80dB. When she used EarZ, she heard the sound as her teacher spoke through EarZ microphone. I was happily surprised to observe her repeating the words correctly without reading the teacher's lip as shown in Video1. The second volunteer whose deafness was about 40dB did the pre-test by listening to the music and media via a bluetooth communication as shown in figure  He was impressed by EarZ headphone as the sound was softer to listen and more comfortable to wear as compared to the normal plug-in hearing-aids headphone. In both cases, EarZ was proved to be beneficial for hearing loss disabilities with a surprisingly impressive delivery of sound and comfort.

 

The next EarZ test was done with data collection on hearing rate of deaf volunteers with different sounds. On Graph1, the 40-75dB deaf volunteers heard the low-mid frequency sound (48-196 Hz) using EarZ with the highest average hearing rate of 9.5, not including the error data at 24.5 Hz. On the other hand, the deaf volunteers with severe (80-90 dB) and profound (90-120dB) hearing loss heard the same sounds with less hearing rate at 5.5 and 4.8. I observed the same trend of hearing rate on the mid-high frequency sounds at 392 - 3136 Hz (Graph2), drumbeat sounds (Graph3) and environment sounds (Graph4). The full set of raw data is documented in File EarZ1.

 

2. Testing on the Diaphragm method

The voice training was done with data collection before and after using the DVTE method and EarZ. The video clips show the test of volunteer before (Video2), during (Video3) and after (Video4) DVTE training. On Graph 5, before the volunteers learned how to use their diaphragm, they had the average vocal power rating of 65.5%. After the volunteers learned how to activate their diaphragm, their the average vocal power rate increased to 90.0%, which is an improvement of 24.5% of their vocal power.

 

 

 

From my discussion with the volunteer's teacher, we were very excited to see a large 24.5% improvement on a volunteer’s pronunciation and vocal power. With the diaphragm technique, the volunteer gained a better and longer breathing cycle. As a result, they could pronounce word correctly with more vocal power. In our opinion,  if the training software is to be further developed, the DVTE should be put as a standard basic method to help hearing-loss patients to improve their vocal power. It can be finally stated that the Diaphragm Voice Training Exercise on deaf-speak training was the second biggest discovery in this science project. The full set of raw data collection on DVTE training is documented in File DVTE1.

Conclusão

My first discovery on bone conduction through jaw-bone inspired me to invent the home-made EarZ hearing-aids device. My data show that the EarZ device can be used to help deaf people to hear sounds better. Deaf volunteers with hearing loss level of 40-75 dB can use EarZ effectively but the severe (80-90 db) and profound (90-120 dB) hearing loss groups were able to hear the sounds less effectively. The EarZ device offered the sound softer and smoother than the sound produce from the normal plug-in hearing-aids device.

My second discovery was the application of DVTE with EZ-Speak software to improve speaking ability of deaf people from 65.5% to 90.0%, increasing by 24.5%. Before DVTE, deaf volunteers had short inhale and exhale breathing cycle, causing the incorrect pronunciation and no vocal power. After DVTE training, deaf volunteers can speak with the right breathing cycle. As a result, their breathing cycle was longer with more vocal power and word correction. It is recommended that DVTE should be used as a Based-Practice method for vocal training for deaf people. This findings will be shared with the school for hearing impairment in Chiang mai to improve their training exercise for deaf students.

Besides the discoveries and invention above, I found the project can give several benefits to disability society below:

  • EarZ device can be further developed as a low cost hearing-aid device that can be accessed by low-income disabilities worldwide.
  • EarZ device can be used not only for vocal training, but also any media such as music, movie and YouTube available in today' s world. 
  • EZ-Speak can apply features available on Chrome and Google, such as, Menu Translation, which can be effortlessly used as a Universal Language and Speech Recognition.
  • EZ-Speak can apply Big data and AI to adjust learning ability of deaf user in the future.

Limitations of the project

  • Skull structure affects the position where EarZ speaker is placed onto and delivers the sound effectively. The study of human skull and design to optimize the best fit are needed in the future.
  • For the EZ-Speak software, I made the reference based on my voice only. People have different voice tones. More voice references such as voices of high, low, male and female tones are needed for more recognition references.  

The two most valuable points of this project are that 1) our deaf volunteers can hear the sound from EarZ effectively; and 2) their speaking ability with vocal power is significantly improved. Both EarZ device and DVTE technique can be used to help the world of hearing loss disability to lead a better life by having better voice communication. If these two technologies are presented to the world, the number of deaf people will use less sign language, but will prefer to speak in the normal language instead. People with hearing loss would have the chance to live normally in our society and be normal like everyone else.

 

Sobre mim

I love playing guitar, singing, producing and composing music but also thinking of something innovatives. I have a perfect pitch with high skill in singing. I used to build the blind helping toy called UNBLINDER and Formular One Model from LEGO NXT Robot in 2014.  In 2017, I discovered the bone conduction while playing my guitar, which made me think of deaf people first. My instinct told me that we could help deaf people to live better lives.

I have been inspired by lots of innovators and scientists such as Steve Jobs, Bill Gates, and Brian May. Steve Jobs is my most inspiring person who strived to accomplish and never gave up on every work. He inspires me on entrepreneurship which I did not have before. This made me want to develop EarZ further to help poor deaf people in Thailand and around the world because poorer families have harder time supporting/accepting their deaf children. As a result, children do not receive support at the right time.

In the future, I want to study at the medical school at Stanford University. I want to be an inventor that combines science and music to create medical technologies which can help deaf people hear and speak better.

Winning Google Science Fair 2018 will mean very much to me. I wish to get funding for my future education and invention. My dream is to be a maker of EarZ and EZ-Speak at low-cost for the poors in society who suffer from hearing loss.

Saúde e segurança

There are no concerns about the health issue because the hearing aids is uses bone conduction which sends vibration through skull and then it vibrates the hammer, anvil and stirrup and sends the vibration to the cochlea, therefore, there are no health issues about this topic. 

Bibliografia, referências e agradecimentos

Bone Conduction: Beethoven

http://www.goldendance.co.jp/English/boneconduct/01.html

●    The Human Auditory System http://www.yamahaproaudio.com/global/en/training_support/selftraining/audio_quality/chapter4/01_ear_anatomy/

●    World Health Organization

http://www.who.int/pbd/deafness/estimates/en/

●    The human auditory system 2017-2023 Hearing Aid Devices market of grow at a CAGR of 5.1% during the forecast period

https://www.reuters.com/brandfeatures/venture-capital/article?id=11672

●    Benefits and Risks of Cochlear Implants

https://www.fda.gov/MedicalDevices/ProductsandMedicalProcedures/ImplantsandProsthetics/CochlearImplants/ucm062843.htm

●    Grand View Research-Hearing Aids Market Analysis By Product Type

https://www.grandviewresearch.com/industry-analysis/hearing-aids-market

●    The foundation fo the deaf under the royal patronage of Her Masjesty the Queen

http://deafthai.wixsite.com/deafthai/untitled-c1xx5

●    Bureau of Special Education Administration

http://special.obec.go.th/belong1.php

●    Hearing loss in person 65 years and older

http://www.who.int/pbd/deafness/news/GE_65years.pdf

●     Best Hearing Aids Brands

https://www.consumeraffairs.com/health/hearing/#guide-criteria

  • Bone Conduction headphones

https://aftershokz.com/

 

I would like to thank my teachers for supporting me on this project in terms of helping me on writing the submission. I would also like to thank Sodsuksa Anusansunthorn deaf school for hosting the testing facility and the deaf volunteer students. Most importantly, I would like to thank my parents both my mom and my dad. They helped me prepare the test, the data collection and in writing the submission. If it wasn't of my parent's support, This project would never be accomplished. I would like to also thank Teacher Dang for hosting the testing facility for us.

 

Thank You