Digital Media & Inclusivity

Listenapp- An AI-Based Mobile Application Platform for Auditory Learning

Abstract
In today’s fast-paced world, students and working-class individuals are increasingly replacing the written word with audio content – primarily podcasts and audiobooks. However, not all content is available in audio format. Print-first media like newspapers and textbooks remain unavailable to the growing group of consumers who prefer the spoken word for its accessibility while commuting, working, or otherwise unable to engage fully with a printed work. Automated text-to-speech solutions do exist but read in a flat affect which fails to communicate the emotions attached to the writing. This paper examines the feasibility of a framework that uses artificial intelligence, including natural language processing to parse meaning and the relatively new field of audio style transfer for speech generation, to convert any written work into a piece of audio, read in a voice chosen by the user. This research intends to interrogate the accessibility of written works by means of automated text-to-speech technologies, to better understand the gaps for individuals with reading difficulties or other learning challenges, the blind and partially sighted community, and those interested in text-to-speech toolsets.
About the Author
Ajith Kumar Balakrishna Pillai

Marketer by day, Filmmaker by passion and Entrepreneur by season.

About the Project


With our day to day activities become more digitalized, technology has started to play a standard function in helping individuals with special needs and also approaching them with far better chances for freedom. Accessibility is not an optional feature; instead, it is a necessity, as 10% of the American population has low vision, [ National Health Interview Survey (NHIS), American Blind Association 2017] and also there is a dire need for more applications to be out there that help the blind as well as visually damaged. There is considerable growth in assistive technology, and AT tools like screen reading, screen magnifiers, or apps like blind square, which is developed for visually implied, have become a blessing. However, it is a minuscule area of problem that is being solved. Areas like accessibility in reading or auditory learning and learning disabilities have not got the attention those require.  [Fonseca C., Efraín & Castro, John & Ramos, Ivette & Rodríguez, Priscila. (2017). Usability of Mobile Applications for Visually Impaired People: An Empirical Study.]  Studies show that there has been a considerable number of individuals who prefer audiobooks over books. With the amount of information available now days, users need the ability to not only learn on the go but also learn in a manner that’s efficient, relatable and generates human emotions. Using the TTS functionality can save you a lot of time if you need to read books, study for tests, review your notes or reports, or if you just feel like listening instead of reading. 



Problem 


In today’s fast-paced world, students and working-class individuals are increasingly replacing the written word with audio content – primarily podcasts and audiobooks. However, not all content is available in audio format. Print-first media like newspapers and textbooks remain unavailable to the growing group of consumers who prefer the spoken word for its accessibility while commuting, working, or otherwise unable to engage fully with a printed work. Automated text-to-speech solutions do exist, but the current technology available, such as text to speech, is merely a sound blurting out words that lacks emotion, natural pauses, the ability to comprehend sarcasm, and punctuation. 


Solution


 AI powered mobile app that creates an audible version of any text that you scan with the addition of emotional prosody. Listenapp just does that! Listenapp simplifies the listening experience by allowing the user to convert any written content to a soundscape that is narrated in his/her favorite voice in less than 5 second.


How it works


After installing the application to the mobile phone, the user scans or takes a picture of the written content that needs to be made into an audiobook then the user indicates their preferences for their preferred narrators as well as what type of soundscape they would like to generate. There will be a set of pre-loaded narrators or an option of creating a new voice


Benefits 


Offers Convenience – Use it on the go! 

Exhibits Emotion- Intangible charm to the experience 

Accessibility- Beneficial for those with special needs or visual impairments  

Learning opportunities – Improve communication skills & social interaction 

Affordable Commercial Usage – No more professional voice over artist hiring cost. 


MY NEONORMAL STORY
''
When it all started in March, I thought the world was being paranoid- I said to myself, media is making a big deal out. Then slowly, we got pushed to locked-down; things started to get on to me. Businesses got hit, bank balance bled, the stock crashed –Eventually depression. The good thing was that I got more time to focus on my MRP, can’t say Covid-19 did not affect the development process as I had to redefine the project objectives and limit scaling. Finally, I managed to pull everything together and was able to finish the project.