LoVi App: Android Application-based Image Classification for Low Vision
DOI:
https://doi.org/10.5614/itbj.ict.res.appl.2023.18.2.3Keywords:
convolutional neural network, deep learning, image classification, low vision, smartphoneAbstract
In Indonesia, many people with visual impairments are drawing public attention to their rights as fellow humans. One of the limitations that individuals with low vision face is their ability to recognize objects and navigate their surroundings due to difficulties in visual perception. In this modern era, deep learning technologies, especially in image classification, can help people with low vision overcome these challenges. In this paper, we discuss a deep learning system that optimizes image classification on users' smartphones to enhance visual support for individuals with low vision. We present an Android-based app, LoVi, designed to assist users with low vision. Powered by core systems within Sherpa models (TrotoarNet, IndoorNet, and CurrencyNet), LoVi has three modes: outdoor, indoor, and currency. The LoVi application provides over 80% accuracy for navigation on sidewalks, indoor object recognition, and currency identification. TrotoarNet aids in sidewalk navigation, IndoorNet assists with indoor object identification, and CurrencyNet recognizes Rupiah banknotes. Additionally, low-vision users can receive voice feedback for further accessibility.
Downloads
References
Press Release: Pertuni?s Strategic Role in Empowering the Blind in Indonesia), Pertuni (Persatuan Tunanetra Indonesia), 04 Maret 2017. (Text in Indonesian) Available: https://pertuni.or.id/siaran-pers-peran-strategis-pertuni-dalam-memberdayakan-tunanetra-di-indonesia/ (08 July 2023).
Vision Loss Expert Group, Magnitude, Temporal Trends, and Projections of the Global Prevalence of Blindness and Distance and Near Vision Impairment: A Systematic Review and Meta-analysis, Lancet Glob Healt, 5(9), pp. e888-e897, 2017.
Irwanto, E.R., Kasim, A., Fransiska, M., Lusli & Siradj, O., Situation Analysis of People with Disabilities in Indonesia: A Desk-Review, Pusat Kajian Disabilitas Fakultas Ilmu-ilmu Sosial dan Politik, Universitas Indonesia, Depok, 2010. (Text in Indonesian)
Garina, L., Prevalence, Characteristics, and Health Services for Children with Special Needs in Indonesia, 2012.
WeWALK, WeWalk, Available: https://wewalk.io/en/.
Tempo, Introducing BriCane, Modern Cane for the Blind Made in Bandung, tempo.co, 31 October 2020. (Text in Indonesian). Available: https://difabel.tempo.co/read/1400939/perkenalkan-bricane-tongkat-difabel-netra-kekinian-buatan-bandung (08 July 2023).
Ministry of Health of the Republic of Indonesia, Disability Situation, December 2014. (Text in Indonesian) Available: https://pusdatin.kemkes.go.id/download.php?file=download/pusdatin/buletin/buletin-disabilitas.pdf (07 July 2023).
Ministry of Health of the Republic of Indonesia, Situation of Vision Impairment and Blindness, October 2014. (Text in Indonesian). Available: https://pusdatin.kemkes.go.id/download.php?file=download/pusdatin/infodatin/infodatin-Gangguan-penglihatan-2018.pdf (08 July 2023).
Karkar, A. & Al-Maadeed, S., Mobile Assistive Technologies for Visual Impaired Users: A Survey, in International Conference on Computer and Applications (ICCA), Doha, 2018.
Lee, C.S., Lee, J.I. & Han, S.E. Deep Learning Based Mobile Assistive Device for Visually Impaired People, in International Conference on Consumer Electronics-Asia (IOCE-Asia), 2021. DOI: 10.1109/ICCE-Asia53811.2021.9641925
Nguyen, H., Nguyen, M., Yang, S. & Le, H., Web-based Object Detection and Sound Feedback System for Visually Impaired People, in International Conference on Multimedia Analysis and Pattern Recognition (MAPR), 2020. DOI: 10.1109/MAPR49794.2020.9237770
Wei-Cheng, W., Yoke-Leng, Y. & Kok-Chin, K., Object Detection and Recognition for Visually Impaired Users: A Transfer Learning Approach, in 2nd International Conference on Artificial Intelligence and Data Sciences (AiDAS), 2021. DOI: 10.1109/AIDAS53897.2021.9574220
Nguyen, H., Nguyen, M., Nguyen, Q., Yang, S. & Le, H. Web-based Object Detection and Sound Feedback System for Visually Impaired People, in International Conference on Multimedia Analysis and Pattern Recognition (MAPR), 2020. DOI: 10.1109/MAPR49794.2020.9237770
Won, W.-C., Yong Y.-L. & Khor, K.-C., Object Detection and Recognition for Visually Impaired Users: A Transfer Learning Approach, in 2nd International Conference on Artificial Intelligence and Data Sciences (AiDAS), 2021. DOI: 10.1109/AIDAS53897.2021.9574220
Caballero, A.R. Catli, K.E.I. & Babierra, A.G.F., Object Recognition and Hearing Assistive Technology Mobile Application using Convolutional Neural Network, in International Conference on Wireless Communication and Sensor Networks (icWCSN), New York, 2020.
Parikh, N., Shah, I. & Vahora, S., Android Smartphone based Visual Object Recognition for Visually Impaired using Deep Learning, in International Conference on Communication and Signal Processing, 2018. DOI: 10.1109/ICCSP.2018.8524493.
Vaidya, S., Shah, N., Shah, S. & Shankarmani, R., Real-Time Object Detection for Visually Challanged People, in 4th International Conference on Intelligent Computing and Control Systems (ICICCS), 2020. DOI: 10.1109/ICICCS48265.2020.9121085.
Sunitha, M. R, F. Khan, G. Ghatge R & H. S, Object Detection and Human Identification using Rasberry Pi, 1st International Conference on Advances in Information Technology (ICAIT), 2019. DOI: 10.1109/ICAIT47043.2019.8987398.
Meshram, V.V., Patil, K., Meshram V.A. & Shu, F.C. An Astute Assistive Device for Mobility and Object Recognition for Visually Impaired People, IEEE Transactions on Human-Machine Systems, 49(5), pp. 449-460, 2019.
Tian, S., Zheng, M., Zou, W., Li. X. & L. Zhang, Dynamic Crosswalk Scene Understanding for the Visually Impaired, IEEE Transactions on Neural Systems and Rehabilitation Engineering, 29, pp. 1478-1486, 2021.
Mataro, T.V., Masulli, F., Rovetta, S., Cabri, A., Traverso, C., Capris, E., & Torretta, S., An Assistive Mobile System Supporting Blind and Visual Impaired People when Are Outdoor, in IEEE 3rd International Forum on Research and Technologies for Society and Industry (RTSI), 2017. DOI: 10.1109/RTSI.2017.8065886.
Sivate, T.M., Pillay, N., Moorgas, K. & Singh, N., Autonomous Classification and Spatial Location of Objects from Stereoscopic Image Sequences for the Visually Impaired, in 2022 International Conference on Electrical, Computer and Energy Technologies (ICECET), 2022.
Widayani, A., Kusuma, H. & Purwanto, D., Visually Impaired Person Detection Using Deep Learning for Dangerous Area Warning System, in 2022 International Seminar on Intelligent Technology and Its Applications (ISITIA), 2022.
Basheshankar, A., Lande, A., Nandeshwar, A., Sarkar, Udapure, A. & Chandankhede, D.H., Assistive Cane for Visually Impaired People, in 2022 10th IEEE International Conference on Emerging Trends in Engineering & Technology Signal and Information Processing (ICETET-SIP-22), 2022.
Wikipedia, Sherpa People, Available: https://en.wikipedia.org/wiki/Sherpa_people. (8 March 2022).
TensorFlow, Available: https://www.tensorflow.org/lite. (01 March 2022)
Keras, Available: https://keras.io/. (01 March 2022)
Ioffe, S. & Szegedy, C., Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, in 32nd International Conference on International Conference on Machine Learning, 2015. DOI: 10.48550/arXiv.1502.03167.
