ReadSmart: Generative AI and Augmented Reality Solution for Supporting Students with Dyslexia Learning Disabilities

Abstract

Dyslexia is a learning disability that significantly hinders students' abilities to read and comprehend educational materials, posing a substantial challenge within educational environments. This paper introduces an innovative educational system, ReadSmart, that integrates both Augmented Reality (AR) and Generative Artificial Intelligence (GenAI) technologies, specifically designed to support students with dyslexia. ReadSmart utilizes Microsoft HoloLens 2, Unity3D, and the Mixed Reality Toolkit (MRTK2) to develop an immersive AR environment. Key functionalities of ReadSmart include Optical Character Recognition (OCR) facilitated by Azure Cognitive Services, which extracts text from images captured in real-time, and OpenAI's GPT-4 model, which is responsible for generating concise text summaries and illustrative images to enhance comprehension. Moreover, ReadSmart incorporates Text-to-Speech (TTS) capabilities to improve the accessibility of content for dyslexic learners. Evaluations conducted through self-assessment and peer feedback suggest that ReadSmart has significant potential in reducing the time required for reading and in enhancing comprehension. Initial findings indicate that this system could substantially improve the educational experience for students with dyslexia. Future research will focus on refining ReadSmart's features and conducting comprehensive trials to validate its effectiveness and explore its broader applicability.