An Augmented Reality-Based Word-Learning Mobile Application for Children with Autism to Support Learning Anywhere and Anytime: Object Recognition Based on Deep Learning

Document Type

Conference Proceeding

Publication Date

1-1-2019

Abstract

An abundant earlier controlled studies have underscored the importance of early diagnosis and intervention in autism. Over the past several years, thanks to technological advances, we have witnessed a large number of technology-based teaching and learning applications for children with autism. Among them, augmented reality-based ones have gained much attention recently due to its unique benefits of providing multiple learning stimulus for these children via accessing a kinesthetic moving simply using a mobile device. Despite it, few have been developed for these young children in China, which motivates our study. In particular, in this paper, we present a mobile vocabulary-learning application for Chinese autistic children especially for outdoor and home use. The core object recognition module is implemented within the deep learning platform, TensorFlow; unlike other sophisticated systems, the algorithm has to run in an offline fashion. We conducted two small-scale pilot studies to assess the system’s feasibility and usability with typically developing children, children with autism, their parents and special education teachers with very promising and satisfying results. Our studies did suggest that the downside of the application is the performance of the object-recognition module. Therefore, before we further examine the benefits of such AR-based learning tools in clinical settings, it is crucial to fine-tune the algorithm in order to improve its accuracy. Despite it, since the current literature of AR-technology on Chinese word-learning for children with special needs is still in its infancy, our studies offers early glimpse into the usefulness, usability and applicability of such AR-based mobile learning application, particularly to facilitate learning at anytime and anywhere.

Publication Title

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

First Page Number

182

Last Page Number

192

DOI

10.1007/978-3-030-23563-5_16

This document is currently not available here.

Share

COinS