There is a promise in AR/VR technology to help students on the Autism Spectrum to learn better

3DBear is part of Bit the spectrum EU project, where VR and AR exercises are developed to support the students on the autism spectrum. The project also aims to support the teachers working with ASD students - giving them understanding of the possibilities of VR and AR technologies. During the project, barriers for the technology usage are also researched. Here is how 3DBear’s CEO, Jussi Kajala, reflects on the findings of the project so far.

In the BITTHESPECTRUM project, we have now worked for a year to find ways of helping special needs students with the help of immersive technology. In a consortium consisting of University of Magdeburg in Germany, Skillsdivers in Spain, IRSEI in Italy, Vocational College Luovi in Finland and 3DBear, we are investigating, experimenting with, developing and piloting new ways of using augmented reality (“AR”) and virtual reality (“VR”) technologies -much hyped Metaverse technologies- to improve teaching special needs students, considering specifically Autism Spectrum Disorder (ASD).

In this VR-environment, students can practice working in a kitchen in their own phase without social pressure, learning about the procedures and timing. This VR-environment was co-developed with Finnish special education vocational college Kiipula.

As the first year of the project comes to a close, we have completed an extensive desk research on the topic as well as conducted a survey research targeted to people working on the ASD field. Moreover, we’ve held focus groups to quantitatively verify the initial research findings and hypotheses. 

Here is a summary of the observations. Based on the work we have been doing we believe that there is very high potential in using AR/VR to support special needs teaching.

With simulations in VR students can practice everyday tasks or social interactions, or seeing the others’ point of view without any actual social pressure in the learning event. 

Prior research has focused on improving social interaction and proving that technology is not harmful. As it is usual in a developing field, people always fear the new technology. It is also very understandable considering the nature of this particular user group. Therefore it is not surprising to find that first research has focused on examining whether immersive technologies are harmful for learning for ASD students. The pre-existing research shows that AR or VR headset devices do not impair learning for ASD but there’s no conclusive studies regarding effectiveness of impact. The latter observation is partly explained by the facts that there is very little immersive content for teaching special needs students, and this content needs to be built first in order to evaluate its impact properly.

In this VR-environment, students can practice working in a kitchen in their own phase without social pressure, learning about the procedures and timing. This VR-environment was co-developed with Finnish special education vocational college Kiipula.

AR/VR in the context of education of ASD students has been explored primarily in the field of improving social interaction, such as recognizing emotions from different facial expressions, speech, or identifying socially undesirable characteristics. There is evidence supporting that social interaction can be improved through AR/VR technology. With simulations in VR students can practice everyday tasks or social interactions, or seeing the others’ point of view without any actual social pressure in the learning event. This is in particular useful for students on the Autism Spectrum. 

Also some very specific hard-coded simulations e.g. for car driving or practicing crossing the street in a safe environment have been developed, but there is basically no pre-existing AR/VR content for wider teaching special needs students. There has been a VR learning environment developed for first responders but no significant training environments in connection with ASD teaching have been developed.

People working on the field need content, tools, and training to harness the technology.

In summary, using VR/AR training environments for ASD education is still in baby shoes although the research shows that there could be significant potential applying the technology in this field as there is pre-existing evidence that VR/AR technology improves social interaction capabilities. 

Moreover, there is very little training material in VR/AR available to support VET teachers or others to learn how to work with ASD students in the most effective way. Teachers do not have yet the capacity to produce or co-create their own content in AR/VR, whereas it would bring great benefits to develop this capacity.  People working on the field need content, tools, and training to harness the technology. 

Survey results reinforce the results of the desk research. We conducted a survey that reached more than 50 people working with students with autism spectrum disorder in Germany, Spain, Italy and Finland. People participating were mostly forerunners, e.g. those who use the most technology in this field.

With AR it is possible to e.g. analyse challenging social situations (Juniper-project done with Finnish National agency of Education) 

As a conclusion, AR/VR scenarios are not practically in use of ASD teaching anywhere. Moreover, educators and other professionals working in the field need content, tools and training to help them in developing social interactions of ASD students in a safe environment, recognizing emotions, motivating students, visualizing concepts, and helping during the occurrence of problem behavior. However, usually it is not understood well what opportunities AR/VR technologies offer to satisfy these needs as examples are missing.

The possibilities seen in which AR/VR technology can be applied are very concrete: factual transfer of knowledge without social component, allowing to better process visual rather than verbal stimuli, learning everyday skills in a safe environment, role playing, recognition of feelings, reinforcement of behavior through revision, and even gradually exposure to unpleasant stimuli to realize tolerance skills, and safety skills. These could be involved in many different areas of ASD education.

Teachers yearn for more content and training in order to use and remix content, and create their own. They need tools for this. This is exactly what we are doing in the BITTHESPECTRUM project.