Mentor Introduction
The following Google Summer of Code (GSoC) project was born out of the identification of a need for fun and educational games for children who require specific alternative methods for interacting with digital technology. Motivated by conversations with Dana Cappell at Beit Issie Shapiro in Israel, designers and developers at the Inclusive Design Research Centre proposed GSoC projects that would address these gaps. Below is a description shared with the students at the beginning of the program:
Project Description:
Young children often learn through play, exploration, and discovery. Kids who use Tobii Eyegaze or switch interfaces are often limited to using games that have simple interfaces such as ‘choose an answer’ style games. This does not give kids a chance to play, discover, and explore. How can you create a game with simple enough controls that still preserves the ability to explore and discover through play?
An additional use case involves very young, pre-verbal, children who are blind. While sighted toddlers are swiping and playing digital games at an early age (and thereby acquiring digital literacy), their peers who are blind are unable to do the same. With the current games, until they have language, children who are blind are unable to acquire digital literacy. How can a game be created that works for toddlers who are blind and helps them begin a journey toward digital literacy?
(This project will be done in partnership and close collaboration with Beit Issie in Tel Aviv, Israel).
A team of mentors made up of both designers and developers, including Dana Ayotte, Alan Harnum, Jess Mitchell and Gregor Moss, had the pleasure of working with two students, Christine Hu and Ria Bhatia, on developing these games for children. The students also worked closely with Dana Cappell at Beit Issie Shapiro in Israel. This article focuses on two games created, designed, and developed by Ria Bhatia: SPIFIND and Music Mania.
Mentorship on this project included support for coding and design, practicing effective ways of communicating in an open source community, and the incorporation of inclusive design values throughout the process. Although the focus of GSoC projects is on the coding work, we believe that all of these aspects of any project are equally important to practicing inclusive design.
_____________
Google Summer of Code 2018
By Ria Bhatia
This post is regarding my GSoC 2018 project, where I developed and designed two games: Spifind & Music Mania for children with unique needs and preferences. An introduction to my GSoC project, detailed information about both the games, implementation along with the problems faced, and how I finally met the objectives are explained here. Also possible future improvements are described at the end of this post. Hope you will enjoy!
Contributor: Ria Bhatia
Ria is currently in her third year pursuing a B.Tech in Computer Science Engineering at Indira Gandhi Delhi Technical University for Women in India. She loves to code and develop new and exciting applications. Apart from this, Ria is actively involved in community development, social work and peer education programs at college.
Organization: Inclusive Design Institute
Mentors: Jess Mitchell, Dana Ayotte, Alan Harnum, Gregor Moss
Partners: Beit Issie Shapiro
GitHub Repositories
Objectives
One of the main objectives of this project was to develop an online game for young children using eye gaze technology, in order to help children develop various skills (including digital, observational, motor and cognitive) through exploration and discovery. The ultimate aim with any child using alternative means of access to technology is to enable the use of the same resources as their peers and help them gain confidence, clarity, understanding, social integration and self sufficiency.
Another aim was to develop an easy to learn, engaging and fun game for pre-verbal children born blind (up to 3 or 4 years old) to help them begin a journey toward digital literacy. As they cannot learn by seeing what is going on around them, they must learn “by doing” and by interacting with their environment. 1
Introduction
The first game I created (Spifind) is a game that can be played using eye gaze technology. Eye gaze or eye tracking is an alternative way of accessing a computer using a mouse that is controlled with eye movement. Use of an eye tracker provides an alternative for children and adults with certain disabilities that may limit their use of a desktop mouse. This technology empowers users to engage in many digital activities, including playing games, accessing the internet and communicating with family and friends (e.g. sending text messages or selecting pictographs in communication software). An issue with most eye gaze based games is that they are often limited to very simple interfaces such as ‘choose an answer’ style games. This doesn’t give kids a chance to play, discover and explore.
The second game I created (Music Mania) is intended to allow pre-verbal children born blind to develop digital skills and comfort with technology. While sighted children are interacting with technology (swiping and playing digital games) at an early age, their peers who are non-visual are unable to do the same. Children born blind rely on spoken language in order to interact with the games and other digital technologies that are currently available, and are therefore unable to acquire digital literacy at the same early age as their sighted peers.
Interacting with kids of different abilities over the past few years has helped me to understand that their ambitions and desires are not defined by their abilities, but that all children have a broad range of unique needs and preferences. This was one of the things that motivated me to work on this project. My keen interest in JavaScript was also a driving force that helped me break down barriers and provide equal opportunities for all children, regardless of ability.
About the projects
SPIFIND is an eye gaze based game for children. In the game, there is a dark room , spiders, and a flashlight. The individual playing the game has control of the flashlight. As the beam of light from the flashlight is moved around the room the spiders appear. The goal is to capture as many spiders as possible. The movement of the flashlight is operated by the movement of the game-player’s eyes. When the beam of light is on the spider, the child can dwell (hold focus) or blink to capture it. Capturing a spider needs to be done in a given time limit. The number of spiders captured is reflected on the scoreboard.
This game aims to provide opportunities for free exploration and lets the child explore, play and make mistakes without direct intervention from an adult. It helps refine eye gaze
control and accuracy at an early age.
Play Spifind here. Use a free eye tracking software.
MUSIC MANIA is a simple game where kids can play music using computer keyboard keys. Music is a wonderful resource for an infant born blind, as exposure to music can enhance a child’s natural ability to decode sounds and words.2, 3 When the child presses a key, it produces a sound. Eventually, the child will figure out that he or she is making that sound him/herself and this is an important step towards mental development. Pressing any key produces a different musical sound which can provide a sense of accomplishment in a fun and joyful way.
Children often sway and move their hands in response to music.4 It can help improve motor skills and be a valuable step towards digital literacy. Music Mania was created to support the learning of the location of different keys on a conventional keyboard (QWERTY). Understanding the layout of a conventional keyboard can then help with the exploration and discovery of different technologies.
Play Music Mania here.
Implementation & Technical Details
Both of the games are web applications and were developed using standard web technologies HTML, CSS & JavaScript. For better maintainability and composability of code, I have used the popular modern JavaScript framework ReactJs. Babel was used to transpile the code to support all browsers. For bundling of the web application, I used webpack bundler. webpack is a very popular bundler with many plugins like uglify/minify and css imports. More information about the technology stack can be found here for SPIFIND and here for MUSIC MANIA.
Challenges Faced
- Relaying instructions: finding ways to communicate how the game works and finding solutions to do this was really tough and challenging. I created visual tutorials with animated gifs to help children understand the game.
- Designing the scenes and deciding the appropriate animations and music required a lot of research that included the following sources:
- http://www.gazeplay.net/
- https://www.helpkidzlearn.com/downloads/EyeGaze_Guidebooks/Spectronics-EyeGaze-Leaflet-Web.pdf
- https://docs.google.com/document/d/1A9rZAL3beDyLzO9u92KMELURc9shbDDrhEnUJmzunG0/edit?usp=sharing (audio related research)
- https://thinksmartbox.com/products/eye-gaze/
- https://www.tobii.com/
- http://www.playgroupnsw.org.au/ParentResources/EarlyChildhoodDevelopment/importance-of-music-toddler-development‘
- Arranging eye gaze equipment was a big challenge. The US based Tobii group helped by sending me complimentary eye gaze equipment for my project. I am really grateful for their kind help and support.
- Based on feedback from my mentors a number of design changes had to be implemented from time to time to make sure the game remained fun and accessible. When facing these challenges, my mentors were very supportive.
Further Enhancements
Music Mania is at an early stage of development and there are many details to be worked out. I plan to do more research and hope to continue development of this game beyond the GSoC project timeframe.
Pillars of Success — My mentors!
This was the first time I participated in GSoC. Even though I had some previous experiences working with open source communities, I could never have imagined getting this much exposure.
The support and responsiveness of my mentors made it easy for me to achieve milestones within the timeline. Meeting with them every week and discussing my progress and problems helped me to keep things moving. Always providing quick feedback to my questions and concerns, they made it really easy for me to solve issues which occurred during implementation. I cannot thank them enough for always guiding me and helping me emerge as a better programmer.
Finally, I would like to thank the Inclusive Design Institute for giving me this wonderful opportunity. Without my mentors, the IDRC team, and the Beit Issie Shapiro Team, I wouldn’t have been able to complete this project. I hope that I was able to give something back to the community. I’m planning to contribute to the community in the near future and provide help in whatever way possible.
Thank you Google!
Doing a big open source contribution was one thing I always dreamed of. Without Google I wouldn’t have gotten this opportunity to meet these amazing people and get this invaluable experience. Thank you Google for giving me this opportunity!