Computer science researchers create augmented reality education tool

In the film Fantastic Voyage, a brilliant scientist develops a way to shrink objects — and people — to the size of a micrometer. In an effort to save his life, a group of intrepid CIA agents and medical experts are shrunk and injected into the scientist, travelling through his body to locate and destroy a blood clot in the scientist’s brain.

When the movie was released in 1966, this concept existed in science fiction alone. Researchers in the Department of Computer Science at the University of Calgary, however, are pushing the limits of augmented reality (AR) technology and exploring its practical uses in health care and education.

Christian Jacob and Markus Santoso are trying to re-create the experience of the aforementioned agents in Fantastic Voyage. Working with 3D modelling company Zygote, they and recent MSc graduate Douglas Yuen have created HoloCell, an educational software. Using Microsoft’s revolutionary HoloLens AR glasses, HoloCell provides a mixed reality experience allowing users to explore a 3D simulation of the inner workings, organelles, and molecules of a healthy human cell.

Jacob has plenty of experience in bioinformatics as the head of the Lindsay Virtual Human Project. By combining forces with Santoso, an Eyes High Postdoctoral Fellow with an extensive background in AR research, the pair are taking interactive education to the next level. “We’re finessing the cell right now, but eventually we will expand this software to include the entire body,” explains Jacob, who says that upscaling the project will be relatively straightforward and will eventually become a widely-used educational tool.

Jacob cites oncology as an example of one of the practical purposes of AR technology, a field into which he and Santoso — together with Wellness Computational, a Calgary-based company that develops computational medical illustrations and health analytics solutions — are currently expanding their work through HoloTumor. “Let’s say a doctor and a patient are talking about a cancer diagnosis. The doctor can have a more informative conversation with the patient when they can both put on the AR glasses and see the same hologram of a tumour,” he says.

“We are so used to a screen, a flat surface. But we can put a lot more information into a 3D space than into a 2D space. It changes the whole paradigm of interacting with information and interacting with computers as well,” explains Jacob. “Being in computer science, we always try to push the limits. There’s always new technology and we try to get as much out of it as we can, or more out of it than what it was built for.”

This past weekend, Santoso touted the AR research environment at the University of Calgary as one of the keynote speakers at the Augmented World Expo (AWE) Asia 2016, the world’s largest AR conference and expo, in Xi'an, China.

“Lately, it seems like our work is getting a lot of recognition,” he says. “I’m so happy to be conducting my research here. Although I am still a fairly junior academic, I am getting the opportunity to speak at such a major event.”

Santoso, who has been researching AR through some of the world’s most innovative institutions since 2011, says that Jacob’s lab at the University of Calgary is already making strides toward becoming a leader in AR software development. “It takes time to really send this technology forward for commercial purposes,” he says, “but we’re trying to use future technology to get ahead so that by the time everyone gets used to it, we’ll already be prepared with the next steps forward.”