You are now in the main content area

Beyond the lab: Naimul Khan makes an impact using VR and AR technologies in his industry partnerships

Innovative collaborations enhance user experiences and advance medical research
December 16, 2021
Naimul Khan, co-director of the Ryerson Multimedia Research Laboratory and engineering professor

Naimul Khan, co-director of the Ryerson Multimedia Research Laboratory and engineering professor

Naimul Khan, assistant professor in the Department of Electrical, Computer, and Biomedical Engineering, incorporates augmented reality (AR) and virtual reality (VR) technologies in his most notable partnerships. An expert in designing user-centred VR systems, Khan’s research improves user engagement and enhances medical experiences by developing algorithms, demonstrating their capabilities and creating ground-breaking solutions for industry partners. 

A former Ryerson PhD student and current co-director of Ryerson Multimedia Research Laboratory, Khan understands the value of collaborative partnerships to the university. According to Khan, the Office of the Vice President, Research and Innovation (OVPRI) was very helpful in navigating granting agencies, drafting partner agreements and writing proposals, among other things. “The university has a strong history of working closely with the industry, leading to practical solutions and immediate impacts outside of the classroom,” said Khan.

VR walkthrough of Fort York

Khan’s collaboration with AWE Company Ltd. (external link)  began with the creation of an outdoor guided VR tour of Toronto’s Fort York National Historic Site. Using virtual markers, visitors can now bring to life pivotal historical events that took place there, such as the Battle of York. Khan’s team solved the computer vision challenges of an outdoor exhibit of such scale and user-focused challenges of inputting virtual marker data while wearing VR headsets. 

The project succeeded in enhancing the city’s virtual tourism and educational experiences leading Khan to work with AWE on new AR projects that further finesse high-level interactive opportunities.

AR museum exhibit app Augmented Alley

Following the success of the Fort York tour, Khan partnered with SimentIT Inc. and SEED Interactive to create an AR museum exhibit for the Canada Science and Technology Museum (external link) . Collaborating with FEAS colleagues, including professors Ling Guan and Andy Ye, Khan overcame several challenges to create an image processing and artificial intelligence (AI) algorithm for an AR app, Augmented Alley. This app provides visitors with a direct link to animate exhibited objects on their smartphones without the extra step of scanning a quick response (QR) bar code.

“Designing computer vision solutions for low-light environments where there is a lot of movement from people who may be obscuring the objects was a huge challenge,” said Khan.

Naimul Khan tests one of the virtual reality games designed by Shaftesbury.

Naimul Khan tests one of the virtual reality games designed by Shaftesbury. Photo credit: Victoria Paterson

VR therapy games enhanced by AI

Another notable partnership saw media company Shaftesbury (external link)  invite Khan to enhance the interactivity of their VR therapy games. The games aim to alleviate stress and anxiety in children before they undergo medical procedures and surgery. “Shaftesbury was looking for a way to attain an accurate estimation of biofeedback to make their medical rehabilitation games more interactive and effective,” said Khan. 

To that end, Khan’s team enabled a wide range of physiological sensor data – such as a player’s heart rate – to be captured during a game, leading to accurate readings of the user’s stress and engagement levels. “The innovation lies in the AI aspect of the project,” said Khan. “The captured data act as signals that allow us to gauge whether or not an experience is working for the user. The signals then adjust the game accordingly to enhance players’ experiences.”

A roadside view of an urban refugee settlement in Kampala, Uganda

A roadside view of an urban refugee settlement in Kampala, Uganda

VR and AR interventions for mental health and resource security

Khan currently collaborates with Carmen Logie, a well-known name in global health research for marginalized populations. After discovering the interactive value of VR and AR technologies, Logie reached out to Khan to help her incorporate them into her research projects and act as her co-principal investigator. Two of their projects focus on the mental health and resource security of urban refugee youths in Kampala, Uganda.

One collaboration involves a VR intervention study that aims to reduce mental health stigma, increase mental health literacy and decrease symptoms of depression among refugee youths. Khan is leading the project’s VR design, development and implementation. The second project will design an AR therapeutic experience that teaches refugee youths how to cope with food and water insecurity.

While similarly themed research projects have generated a body of evidence from high-income countries, these projects aim to address the lack of documentation and shortage of mental health resources focused on developing countries.

These partnerships received funding from the Natural Sciences and Engineering Research Council of Canada (external link)  (NSERC), the Ontario Centre of Innovation (external link)  (OCE), Grand Challenges Canada (external link)  and the Social Sciences and Humanities Research Council (external link)  (SSHRC).