© Daniel Tauritis
Playing Video Games to Save Lives?
Students and researchers at the university have developed a game that trains “acoustic orientation” – and that one day could be used in medicine.
Plenty of time thanks to the many corona restrictions this fall and winter? Many people are tempted by this to play video games. Why not saving lives in the process? Not straight away, but in the long run – with a game developed by students and researchers at the university. They want to incorporate knowledge gained from its use into medical applications.
A look inside the operating theater: Surgeons today have access to a completely different level of technical support during operations than 10 or 20 years ago. For example, 3D computer models allow for spatial orientation and are used to plan operations, for example, the optimal point of entry, the best angle, and the optimal depth with an ablation needle that is used to destroy a tumor. During the operation, the tip of the surgical needle is tracked and displayed as a virtual needle in the computer model. This enables surgeons to place the needle as intended by using the model on the screen.
New Acoustic Approach
“We have an entirely new approach – namely an acoustic one“, says Dr. Tim Ziemer. The music scholar works on the technical implementation of psychoacoustics in spatial audio systems at the Bremen Spatial Cognition Center (BSCC) of the University of Bremen. Together with the cognitive scientist PD Dr. Holger Schultheis, the psychology doctoral student Tina Vajsbaher and eleven master’s students, he developed a game that runs on Android, Windows, Linux, and Mac – only an iOS version could not quite be completed in the runtime of the CURAT project.
© Screenshot: Lukas Meirose/Daniel Tauritis
The core of the project is formed by a foreign sound – the “psychoacoustic sonification.” The term refers to the displaying of data by means of sound. This is why the game is called Sonification Game. The various characteristics of the sound relay a position in a 3D space to the players. During the course of the game, the players then learn to interpret the individual dimensions of the sound: How far to the left or to the right, to the top or bottom, to the front or back is the target? Additionally, players initially learn to combine two and then finally three dimensions so that they can navigate within the 3D space when blind.
Listening to Sounds During Surgery
“Hopefully, the players just have a great deal of fun. They are entertained, are training their hearing, trying to reach expert status, and break the high score,” says Tim Ziemer. “The fact that they will maybe be able to save lives in the future due to their playing is an additional motivation.” One day, the acoustic orientation in space is to help surgeons with complex, spatial target finding during minimally invasive procedures, as visualization does.
On which level does an inexperienced player start and what do learning curves look like? How quickly does a user become how precise, how prone is the user to mixing up axes or directions, and what is the top performance level? These and many more questions can be answered by the information gained through the game.
Listening Is Less Tiring Than Seeing
The aim of the “game-supported research” is to add an “acoustic orientation guide” to the “optical orientation guide.” “Twice as good,” explains Holger Schultheis. “One day, we want to additionally offer psychoacoustic sonification for complicated surgeries.” The problem with imaging processes is that spatial orientation uses a great deal of brain resources. The computer monitor does not show the spatial constellation from the viewpoint of the surgeon. They must mentally scale, rotate, move, and especially interpret the depth dimension from the two dimension graphic on the monitor. “This requires a great deal of training and procedures can be tiring. The acoustic orientation guide lessens the strain on surgeons and can increase patient safety in this way,” according to Schultheis.
© Screenshot: Lukas Meirose/Daniel Tauritis
“The psychoacoustic sonification is a world first, for which we have already received prizes”, says Tim Ziemer. The team has shown that after a 30-minute training session with the “acoustic orientation guide”, people can find a four-millimeter target in a 20 by 20 centimeter large space just as reliably as with visual aids.
Information from the Game Important for Further Development
“We can optimize the tones in surgeries thanks to the information that we are collecting with the game outside of surgeries. We are currently also preparing experiments in surgical surroundings,” states Holger Schultheis. “But only a few dozen participants are involved in experiments and they always only take between one and two hours. In order to be able to make statements regarding long-term training effects and a larger population, we need data from the game.”
Alongside surgery, further scenarios are thinkable for the implementation of psychoacoustic sonification – for example, the controlling of drones and underwater robots or the monitoring of patient data.
Further Information:
On this Webseite Cognitive Surgical Assistance you can find more information, demo videos, and the game download link.
Video: CURAT-Projectday 2020 (English w. Subtitles), Presentation of the Masterproject
