by Netherlands Institute for Neuroscience

How does a visual prosthesis work? Credit: Charlotte Daemen, Netherlands Institute for Neuroscience

In collaboration with their colleagues at the Donders Institute, researchers at the Netherlands Institute for Neuroscience have developed a simulator that enables artificial visual observations for research into visual prostheses. This open-source tool is available to researchers and offers those who are interested an insight into future applications.

Blindness affects approximately forty million people worldwide and is expected to become increasingly common in the coming years. Patients with a damaged visual system can be broadly divided into two groups: those in whom the damage is located in front of or in the photoreceptors of the retina and those in whom the damage is further along in the visual system.

Various retinal prostheses have been developed for the first group of patients in recent years, and clinical tests are underway. The problems for the second group are more difficult to tackle.

A potential solution for these patients is to stimulate the cerebral cortex. By implanting electrodes in the brain's visual cortex and stimulating the surrounding tissue with weak electrical currents, tiny points of light known as 'phosphenes' can be generated.

This prosthesis converts camera input into electrical stimulation of the cerebral cortex. In doing so, it bypasses part of the affected visual system and thus allows some form of vision. You could compare it with a matrix sign along the highway, where individual lights form a combined image.

An example fragment of the simulator shows a normal image on the left, an image with thousands of electrodes in the middle, and an image with 100 electrodes on the right. Credit: Netherlands Institute for Neuroscience

How we can ensure that such an implant can actually be used to navigate the street or read texts remains an important question. Maureen van der Grinten and Antonia Lozano, from Pieter Roelfsema's group, along with colleagues from the Donder's Institute, are members of a large European consortium.

This consortium is working on a prosthesis that focuses on the visual cerebral cortex. Maureen van der Grinten emphasizes, "At the moment, there is a discrepancy between the amount of electrodes we can implant in people and the functionalities we would like to test. The hardware is simply not far enough yet. To bridge this gap, the process is often imitated through a simulation."

Simulated phosphene vision

"Instead of waiting until blind people have received implants, we're trying to simulate the situation based on the knowledge we have. We can use that as a basis to see how many points of light people need to find a door, for example. We call this 'simulated phosphene vision'."

"So far, this has only been tested with simple shapes: 200 light points that are neatly orientated, rectangular pixels of equal size on a screen. People can test this with VR glasses, which is very useful, but does not correspond to the actual vision of blind people with a prosthesis."

"To make our simulation more realistic, we collected a whole load of literature, created and validated models, and looked at the extent to which the results correspond to the effects that people reported. It turns out that the dots vary greatly in shape and size depending on the parameters used in the stimulation."

"You can imagine that if you increase the current, the stimulation in the brain will spread further, hit more neurons, and, therefore, provide a larger bright spot. The location of the electrode also determines the size of the dots. By influencing the various parameters, we looked at how this actually changes what people see."

Publicly accessible

"The simulator is currently being used for research in Nijmegen, where they are investigating the impact of eye movements. With this article, we hope to offer other researchers the opportunity to use our simulation as well. We would like to emphasize that the simulator is publicly accessible to everyone, with the flexibility to make adjustments where necessary. It is even possible to optimize the simulation using AI, which can assist you in identifying the necessary stimulation for a specific image."

"We are now also using the simulator to give people an idea of where this research could go and what to expect when the first treatments are carried out in a few years. Using VR glasses, we can simulate the current situation with 100 electrodes, which also highlights how limited vision through a prosthesis is: they may be able to find a door but won't have the ability to recognize facial expressions."

"Alternatively, we can show a situation with tens of thousands of electrodes and what that will bring us when this technology is developed far enough."

The study is published in the journal eLife.

More information: Maureen van der Grinten et al, Towards biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses, eLife (2024). DOI: 10.7554/eLife.85812

Journal information: eLife 

Provided by Netherlands Institute for Neuroscience