AI Closed the Loop on a Vision Implant, in a Study Co-led by UCSB
Most visual implants operate open-loop, sending predetermined stimulation patterns into brains that can respond very differently from one person to the next. Researchers at UCSB, ETH Zurich, and Miguel Hernández University have now used deep learning to close that loop, adapting stimulation to the cortical responses of a blind participant.
Brain-computer interfaces have become increasingly good at reading neural activity. Speech neuroprostheses and cursor-control systems, including those now being tested in human trials, record activity from the brain, decode a user's intent, and translate it into an external action. Visual prostheses face the inverse problem. A cortical implant must write information into the brain by delivering electrical stimulation that produces a useful visual percept.
That turns out to be considerably harder than treating an electrode array like a display. Stimulating one electrode does not simply produce one reliable “pixel.” Electrodes interact, cortical responses vary across time and individuals, and the resulting percepts can be difficult to predict. What has been missing is a sufficiently accurate model of how a particular brain transforms patterns of electrical stimulation into neural activity and, ultimately, perception.
A study published in Neuron takes a step toward solving that problem by closing the loop in a human participant. The work was conducted by researchers at ETH Zurich, Miguel Hernández University, and UC Santa Barbara as part of a feasibility trial in Spain.
Co-first authors are Jacob Granley of UCSB, Pehuén Moure of ETH Zurich, and Fabrizio Grani of Miguel Hernández University. Shih-Chii Liu of ETH Zurich, Eduardo Fernández of Miguel Hernández University, and Michael Beyeler, associate professor of computer science and psychological and brain sciences at UCSB, supervised the work.
The experiments used a 96-channel array implanted in the visual cortex of a 27-year-old man who had lost his vision following a traumatic brain injury. Importantly, the array could both stimulate the cortex and record its responses. The participant was implanted at Hospital IMED Elche in 2024, and members of Beyeler's Bionic Vision Lab traveled to Spain to conduct the experiments in Eduardo Fernández's laboratory at Miguel Hernández University.
The researchers trained a deep neural network as a forward model of the participant's cortex. The model learned both the brain activity present immediately before stimulation and the cortical response produced by a given stimulation pattern. This allowed it to account for the state the cortex was actually in at the time of stimulation, rather than assuming a fixed response profile.
The team then turned the model around. Given a desired pattern of cortical activity, they searched for stimulation patterns predicted to produce it. These model-optimized patterns reached their neural targets more accurately than conventional approaches while requiring less electrical current.
A second result may be just as important. The neural activity recorded after stimulation predicted what the participant reported seeing, including phosphene shape, size, brightness, and color, better than the stimulation parameters themselves did. In other words, measuring what the brain actually did provided more information about perception than knowing what stimulation had been delivered.
“A useful visual prosthesis cannot rely on a fixed recipe,” Beyeler says. “It has to learn how an individual brain responds and adapt the stimulation accordingly. Ultimately, the device should adapt to the person, not the other way around.”
The work builds on research supported by Beyeler's 2022 NIH Director's New Innovator Award, a five-year, $2 million grant aimed at developing computational methods for more intelligent visual neuroprostheses.
Video by Pehuén Moure (Institute of Neuroinformatics, ETH Zurich and University of Zurich). Music by Kevin MacLeod, "Meditation Impromptu 02" (incompetech.com), licensed under CC BY 3.0
First reported by Cameron Walker for CoE: https://engineering.ucsb.edu/news/ai-helps-bionic-eye-research