The brain science collaboration that provides hope to blind folks

Jean Mary Zarate: 00:05

Hello, and welcome to Tales From the Synapse, a podcast brought to you by Nature Careers in partnership with Nature Neuroscience. I’m Jean Mary Zarate, a senior editor at the journal Nature Neuroscience, and in this series we speak to brain scientists all more than the planet about their life, their analysis, their collaborations, and the effect of their operate.

In episode 5, we meet a researcher devoted to understanding the complexities of vision and how to bring eyesight back to the blind.

Pieter Roelfsema: 00:40

So my name is Pieter Roelfsema. I’m the director of the Netherlands Institute for Neuroscience in Amsterdam. And I’m also a professor at the Amsterdam University Health-related Hospital, and at the Totally free University of Amsterdam.

I studied the visual brain considering the fact that quite a few years, and I usually wanted to know what takes place if you truly commence to fully grasp what you see.

And that is far from trivial, mainly because essentially, what you see is just quite a few things, quite a few little image components that fall on your retina.

But then you have to sort of group that collectively into a representation of objects and the many objects that may well surround you. And that is a quite fascinating topic.

So I studied that for, for quite a few years. And in the final couple of years, we located ourselves implanting an escalating quantity of electrodes (and electrodes is just an additional word for wires) into the brain of experimental animals.

And at some point, we realized we are at such a higher quantity, hundreds, let’s ramp it up a small bit.

So we went to 1000, with the concept of stimulating these electrodes and then developing artificial vision.

Pieter Roelfsema: 01:56

So I consider we have many targets. Initial, is truly to fully grasp how vision operates. And one particular of the subjects that I’m especially interested in is interest.

So of all the items that surround you, you can spend interest, choose out one particular of these things, what takes place in the brain when you direct your interest to some thing.

And an additional factor is what takes place if some thing enters into your conscious awareness? So what are quite thrilling concerns, so some items may well get into your awareness and some, some do not. So we are also seeing that.

And one particular of the a lot more applied targets of the lab is to build a visual prosthesis, a visual brain prosthesis, so folks who lost the function in their eyes, the concept will be just to skip these malfunctioning eyes.

And to plug in the visual details from a camera. It is one particular of the central centres, so centres for vision in the brain.

So we strategy that from many angles. So one particular is to do modelling. Second is to study vision in humans, just getting them respond with a button press, understanding what they can and can’t see.

But we also appear at the brain mechanisms in experimental animals. So we appear in mice, and we also appear in monkeys.

Pieter Roelfsema: 03:20

So I began to develop into interested in the brain immediately after reading a book by Doug Hofstadter. It is, I consider it really is a renowned book. It is referred to as Gödel, Escher, Bach, and was quite renowned when I was beginning to study in the ‘80s.

It was a present from my father. And when I study it, I knew what I wanted to do, I wanted to study consciousness. So then, I essentially began to do some projects just in my hobby time, initial on snails, then on rodents. And at some point, I realized I want to study this in an animal that is closer to humans. And I will truly fully grasp what takes place in the brain when we develop into consciously conscious of some thing.

So we, in my lab, studying cognition and the part of interest and how it is modulating the activity in the visual brain, we began to implant escalating numbers of electrodes. So electrodes are just wires.

And so, at some point, we reached a common preparation exactly where we implant, say, 200 electrodes, and then we believed, you know, we could multiply this with a little element. And then we know that from earlier operate, that if you stimulate one particular electrode electrically, you happen to be artificially activating these brain cells close to the tip of the electrode. And a individual or an experimental animal (it can even be a individual who has been blind for a lot more than ten years), they will see a dot of light. And that is with only one particular electrode.

So if you have one hundred or 1000 electrodes, you can build 1000 of these dots of lights, mainly because phosphines. And considering the fact that the area, the visual cortex, exactly where we implant these electrodes, have a map of space, exactly where you stimulate in a map the topic sees at the very same place, the outdoors visual planet, this dot of light.

So if you have 1000, you can essentially operate with them like a matrix board, they can know from the stadium or from the highway. So if you of course, if you flash up one particular bulb, the individual is going to see, properly, a dot of light. But you can build patterns. And that is what we set out to do.

So we’re essentially writing to this matrix board that is in the brain, and, and see irrespective of whether the animals are in a position to recognize them. These, these items that we create as patterns. And we located certainly that this is the case.

So we had been in a position to create, for instance, we educated, we did this in monkeys. We educated them to recognize letters.

And so they knew that if they saw letter A they would have to make an eye movement to the above. If they saw the letter B, to the left, and so on and so forth.

And at some point we educated them visually so these animals had been not blind, they could see.

At some point, we took the visual stimulus away, and we just wrote straight letter A to the brain.

And we had been quite excited to obtain that they had been certainly producing the very same response, as when we would have presented the very same letter visually. We published that in 2020. So one particular-and-a-half years ago.

So in our group, we will need a lot of diverse experience. And some of these varieties of experience are inside our personal group. So we are knowledgeable about how to place wires, electrodes in the brain.

But we also have quite a few collaborations with authorities about the planet, folks who truly know about how to make these electrodes so that they never harm the brain tissue as well a great deal.

We operate with folks in artificial intelligence who assist us to take our camera, to take camera pictures and translate them into brain stimulation patterns.

We also collaborate with neurosurgeons who can inform us how to truly make this device and make it some thing that is going to be feasible for a neurosurgeon to truly implant in the brain mainly because that is certainly a quite essential purpose for me, to bring this to a patient.

So the visual procedures will be composed of many elements. The initial is a camera. You can use the camera that you purchase. So there are now many providers that make these glasses that include a camera.

And these camera pictures are sent to a little pc. It can be the size of a telephone. And this will take in the camera image and build from it a pattern to be a post on the matrix boards in the brain.

Then ideal now, we nonetheless have truly a physical connection amongst a connector that is implanted on the skull of the topic. It can be a monkey. It can be a human. We would like to make this wireless, so it will be a wireless interconnects with a brain chip.

And then from the brain chip, there will be many wires operating into the brain. So these are the ones we get in touch with electrodes. And so, primarily based on the image that the camera captures, there is this brain stimulation pattern.

And that then offers a rudimentary type of vision. So you happen to be not going to see complete colour, complete depth as regular vision would give you. It is going to be quite rudimentary, like you are walking about with this large matrix board in front of you, ideal? So it is, it is certainly not going to be excellent, but it is most likely also going to be a great deal greater than nothing at all.

Yes, eyesight operates. It begins, of course, all in the retina, that is at the back of the eye, which is a quite, quite sophisticated device.

So there are huge groups of researchers that are studying the retina. And then from there, the details is transported to the brain by means of the optic nerve. And then it begins in the cortex in the initial area, key visual cortex.

And there are cells, brain cells, neurons, that are chosen for pretty very simple functions of the outdoors planet, say the place and the orientation of an edge of light, irrespective of whether that is a vertical edge or a horizontal edge.

And so they truly do a quite neighborhood processing. So you have quite a few of these processors and in parallel. So one particular would be straight ahead, one particular would be just adjacent, one particular would be in the upper left corner for every single place in the outdoors visual planet, there is a set of neurons that just care about what is going on there.

And then if you go to larger regions, then this details of these person detectors is combined in a lot more and a lot more sophisticated methods. So essentially, what takes place there is that you go from pixels, to ideas. And there are now quite a few folks modelling this.

So there also has been, of course, an artificial intelligence revolution that helped us fully grasp how to go from pixels to ideas. And what these folks in artificial intelligence obtain and how they model this approach essentially, is rather a excellent approximation of what is going on in the human brain.

So also, in the human brain you have all these stages that are involved in this translation from pixels into ideas.

Now, seeing what the idea in front of you is, irrespective of whether it is a bicycle or a chair, that is only one particular of the functions of vision, it is not the only one particular. You also can steer your motor behaviour. And there are other brain regions that are involved in that. So they essentially localize the edges. So if you want to choose some thing up, you will need to know exactly where your fingers are going to touch the object that you want to choose up.

And you will need to know exactly where it is, you will need to know how to position all your joints, all your joints. And so all these transformations, they are also in component informed by vision. So that is an additional quite essential part for vision to play.

Pieter Roelfsema: 11:42

So one particular factor that is, I consider, thrilling in this domain is the possibility now to sometimes record also neurons from human individuals, and some researchers in my lab are carrying out this.

So these are individuals who have serious types of epilepsy. And the neurologist does not obtain the ideal cocktail to suppress these epileptic attacks. So then the neurosurgeon comes into the play, and in some instances it is clear what is the problematic area of the brain, but there are some occasional circumstances exactly where the neurosurgeon is not one hundred% positive.

And then these individuals get the set of electrodes, a set of wires in their brain, for about two weeks. And we have ethical approval then to attach to these clinical electrodes, quite tiny wires.

And by means of these, we can record single neurons. So that was a system that was created by Itzhak Fried many years ago. That offers you the exceptional chance to also record from brain cells that are tuned to certain men and women.

So if you record from these single neurons, you can also do astounding items. So other folks, but also in our lab, from time to time you can, for instance, make associations amongst stimuli.

So suppose that, when you associate a renowned individual, say Jennifer Aniston, with an additional renowned individual, say, Barack Obama, then we demonstrated that if folks recall these associations…so you give them a image of Jennifer Aniston, and you ask them to recall what was linked with them, then you have some neurons that only respond to Barack Obama. And then they will develop into active the moment you speak about Jennifer Aniston, and ask them to recall this association.

So these items I’m also quite excited about. Simply because the neurons that code these ideas are typically also promptly the ideas that are essentially in your consciousness, these are the items you happen to be considering about.

So that offers you a quite close hyperlink to what is truly on the subject’s thoughts. And what you can see in the activity of neurons, which I obtain fascinating.

Yeah, so we are not recreating the eye. So we’re just skipping it. So I consider that is also why the vision that we’re going to generate is, it is just a great deal much less excellent than the regular vision.

Simply because we are implanting electrodes in the person brain, and if we stimulate these, we activate a set of neurons that would ordinarily by no means be activated in that constellation, That offers you just a dot of lights.

And it does not give us the possibility to build diverse colours, for instance, mainly because neurons that are chosen for diverse colours are intermingled, and you can’t just selectively only activate the green cells, or the blue cells, or red cells. So that is why it is somewhat rudimentary.

But the challenges if you sort of recognize that you happen to be by no means going to be as excellent as regular vision, then are to get a excellent coverage of the visual fields mainly because of the nature of the map of the outdoors planet in the brain.

You have to recognize that the key visual cortex, which is the initial area exactly where the details comes from, the visual details that is processed in the cortex is large. It has a surface location of 25 square centimetres on the left, and an additional 25 square centimetres on the ideal.

And to get wires everywhere in that area, which is also rather folded, is going to be difficult. So that is one particular of the massive challenges that we’re considering about, how to make positive that we cover the map with electrodes.

If you only cover a little component of the map with electrodes, then the topic is only going to be in a position to see in a little area of the visual field, then they will be blind at all other places. That is rather undesirable.

A different massive challenge is to make an interface with electronics in the brain that has a enough longevity.

So we are now at the moment applying so-referred to as Juta electrode arrays. So these are arrays of stiff silicone shanks, we get in touch with them. So like, like a bed of nails, is essentially what it appears like.

And we know that they operate, commonly for a year, possibly a small bit longer. But you know, you do not want to implant a patient with a prosthetic device to obtain out that immediately after one particular or two years these electrodes are encapsulated by glial cells. So essentially the fibrosis, fibrous tissue that encapsulates the electrodes, and you drop the get in touch with with the nerve cells. So in that case, you can’t effectively stimulate any longer.

So that is an additional challenge. We have to obtain electronic components that have enough longevity. So if you implant them now, they will nonetheless be functioning, say, in 5 years, or ten years, or even 15 years. I consider these are two key challenges.

Pieter Roelfsema: 16:58

So I get the occasional request. And I have to clarify to these folks who get in touch with me, this is not a clinically authorized device. So it is analysis. And our ambition will be to go to humans in the subsequent say, two years, or possibly a small bit later.

But in that case, it is nonetheless going to be analysis. So do not count on from us in the coming 5 years a remedy. It is just analysis. And of course, the analysis is quite essential mainly because it really is going to assist us make the subsequent step, and go towards a device that is clinically authorized.

Ahead of we are there there are all sorts of regulations, which are there for a excellent cause. And we have to show that we comply with all these regulations.

With the technologies we’re applying now, it really is usually going to be rudimentary. But I would be quite excited if you are in a position to build a prosthesis that has, say 1000, or even ten,000, or even 50,000 pixels, really should recognize that your eye has 1 million pixels.

So if you count the quantity of fibres in the optic nerves, it is about 1 million. So 50,000 is what we may well aim for at some point. Sounds ambitious, but it really is only five% of the regular, of the regular eye. And it is, that is going to be difficult, but if I would appear back on my profession, and we would have been in a position to build a device that has 50,000 pixels, and many folks are applying it and it is catching up, I will be tremendously happy about it.

Jean Mary Zarate: 18:51

Now that is it for this episode of Tales From the Synapse. I’m Jean Mary Zarate, a senior editor at Nature Neuroscience. The producer was Don Byrne. Thanks once again to Professor Pieter Roelfsema. And thank you for listening.