Restoring Vision: Code Breaking and Optogenetics

On Tech and Vision Podcast with Dr. Cal Roberts

The Enigma machines that Germany used to encode messages during World War II were notorious for their complexity. Two Enigma experts — Dr. Tom Perera, a retired neuroscientist, and founder of EnigmaMuseum.com, and Dr. Mark Baldwin, an expert on the story of Enigma machines — tell us how the Allies were able to crack the code, by using input-output mapping.

The human brain is similarly complex. Until recently, no one knew the code the retina used to communicate with the brain to create sight. Our guest Dr. Sheila Nirenberg, a neuroscientist at Weill Cornell, and Principal and Founder of Bionic Sight has — using input-output mapping —  cracked the retina’s neural code, enabling her to recreate the electric signals to the brain that could restore sight in people with retinal degeneration. She has created a set of goggles that convert a camera’s images into the code, via pulses of light. And she relies on optogenetics, a relatively new procedure in neuroscience that helps neurons become responsive to light. In her clinical trial, Dr. Nirenberg injects the optogenetic vector into the eye, and trial participants who are completely blind, like Barry Honig, who we speak with on this program, report being able to see light. In early studies, coupling the effects of the optogenetics with the code-enabled goggles has an even more impressive effect on patients’ vision. Dr. Nirenberg is also using her knowledge of the visual neural code to inform machine learning applications that could also be further used to support people who are blind or visually impaired. Clinical trial participants are important partners in the journey of discovery, Dr. Nirenberg says. Barry Honig agrees. He was happy to participate to help ease the burden on future children diagnosed with eye diseases that would otherwise result in blindness, but thanks to these advancements, someday may not. 

Podcast Transcription

Woman: We’ve already tried 400 rotor settings.  They’ll change the codes at midnight.

Man: We can’t work day-to-day like this.  We have to break Enigma.  The outcome of the war depends on it.

Roberts:  The Enigma machine, the ciphering and deciphering machine employed by Germany in WW II is notorious for its complexity.

Perera:  The complexity and the beauty of this machine is that every time you type in a letter it closes a switch, and so it completely rewires the internal wiring of the Enigma and lights up a different letter every time you press the keyboard.

Roberts:  Dr. Tom Perera is a retired neuroscientist and a collector of Enigma machines.  He runs the online website EnigmaMuseum.com and has published a book detailing the inner workings of Enigma.

Perera:  For instance, if you press the letter T-T-T, you won’t get T-T-T or even Q-Q-Q.  You’ll get P-Z-K, because the wiring has changed with every single input.

Baldwin:  The machine is designed to encipher a message.

Roberts:  This is Dr. Mark Baldwin, also known as Dr. Enigma.  He, too, is an expert on the Enigma machine and its history.

Baldwin:  WW I saw a huge increase in the use of wireless.  Wireless provides no security because anyone can listen into any wireless message.

Roberts:  So Germany’s use of wireless transmission in WW II necessitated advanced encryption.

Baldwin:  And for that purpose the Enigma machine was developed.  So, it’s a portable electromechanical device, and all it does is to change one letter into another.  You type in the plain text, obviously it would be German, usually.  You’d type in the German letters which make up the German words of the message which you want to encipher and then you want to transmit.  If the German receiver types into his Enigma machine the string of Morse code letters that have come over the air as a wireless message, then the machine itself, without any difficulty, will turn the cyphered text back into the plain text.

There was a little red settings book which each operator had to have, and obviously the man to whom he was sending the message had to have a copy of the same settings book.  For most of the war, the codes, or the settings, changed every day at midnight.  But, there’s one final bit of information that’s not included in the book.  And that is the initial settings of the rotors.

Roberts:  So, how did the allies figure out the original settings of the rotors?  It didn’t start in England.  It started long before the war in Poland.

Baldwin:  We must not take away from the Poles the credit for having been the first people ever to break Enigma.  Because nobody else thought it was possible.  About five weeks before the start of the war, they realized they were going to be invaded.  And they decided to destroy everything relating to the success of Enigma decryption.  But, before they destroyed it, they called the British and the French over to Warsaw, and then gave to the British and the French everything that they knew about the machine and how the cyphers might be broken.  They gave us extremely useful information.

Roberts:  Polish intelligence was able to decipher Enigma messages, Dr. Baldwin says, because of a fatal flaw the Germans were making in their message design regarding how they described the initial position of the rotors.

Baldwin:  The three three letter group, obviously the 17,500 of those they could have chosen, they were instructed to type it into their machines twice and it’s a cardinal sin to have any repetition in a cipher technique, because if you do have and the enemy recognizes it, it’s potentially a weakness.  And this was the potential weakness that the Poles exploited for a way of tackling individual messages.  But Turing realized that we needed literally a more generalized and powerful attack on the ciphers than the ones the Poles were using.

Roberts:  Dr. Baldwin is talking about Alan Turing, the now-famous English mathematician known for the cryptanalysis of the Enigma.  He’s also known for designing the proto-computer and for conceiving of the Turing test, which measures a machine’s ability to exhibit intelligent behavior.  And Alan Turing was right.

Baldwin:  That flaw in the instructions, in the official instructions given to the German operators, that flaw was removed in May 1914, and Turing realized this might happen and to that purpose he designs this giant piece of machinery.  It’s a huge, ingenious electro-mechanical testing machine.

Turing’s giant machine, which weighs about 3/4 ton, called the Bombe, confusingly.  It cannot test all the possible settings of Enigma.  The codebreakers have to make a guess in the early hours of the morning, and because the settings of every Enigma machine changed at midnight they have to make an inspired guess that a particular string of letters in a message represents a particular German world or phrase or a name or a rank or a destination or a place name or something like that.

Roberts:  Dr. Tom Perera:

Perera:  Or the word Hitler, or the word General, or the word One.  One of the most common words in the entire German vocabulary is the word “one.”  In German E-I-N-S.  And so, you could look at the electrical activity in an Enigma machine and figure out what the word “eins” might be doing inside the Enigma.

Baldwin:  And the Bombe, this giant machine tests 36 possible sets of Enigma machines simultaneously through 17,500 possibilities to see whether any of the possible arrangements could have produced the cyphertext.  And the machine either says yes, you’re right or no, you’re wrong.  And if it finds a potential match, it then stops.

You actually then have to test that again.  Put the starting position in as revealed by the Bombe and see whether typing the intercept in gave you a sensible German language output.

Roberts:  In addition to collecting and writing about Enigma machines, Dr. Perera is a trained neuroscientist.

Perera:  The analogy between the Enigma and the human brain is rather interesting.  The brain has, and it’s an arguable number, 10 billion brain cells.  And those cells communicate with each other and do all sorts of things in the brain that allow us to be humans, think, and so on.  The Engima machine has 10 to the 114th power possible internal connections.  And that is actually a lot more.

Roberts:  Today’s big idea is very, very big.  Our guest, Dr. Sheila Nirenberg, neuroscientist at Weill Cornell, and Principal and Founder of Bionic Sight, a company the researches new treatments for blindness, has devised a system that may be able to restore sight by bypassing the eyes, without surgery and without implants.  We’re going to break Dr. Nirenberg’s incredible discovery into two parts:  Code Breaking and Optogenetics, two complex and fascinating components of her groundbreaking system.

First, Code Breaking.

Rather than deciphering military codes during wartime, Dr. Nirenberg has deciphered the retina’s neural code – the code that the brain uses to interpret visual images.

Nirenberg:  So, the retina is a complicated system.  It’s an image processor.  And the information goes from the photoreceptors through the retinal circuitry, which is complicated, and ends up in the ganglion cells.  The ganglion cells,  as you know, are the ones that form the optic nerve.  There’s a huge amount of processing that goes on here.  There are a hundred million photoreceptors and one million ganglion cells.  so, what the retina is doing is it’s distilling that information.  It’s pulling out features, features about objects or emotions.  Millions of years of evolution honed the retina into pulling out the right features and then it converts those features into electrical pulses and sends them up to the brain.  That’s what a normal retina does.

When you get a retinal degenerative disease, you lose all that front end part, but you still got your ganglion cells.  So, my claim to fame is cracking that code.

Roberts:  Explain to us what you mean by the term neural code.

Nirenberg:  When I say neural code I mean the mathematical transformation from the visual world, like say a camera, into electrical pulses that carry the information.  We call it the code because it’s something that the brain needs in order to understand the world.  It’s putting it into a secret language that the brain understands that if you were just looking at it you couldn’t tell what it meant.

Roberts:  I asked Dr. Nirenberg to break this down for me.  What did she have to do to break the neural code?  What was the process like?

Nirenberg:  The beauty of the retina – you can take a retina out of an animal, and you could do it with a human, also – and you can put it in a dish and pump saline through it and glucose and keep it alive.  You can’t take a brain out and keep it alive for very long.  But a retina will stay alive for several hours.

So, I can take a retina out and put it on a bed of electrodes so that I can record from the optic nerve cells.  Meanwhile, I send in video, like I went to Central Park and took movies of all sort of things.  Artificial stimuli.  Lots of different kinds of movies.  You’re watching the retina watch a movie and you’re seeing what it would be sending to your brain.  So it will land on the photoreceptors, the input side, while I’m recording from the ganglion cells – the output side.

And I can map the relationship between what was coming in and what those patterns of electrical pulses are.  And then I just used a mathematical technique Bayseians call Maximum Likelihood to map the relationship between the output and the input.  So, I could just get a set of equations that could take me from input to output for any stimulus for any kind of movie.

Roberts:  The way that Alan Turing’s Bombe machine did input output mapping for messages encoded by Enigma machines, Dr. Nirenberg did input output mapping for the visual messages going into our retinas.  But the code she broke was not a code that would be changing at midnight.  The code she broke is a code that has been evolving in humans for all of human history.  How our eyes communicate with our brain is advanced, sublet and elegant.   And now, thanks to Dr. Nirenberg, replicable.

But, that’s just the first part of the story.  Dr. Nirenberg still had to figure out how to get the coded messages into the brain.

Nirenberg:  And that’s where optogenetics comes in.  Something that found by Pan and Boyden and Deisseroth and several others – originally it’s an algae protein, and it has the ability to, if you shine light on it, it will make a neuron fire.  It will send an electrical pulse.

Roberts:  So, what is optogenetics?  Optogenetics pioneers have been a special kind of genetically modified virus based on light responsive algae that can recombine its DNA with the DNA of the host cells.  And, when researches inject this gene therapy vector into retinal ganglion cells, these neurons become responsive to light.

Optogenetics is a game changer in neuroscience helping researches study different parts of the brain.  So, how is Dr. Nirenberg using it in her work?

Nirenberg:  It works kind of as a tool that if we could send a neuro code into those cells in light patterns it could send that code up to the brain.  So, that was really the issue.  I had the code but now I had to figure out how to get it into the brain, so I used optogenetics.

Roberts:  So, vectors are actually viruses that have been specially engineered in order to transport the molecular DNA or RNA necessary in order to get the cells to do what we want them to do.

Nirenberg:  Right.  So, all gene therapy is a virus, but not a scary virus.  It’s like a cold virus.  You just take out all the bad genes, put in your gene and then shoot it into the eye.

Roberts:  And so, it’s an injection into the eye much the way that people who, with macular degeneration, get injections into their eyes, correct?

Nirenberg:  Exactly.  It takes two seconds.  They numb it and then they inject it.  But it’s only once so you don’t have to keep coming back.  I mean, we don’t know for sure that you don’t have to come back in a few years, but right now it’s not every three months the way it is for macular degeneration.

Roberts:  And that’s because the genetic information that you’re injecting gets imbedded into the cells, becomes part of those cells, and so, permanently changes the cells rather than the temporary effect you might get say, from a medication.

Nirenberg:  Exactly.  So it keeps making that protein that you need, steadily.

Roberts:  We are at the early stages, but, as you pointed out, one without the other wasn’t sufficient.  You had to know the code.  You had to know how retinas speak to optic nerves.  And then you had to get the optic nerves to receive it.  So, we’ve spoken about retinal prostheses in the past, and about putting in a chip in the back of your eye that can then convert light into an electrical signal.  How is that different from what you’re doing?

Nirenberg:  Well, I could have done this with the code and that electrode array.  But the problem is that that array – it’s just not enough electrodes, and each one targets too many cells.  When you do it with optogenetics you can get 10,000 cells as opposed to, let’s say 64.  And you can get really fine resolution.  Individual cells.  Not a whole bunch at once.

However, what that company Second Sight is doing now in the brain where they’re putting implants in to drive cortical cells is quite fantastic.  It opens the door for so many patients because we can only help people who have retinal degenerative diseases like RP and macular degeneration.  What about people with glaucoma or have an accident where they have a detached retina.  The cortical implants open the door to helping those patients.

Bob Greenberg, who runs that company, is a great guy and I think the world of him.

Roberts:  And we do, too.  Part and parcel of this are some special glasses that people wear.  Explain what these glasses are.

Nirenberg:  These glasses contain a device that has the neural code.  It has a camera and it will take images in as you’re looking and then will covert it to neural code.  And then it will convert the output  – the neural code output would be electrical pulses.  So we just turn them into light pulses to drive the optogenetic gene.  They wear the glasses and they can look around.  We have a fixed set of glasses that everybody uses right now, although we’re making a wearable one.

And so, you need the combination of the to.  So, you have the optogenetic in your eye – you can see a little bit.  For some patients they can see a little.

Honig:  I went from sort of bare light perception to, this past Chanukah, I said to my wife, I think it was 8th day, the Menorah was lit.

Roberts:  This is Barry Honig, one of Dr. Nirenberg’s first trial participants.

Honig:  And I said, that’s where the candles are, right?  And she said yeah, you could see that?  I said, yeah, I can see the light of the Menorah.  I was the second patient in the clinical trial and the whole thing is that you have to start at one dose and then she has to progress her way up, but needed volunteers to get the process going.  So, I was one of the early volunteers.

Even with the lowest dose what’s been measured is something like 80 times improvement in light perception in my left eye.  As months go by it has gotten better improved.  I can, at this point, the refrigerator is open.  Which unfortunately I do far too often.  I can see the light coming on in the refrigerator, which I haven’t been able to do for a long time.  And actually, on sunny days I really have to wear sunglasses because I perceive the light as being very bright.

Roberts:  Barry has had a long career in technology, banking, commodity trading and risk management.  And has even founded his own executive search and management consulting firm called Honig International.  And he has LCA, Leber Congenital Amaurosis.

Barry Honig:  People who have had larger doses have had better improvement.  But, even going to the point of being able to see the Menorah or seeing the light coming on in the fridge or being able to see sunlight when I walk in the kitchen.  That’s pretty cool.

Roberts:  Like Barry, other patients have had measurable results on the optogenetics alone.  They can see even more when they pair the optogenetics with a code-enhanced set of glasses or goggles in her lab.  She describes the experience of another patient.

Nirenberg:  He has a black lab and it was running in the snow.  He couldn’t see the details of the dog, but he could see it.  So, that was very encouraging because it meant that the vector got in.  It was expressing.  Then we could pair it with the goggles for them to be able to see much more.  Then they can see objects moving to the left or right.  They can see when I’m waving at them with my hand.  They can see my fingers moving.

Now that we’ve gotten five patients into it, I just hired somebody to help us build a version of the goggles that patients can take home with them.  A wearable version.  Because they’ll learn so much faster.

Roberts:  Now, others in this area are doing what they call optogenetics but without the glasses.  Why would that be?

Nirenberg:  Because I figured out the code.  You’re embarrassing me and I don’t know what to say.  Another company, they also have goggles, but it’s not with a code.  It’s just with very bright light.  So these optogenetic proteins require bright light.  So they make goggles that deliver bright light to the patient.

Roberts:  What I’m trying to say is that while the injection of the genetic material primes these cells to be reactive, it’s more than just the cells being able to receive the light.  If they have to understand this light by means of the code.  And that’s really what makes your work so different and so unique compared to what anyone else has done.

Nirenberg:  Thank you.  With the regular optogenetics, the patient will still see light.  When we don’t have the goggles we get a feel for what the other researchers can do.  The patient will still see light.  Probably they’re be able to see motion, because it’s sort of like a big sweeping thing moving in front of you, but not a lot more than that.  So we really are trying so hard to do better.

Roberts:  Where do you go from here?  What’s the next step?

Nirenberg:  The next step is to go to a higher dose.  We had to start with a dose escalation study.  We had to start with the lowest dose  and the next dose up from that.  So we’ve done dose one and dose two and we got what we needed to know really, which is that the vector gets in, it expresses and we can drive it.  Of the first five patients, all of them can now see light and motion with several of them seeing the direction of motion or my hand waving.  And these were completely blind, or almost completely blind people.  And we had to start with very blind people for safety reasons.

And so we’ve now injected three people with the next dose up but we haven’t tested them yet because it’s still too early.  We expanded the population slightly.  One patient has already said that suddenly she can see that her couch is read.  She thought it was black all these years, and it’s red.  Little things are happening.  But I haven’t tested her myself.  This is anecdotal and it’s wonderful and I’m so happy, and she’ll come soon.  But we have to wait three months from the injection until we can do the testing because it takes a while for the gene to express.

And there’s one more dose after that, and that’s the dose I’m most counting on.  You’re catching us in the rising phase of this.  And we’ll see how far we can get.

Roberts:  But Dr. Nirenberg’s research isn’t limited to the human brain.  She’s taking her understanding of the retina’s neural code and using it to help machines see more like we do.

Nirenberg:  The same code that speaks to the brain through the optic nerve and tells the brain what you’re seeing could also be used in computer vision, machine vision.  I paired up the neural code with downstream machine vision algorithms to make robots that can see.  I started working on that and I partnered with Intel, the chip company.  I’ve worked with Ford, the car company on this also.

The code is fixed because evolution built it and it works.  If you have the code get sent to a machine learning algorithm, that algorithm will learn now using neural code, using the representation of the neural code rather than using the real world.  The brain is using the code, and the robot’s brain is using the code, but the code stays the way it is because I kind of trust the judgement of evolution.  You know what I mean – it really had a long time to get it right.  And it does this so well because evolution discovered that it’s much easier to learn from the neural code than it is to learn from the raw image.  The raw image has too much information in it.

If you’re looking at me right now and there’s a picture behind me, if you send that into a machine learning algorithm, how does it know that I’m the thing that you’re trying to learn and not the picture behind me.  But the neural code gets rid of a lot of these things for you.  It distills what’s important from the image.

Roberts:  And, Dr. Nirenberg, the developments in computer vision can have benefits for people who are blind and visually impaired as well.  Imagine, her neural code paired with existing smart glasses, like OrCam for example.  Dr. Nirenberg’s discovery of the neural code and her commitment to turning it into a system to potentially restore sight to people who are blind is extraordinary.

I mentioned to her that how important it is that people like Barry Honig volunteer to participate in her clinical trial.

Nirenberg:  It is.  It’s an extraordinary trait.  Of the first five patients, four of them are business people running their own companies.  Entrepreneurs.  So they have that risk-taking, brave personality.  These guys are heroes and they did this with me in the early parts.  And I just won’t let them down if there’s anything in my power that I can do.

What’s amazing about the patients that are participating in a clinical trial, they become collaborators, because, as you’re trying something you have your hypothesis of how it should look to them.  But you have to let them talk and tell you what does it look like to you so in case I have the wrong preconceived notion I can get the feedback.

So, a friendship developed as we’re figuring it out.  Initially they’ll call and tell you what their perceptions are.  then they come to the lab and test it quantitatively.  But there’s also just a whole part of exploratory work with them trying to understand what works, what doesn’t work.

Roberts:  And so, collaborations are not just scientist with scientist, but collaborations are scientists with patients working together to advance this.  Barry Honig:

Honig:  I’m an engineer by training, and you can’t advance science unless people are willing to do this stuff.  Obviously I wanted to see my wife, or see my daughter God-willing when she gets married walk down the aisle.  But, the truth of the matter is, when parents of a kid and the doctor says to them – look, I’m sorry, your kid’s got RP, LCA – whatever degenerative eye disease that this treatment may be able to help in the future.  And the doctor says to the parents, your child is going to be not quite blind yet, has low vision, maybe is already blind and is certainly going to go blind.  

Most parents either freak out about it or it’s obviously an incredibly disturbing thing for most parents because they think about, how do I provide the resources for the kid.  How do I raise the kid?  How to I deal with it.  The truth of the matter is, for most blind people it doesn’t end up particularly well as witnessed by the fact that there is a 70% unemployment rate in this country among visually impaired people, which is an astounding number.  And the idea that the doctor in the future could say, well, yes, your child has that diagnosis, but here’s the good news.  The good news is we’ve got this injection and we can give your child an injection in their eyes, and then we’ll be able to give them this set of cool looking goggles/glasses that they’ll be able to wear to supplement that optogenetic treatment and they’re going to be able to see okay.  Whether or not that’s perfect, time will tell.  But they’ll be able to function very well in the world.

To be able to spare a kid of having to grow up the way I did – to be able to spare kids that, and equally as importantly to spare their parents that.  If I could just take an injection and be part of it, then that’s a lot of reward.

Roberts:  Like the Enigma machine, the human brain is notorious for its complexity.  But by using input output mapping to decipher the neural code for the retina, Dr. Nirenberg has achieved something no one had before.  Like when Alan Turing invented the Bombe machine to decipher Enigma.  As Dr. Turing’s word inspired the new field of computer science, Dr. Nirenberg’s discovery will inspire new approaches in neuroscience.

For example, Dr. Nirenberg’s approach to mapping visual code could potentially be applied to the neural codes for other sensory modalities.  Who will be the researcher who breaks the auditory neural code?  The tactile neural code?  The olfactory neural code?  For her part, Dr. Nirenberg continues to apply her discovery to restoring human vision in people who are blind.  And we are so grateful she does.

If her trials are successful, she has a real chance to restore visual perception in patients with blindness.  Our brains may be enigmas, but thanks to work like Dr. Nirenberg’s, we are on our way to cracking the code.

I’m Dr. Cal Roberts and this is On Tech & Vision.  Thanks for listening.

Did this episode spark ideas for you?  Let us know at podcasts@lighthouseguild.org.  And if you liked this episode please subscribe, rate and review us on Apple Podcasts or wherever you get your podcasts.

I’m Dr. Cal Roberts.  On Tech & Vision is produced by Lighthouse Guild.  For more information visit www.lighthouseguild.org.  On Tech & Vision with Dr. Cal Roberts is produced at Lighthouse Guild by my colleagues Jaine Schmidt and Annemarie O’Hearn.  My thanks to Podfly for their production support.