Miguel Nicolelis: Brain-to-brain communication has arrived. How we did it

Recorded atOctober 15, 2014
EventTEDGlobal 2014
Duration (min:sec)18:44
Video TypeTED Stage Talk
Words per minute156.11 medium
Readability (FK)49.78 difficult
SpeakerMiguel Nicolelis
CountryBrazil
Occupationbiologist, physician, neuroscientist, neurologist
DescriptionBrazilian neuroscientist

Official TED page for this talk

Synopsis

You may remember neuroscientist Miguel Nicolelis — he built the brain-controlled exoskeleton that allowed a paralyzed man to kick the first ball of the 2014 World Cup. What’s he working on now? Building ways for two minds (rats and monkeys, for now) to send messages brain to brain. Watch to the end for an experiment that, as he says, will go to "the limit of your imagination."

Text Highlight (experimental)
     
100:12 On June 12, 2014, precisely at 3:33
200:18 in a balmy winter afternoon in São Paulo, Brazil,
300:23 a typical South American winter afternoon,
400:26 this kid, this young man that you see celebrating here
500:29 like he had scored a goal,
600:31 Juliano Pinto, 29 years old, accomplished a magnificent deed.
700:38 Despite being paralyzed
800:40 and not having any sensation from mid-chest to the tip of his toes
900:45 as the result of a car crash six years ago that killed his brother
1000:49 and produced a complete spinal cord lesion that left Juliano in a wheelchair,
1100:57 Juliano rose to the occasion, and on this day did something
1201:02 that pretty much everybody that saw him in the six years deemed impossible.
1301:09 Juliano Pinto delivered the opening kick
1401:14 of the 2014 Brazilian World Soccer Cup here
1501:20 just by thinking.
1601:22 He could not move his body,
1701:24 but he could imagine the movements needed to kick a ball.
1801:29 He was an athlete before the lesion. He's a para-athlete right now.
1901:32 He's going to be in the Paralympic Games, I hope, in a couple years.
2001:36 But what the spinal cord lesion did not rob from Juliano
2101:40 was his ability to dream.
2201:43 And dream he did that afternoon, for a stadium of about 75,000 people
2301:49 and an audience of close to a billion watching on TV.
2401:54 And that kick crowned, basically, 30 years of basic research
2502:00 studying how the brain,
2602:02 how this amazing universe that we have between our ears
2702:06 that is only comparable to universe that we have above our head
2802:10 because it has about 100 billion elements
2902:13 talking to each other through electrical brainstorms,
3002:16 what Juliano accomplished took 30 years to imagine in laboratories
3102:21 and about 15 years to plan.
3202:24 When John Chapin and I, 15 years ago, proposed in a paper
3302:28 that we would build something that we called a brain-machine interface,
3402:33 meaning connecting a brain to devices
3502:36 so that animals and humans could just move these devices,
3602:40 no matter how far they are from their own bodies,
3702:42 just by imagining what they want to do,
3802:44 our colleagues told us that we actually needed professional help,
3902:49 of the psychiatry variety.
4002:52 And despite that, a Scot and a Brazilian persevered,
4102:57 because that's how we were raised in our respective countries,
4203:02 and for 12, 15 years,
4303:04 we made demonstration after demonstration suggesting that this was possible.
4403:08 And a brain-machine interface is not rocket science,
4503:11 it's just brain research.
4603:13 It's nothing but using sensors
4703:16 to read the electrical brainstorms that a brain is producing
4803:19 to generate the motor commands
4903:21 that have to be downloaded to the spinal cord,
5003:24 so we projected sensors that can read
5103:26 hundreds and now thousands of these brain cells simultaneously,
5203:30 and extract from these electrical signals
5303:33 the motor planning that the brain is generating
5403:36 to actually make us move into space.
5503:39 And by doing that, we converted these signals into digital commands
5603:42 that any mechanical, electronic, or even a virtual device can understand
5703:47 so that the subject can imagine what he, she or it wants to make move,
5803:54 and the device obeys that brain command.
5903:57 By sensorizing these devices with lots of different types of sensors,
6004:01 as you are going to see in a moment,
6104:03 we actually sent messages back to the brain to confirm
6204:07 that that voluntary motor will was being enacted, no matter where --
6304:11 next to the subject, next door, or across the planet.
6404:16 And as this message gave feedback back to the brain,
6504:19 the brain realized its goal: to make us move.
6604:23 So this is just one experiment that we published a few years ago,
6704:27 where a monkey, without moving its body,
6804:29 learned to control the movements of an avatar arm,
6904:33 a virtual arm that doesn't exist.
7004:35 What you're listening to is the sound of the brain of this monkey
7104:39 as it explores three different visually identical spheres
7204:43 in virtual space.
7304:45 And to get a reward, a drop of orange juice that monkeys love,
7404:49 this animal has to detect, select one of these objects
7504:52 by touching,
7604:54 not by seeing it, by touching it,
7704:56 because every time this virtual hand touches one of the objects,
7804:59 an electrical pulse goes back to the brain of the animal
7905:02 describing the fine texture of the surface of this object,
8005:06 so the animal can judge what is the correct object that he has to grab,
8105:10 and if he does that, he gets a reward without moving a muscle.
8205:16 The perfect Brazilian lunch:
8305:18 not moving a muscle and getting your orange juice.
8405:22 So as we saw this happening,
8505:25 we actually came and proposed the idea that we had published 15 years ago.
8605:29 We reenacted this paper.
8705:31 We got it out of the drawers,
8805:33 and we proposed that perhaps we could get a human being that is paralyzed
8905:37 to actually use the brain-machine interface to regain mobility.
9005:42 The idea was that if you suffered --
9105:44 and that can happen to any one of us.
9205:46 Let me tell you, it's very sudden.
9305:48 It's a millisecond of a collision,
9405:50 a car accident that transforms your life completely.
9505:53 If you have a complete lesion of the spinal cord,
9605:56 you cannot move because your brainstorms cannot reach your muscles.
9706:00 However, your brainstorms continue to be generated in your head.
9806:04 Paraplegic, quadriplegic patients dream about moving every night.
9906:08 They have that inside their head.
10006:10 The problem is how to get that code out of it
10106:13 and make the movement be created again.
10206:16 So what we proposed was, let's create a new body.
10306:19 Let's create a robotic vest.
10406:21 And that's exactly why Juliano could kick that ball just by thinking,
10506:26 because he was wearing the first brain-controlled robotic vest
10606:30 that can be used by paraplegic, quadriplegic patients to move
10706:34 and to regain feedback.
10806:35 That was the original idea, 15 years ago.
10906:38 What I'm going to show you is how 156 people from 25 countries
11006:44 all over the five continents of this beautiful Earth,
11106:47 dropped their lives, dropped their patents,
11206:50 dropped their dogs, wives, kids, school, jobs,
11306:54 and congregated to come to Brazil for 18 months to actually get this done.
11406:59 Because a couple years after Brazil was awarded the World Cup,
11507:03 we heard that the Brazilian government wanted to do something meaningful
11607:07 in the opening ceremony
11707:08 in the country that reinvented and perfected soccer
11807:12 until we met the Germans, of course.
11907:14 (Laughter)
12007:16 But that's a different talk,
12107:18 and a different neuroscientist needs to talk about that.
12207:22 But what Brazil wanted to do is to showcase
12307:24 a completely different country,
12407:25 a country that values science and technology,
12507:28 and can give a gift to millions, 25 million people around the world
12607:32 that cannot move any longer because of a spinal cord injury.
12707:35 Well, we went to the Brazilian government and to FIFA and proposed,
12807:39 well, let's have the kickoff of the 2014 World Cup
12907:42 be given by a Brazilian paraplegic
13007:44 using a brain-controlled exoskeleton that allows him to kick the ball
13107:49 and to feel the contact of the ball.
13207:52 They looked at us, thought that we were completely nuts,
13307:54 and said, "Okay, let's try."
13407:57 We had 18 months to do everything from zero, from scratch.
13508:01 We had no exoskeleton, we had no patients,
13608:04 we had nothing done.
13708:06 These people came all together
13808:08 and in 18 months, we got eight patients in a routine of training
13908:12 and basically built from nothing this guy,
14008:16 that we call Bra-Santos Dumont 1.
14108:21 The first brain-controlled exoskeleton to be built
14208:24 was named after the most famous Brazilian scientist ever,
14308:28 Alberto Santos Dumont,
14408:30 who, on October 19, 1901, created and flew himself
14508:36 the first controlled airship on air in Paris for a million people to see.
14608:43 Sorry, my American friends,
14708:45 I live in North Carolina,
14808:47 but it was two years before the Wright Brothers flew
14908:52 on the coast of North Carolina.
15008:54 (Applause)
15108:57 Flight control is Brazilian. (Laughter)
15209:02 So we went together with these guys
15309:05 and we basically put this exoskeleton together,
15409:08 15 degrees of freedom, hydraulic machine
15509:11 that can be commanded by brain signals
15609:14 recorded by a non-invasive technology called electroencephalography
15709:18 that can basically allow the patient to imagine the movements
15809:22 and send his commands to the controls, the motors,
15909:26 and get it done.
16009:27 This exoskeleton was covered with an artificial skin
16109:30 invented by Gordon Cheng, one of my greatest friends, in Munich,
16209:34 to allow sensation from the joints moving and the foot touching the ground
16309:39 to be delivered back to the patient through a vest, a shirt.
16409:43 It is a smart shirt with micro-vibrating elements
16509:46 that basically delivers the feedback and fools the patient's brain
16609:51 by creating a sensation that it is not a machine that is carrying him,
16709:55 but it is he who is walking again.
16809:58 So we got this going, and what you'll see here
16910:01 is the first time one of our patients, Bruno, actually walked.
17010:06 And he takes a few seconds because we are setting everything,
17110:08 and you are going to see a blue light cutting in front of the helmet
17210:12 because Bruno is going to imagine the movement that needs to be performed,
17310:16 the computer is going to analyze it, Bruno is going to certify it,
17410:19 and when it is certified,
17510:21 the device starts moving under the command of Bruno's brain.
17610:24 And he just got it right, and now he starts walking.
17710:29 After nine years without being able to move,
17810:32 he is walking by himself.
17910:34 And more than that --
18010:36 (Applause) --
18110:39 more than just walking,
18210:41 he is feeling the ground,
18310:43 and if the speed of the exo goes up,
18410:45 he tells us that he is walking again on the sand of Santos,
18510:50 the beach resort where he used to go before he had the accident.
18610:54 That's why the brain is creating a new sensation in Bruno's head.
18710:57 So he walks, and at the end of the walk -- I am running out of time already --
18811:01 he says, "You know, guys,
18911:04 I need to borrow this thing from you when I get married,
19011:06 because I wanted to walk to the priest
19111:09 and see my bride and actually be there by myself.
19211:13 Of course, he will have it whenever he wants.
19311:16 And this is what we wanted to show during the World Cup, and couldn't,
19411:20 because for some mysterious reason, FIFA cut its broadcast in half.
19511:25 What you are going to see very quickly is Juliano Pinto in the exo doing the kick
19611:33 a few minutes before we went to the pitch
19711:35 and did the real thing in front of the entire crowd,
19811:38 and the lights you are going to see just describe the operation.
19911:41 Basically, the blue lights pulsating indicate that the exo is ready to go.
20011:46 It can receive thoughts and it can deliver feedback,
20111:49 and when Juliano makes the decision to kick the ball,
20211:52 you are going to see two streams of green and yellow light
20311:55 coming from the helmet and going to the legs,
20411:58 representing the mental commands that were taken by the exo
20512:02 to actually make that happen.
20612:04 And in basically 13 seconds,
20712:07 Juliano actually did.
20812:09 You can see the commands.
20912:11 He gets ready, the ball is set, and he kicks.
21012:15 And the most amazing thing is,
21112:17 10 seconds after he did that, and looked at us on the pitch,
21212:20 he told us, celebrating as you saw,
21312:22 "I felt the ball."
21412:25 And that's priceless.
21512:27 (Applause)
21612:29 So where is this going to go?
21712:31 I have two minutes to tell you
21812:33 that it's going to the limits of your imagination.
21912:36 Brain-actuating technology is here.
22012:38 This is the latest: We just published this a year ago,
22112:41 the first brain-to-brain interface
22212:43 that allows two animals to exchange mental messages
22312:46 so that one animal that sees something coming from the environment
22412:50 can send a mental SMS, a torpedo, a neurophysiological torpedo,
22512:56 to the second animal,
22612:57 and the second animal performs the act that he needed to perform
22713:01 without ever knowing what the environment was sending as a message,
22813:05 because the message came from the first animal's brain.
22913:09 So this is the first demo.
23013:11 I'm going to be very quick because I want to show you the latest.
23113:15 But what you see here is the first rat getting informed
23213:21 by a light that is going to show up on the left of the cage
23313:24 that he has to press the left cage to basically get a reward.
23413:27 He goes there and does it.
23513:29 And the same time, he is sending a mental message
23613:31 to the second rat that didn't see any light,
23713:35 and the second rat, in 70 percent of the times
23813:37 is going to press the left lever and get a reward
23913:41 without ever experiencing the light in the retina.
24013:45 Well, we took this to a little higher limit
24113:49 by getting monkeys to collaborate mentally in a brain net,
24213:54 basically to donate their brain activity
24313:56 and combine them to move the virtual arm that I showed you before,
24414:00 and what you see here is the first time the two monkeys combine their brains,
24514:05 synchronize their brains perfectly to get this virtual arm to move.
24614:09 One monkey is controlling the x dimension,
24714:12 the other monkey is controlling the y dimension.
24814:14 But it gets a little more interesting when you get three monkeys in there
24914:19 and you ask one monkey to control x and y,
25014:22 the other monkey to control y and z,
25114:25 and the third one to control x and z,
25214:28 and you make them all play the game together,
25314:30 moving the arm in 3D into a target to get the famous Brazilian orange juice.
25414:36 And they actually do.
25514:38 The black dot is the average of all these brains working
25614:43 in parallel, in real time.
25714:46 That is the definition of a biological computer,
25814:49 interacting by brain activity and achieving a motor goal.
25914:53 Where is this going?
26014:55 We have no idea.
26114:58 We're just scientists.
26214:59 (Laughter)
26315:01 We are paid to be children,
26415:03 to basically go to the edge and discover what is out there.
26515:07 But one thing I know:
26615:09 One day, in a few decades,
26715:11 when our grandchildren surf the Net just by thinking,
26815:15 or a mother donates her eyesight to an autistic kid who cannot see,
26915:19 or somebody speaks because of a brain-to-brain bypass,
27015:22 some of you will remember that it all started on a winter afternoon
27115:28 in a Brazilian soccer field with an impossible kick.
27215:32 Thank you.
27315:34 (Applause)
27415:43 Thank you.
27515:59 Bruno Giussani: Miguel, thank you for sticking to your time.
27616:03 I actually would have given you a couple more minutes,
27716:05 because there are a couple of points we want to develop, and, of course,
27816:08 clearly it seems that we need connected brains to figure out where this is going.
27916:12 So let's connect all this together.
28016:14 So if I'm understanding correctly,
28116:16 one of the monkeys is actually getting a signal
28216:18 and the other monkey is reacting to that signal
28316:21 just because the first one is receiving it and transmitting the neurological impulse.
28416:25 Miguel Nicolelis: No, it's a little different.
28516:27 No monkey knows of the existence of the other two monkeys.
28616:30 They are getting a visual feedback in 2D,
28716:33 but the task they have to accomplish is 3D.
28816:36 They have to move an arm in three dimensions.
28916:38 But each monkey is only getting the two dimensions on the video screen
29016:42 that the monkey controls.
29116:44 And to get that thing done,
29216:47 you need at least two monkeys to synchronize their brains,
29316:49 but the ideal is three.
29416:51 So what we found out is that when one monkey starts slacking down,
29516:55 the other two monkeys enhance their performance
29616:57 to get the guy to come back,
29716:59 so this adjusts dynamically,
29817:02 but the global synchrony remains the same.
29917:06 Now, if you flip without telling the monkey
30017:09 the dimensions that each brain has to control,
30117:11 like this guy is controlling x and y,
30217:13 but he should be controlling now y and z,
30317:16 instantaneously, that animal's brain forgets about the old dimensions
30417:20 and it starts concentrating on the new dimensions.
30517:23 So what I need to say is that no Turing machine,
30617:26 no computer can predict what a brain net will do.
30717:30 So we will absorb technology as part of us.
30817:33 Technology will never absorb us.
30917:35 It's simply impossible.
31017:38 BG: How many times have you tested this?
31117:41 And how many times have you succeeded versus failed?
31217:44 MN: Oh, tens of times.
31317:46 With the three monkeys? Oh, several times.
31417:48 I wouldn't be able to talk about this here unless I had done it a few times.
31517:53 And I forgot to mention, because of time,
31617:55 that just three weeks ago, a European group
31717:59 just demonstrated the first man-to-man brain-to-brain connection.
31818:04 BG: And how does that play?
31918:06 MN: There was one bit of information -- big ideas start in a humble way --
32018:10 but basically the brain activity of one subject
32118:17 was transmitted to a second object, all non-invasive technology.
32218:21 So the first subject got a message, like our rats, a visual message,
32318:25 and transmitted it to the second subject.
32418:28 The second subject received a magnetic pulse in the visual cortex,
32518:32 or a different pulse, two different pulses.
32618:36 In one pulse, the subject saw something.
32718:38 On the other pulse, he saw something different.
32818:40 And he was able to verbally indicate
32918:42 what was the message the first subject was sending
33018:45 through the Internet across continents.
33118:48 Moderator: Wow. Okay, that's where we are going.
33218:51 That's the next TED Talk at the next conference.
33318:53 Miguel Nicolelis, thank you. MN: Thank you, Bruno. Thank you.
S M L