| Recorded at | July 20, 2019 |
|---|---|
| Event | TEDSummit 2019 |
| Duration (min:sec) | 11:30 |
| Video Type | TED Stage Talk |
| Words per minute | 148.6 very slow |
| Readability (FK) | 48.53 difficult |
| Speaker | Diego Prilusky |
Official TED page for this talk
Synopsis
In this talk and tech demo, filmmaker Diego Prilusky introduces the next chapter in moviemaking: volumetric video, a 360-degree experience powered by hundreds of cameras that capture light and motion from every angle. Check out how this technology creates a new type of immersive storytelling -- and catch Prilusky's reshoot of an iconic dance number from "Grease" captured with volumetric video.
| 1 | 00:12 | I love making movies. | ||
| 2 | 00:17 | Motion pictures have been in existence for more than a hundred years. | ||
| 3 | 00:22 | Filmmaking hasn't changed for the dimensional mindset. | ||
| 4 | 00:27 | Placing the camera in a scene and pressing "record" hasn't changed. | ||
| 5 | 00:31 | Filmmaking is still a frontal experience, | ||
| 6 | 00:34 | and creating the film has the possibility | ||
| 7 | 00:37 | to follow the same direction of the content creation. | ||
| 8 | 00:42 | We still stand in front of a flat image, | ||
| 9 | 00:45 | watching the fiction. | ||
| 10 | 00:48 | There's nothing wrong with it. | ||
| 11 | 00:49 | I love watching movies and going to the theaters. | ||
| 12 | 00:52 | The experiences can be such emotional experiences. | ||
| 13 | 00:55 | The art and craft of emotional experiences within a frame | ||
| 14 | 00:59 | can be so strong to drive a stronger emotion. | ||
| 15 | 01:03 | The question we're asking is, | ||
| 16 | 01:05 | How the experience of motion pictures can exist beyond the flat screen. | ||
| 17 | 01:10 | How can we start creating content | ||
| 18 | 01:12 | for the next generation of content experiences? | ||
| 19 | 01:16 | Traditionally, when we imagine a scene, | ||
| 20 | 01:19 | we look at the frame and the composition. | ||
| 21 | 01:22 | We have to think about how we create depth and parallax | ||
| 22 | 01:25 | using foreground, background elements as the camera moves. | ||
| 23 | 01:30 | With the technology today and devices of VR glasses, | ||
| 24 | 01:33 | AR glasses, smart devices, | ||
| 25 | 01:36 | allowing three-dimensional and full navigation in space, | ||
| 26 | 01:40 | we have the possibility to enable audiences | ||
| 27 | 01:43 | to experience content from multiple perspectives. | ||
| 28 | 01:46 | What we have to think about is how we take this technology, | ||
| 29 | 01:50 | all the capabilities, | ||
| 30 | 01:51 | and enable the experience to move farther away inside the scene. | ||
| 31 | 01:57 | Now we're not talking about video games or computer-generated actors, | ||
| 32 | 02:01 | which look tremendously realistic. | ||
| 33 | 02:04 | We're talking about real actors and real performance, | ||
| 34 | 02:07 | performing onstage. | ||
| 35 | 02:10 | We have to start thinking how we capture the actors | ||
| 36 | 02:14 | and how we capture the real scene | ||
| 37 | 02:16 | in order to immerse inside. | ||
| 38 | 02:18 | Now, we're familiar with the 360-degree video, | ||
| 39 | 02:21 | where you place a camera inside the scene | ||
| 40 | 02:24 | and you can create this beautiful panoramic image all around you, | ||
| 41 | 02:28 | but from the same aspect, | ||
| 42 | 02:29 | filmmaking is still frontal. | ||
| 43 | 02:32 | In order to emerge fully inside the scene, | ||
| 44 | 02:35 | we will need to capture the light from all the possible directions. | ||
| 45 | 02:39 | We will have to surround the scene with an enormous amount of sensors, | ||
| 46 | 02:44 | with all possible capabilities to capture the light | ||
| 47 | 02:47 | and enable us to emerge inside afterwards again. | ||
| 48 | 02:51 | Now, in this setup, | ||
| 49 | 02:53 | there's no more foreground or background | ||
| 50 | 02:55 | or a camera placed in space | ||
| 51 | 02:57 | but hundreds of sensors capturing the light | ||
| 52 | 03:01 | and capturing the motion from all the possible directions. | ||
| 53 | 03:05 | With the new technological advancements, | ||
| 54 | 03:07 | we can start looking at 3D photography, | ||
| 55 | 03:09 | capturing the light from multiple perspectives, | ||
| 56 | 03:12 | enabling us to reconstruct the object. | ||
| 57 | 03:15 | This is like photography in 3D space. | ||
| 58 | 03:18 | Now, with these technological advancements, | ||
| 59 | 03:21 | we can record video not just as a flat image | ||
| 60 | 03:24 | but as a volume. | ||
| 61 | 03:26 | This is what we call "volumetric video," | ||
| 62 | 03:29 | and it has the capability to record every action of the scene | ||
| 63 | 03:32 | as a full three-dimensional volume. | ||
| 64 | 03:35 | Now, what is a voxel? | ||
| 65 | 03:38 | A voxel is like a three-dimensional pixel, | ||
| 66 | 03:41 | but instead of being a flat image square, staying light and colored, | ||
| 67 | 03:46 | it's like a three-dimensional cube in space, | ||
| 68 | 03:49 | with x, y, and z positions. | ||
| 69 | 03:53 | This enables us to create a full capture of the scene | ||
| 70 | 03:56 | from any perspective. | ||
| 71 | 03:58 | Now this renders a fully light-immersive scene | ||
| 72 | 04:03 | from multiple perspectives. | ||
| 73 | 04:06 | This capability requires an insane amount of information to be processed. | ||
| 74 | 04:10 | We will have to capture the light from an enormous amount of cameras | ||
| 75 | 04:15 | to create this information. | ||
| 76 | 04:18 | Now, in order to do such a thing, | ||
| 77 | 04:21 | we would need a setup that would host a numerous amount of cameras | ||
| 78 | 04:28 | installed in a stage | ||
| 79 | 04:30 | and a stage big enough in order to fit a full cinematic experience. | ||
| 80 | 04:36 | Now that sounds like a crazy idea, but that's exactly what we did. | ||
| 81 | 04:41 | For the last three years, | ||
| 82 | 04:43 | we have been building a huge volumetric camera chamber. | ||
| 83 | 04:48 | It's 10,000 square feet of a stage, | ||
| 84 | 04:51 | enabling to capture the action from any location. | ||
| 85 | 04:55 | We have deployed hundreds of cameras, | ||
| 86 | 04:57 | sending a tremendous amount of information | ||
| 87 | 04:59 | to a huge data center powered by Intel supercomputers. | ||
| 88 | 05:04 | The ability to have this 10,000 feet | ||
| 89 | 05:08 | enables us to fit any kind of action, | ||
| 90 | 05:11 | any kind of performance. | ||
| 91 | 05:13 | It is the size of an average Broadway stage. | ||
| 92 | 05:17 | We call it Intel Studios, | ||
| 93 | 05:19 | and it's the largest volumetric stage in the world, | ||
| 94 | 05:22 | with the objective of enabling and exploring | ||
| 95 | 05:26 | the next generation of this immersive media filmmaking. | ||
| 96 | 05:30 | Now, to test these ideas, | ||
| 97 | 05:33 | we were thinking about what we can do as the first scene to try it out. | ||
| 98 | 05:39 | So we chose the Western scene. | ||
| 99 | 05:42 | We brought horses, set designers, dirt, | ||
| 100 | 05:46 | everything needed to create the full scene of a Western. | ||
| 101 | 05:50 | But this time, there was no camera inside. | ||
| 102 | 05:54 | There was nothing really moving besides all the cameras | ||
| 103 | 05:58 | installed outside. | ||
| 104 | 06:00 | The challenge of the actors was tremendous. | ||
| 105 | 06:03 | They have to perform a flawless action visible from all the directions. | ||
| 106 | 06:09 | There's no possibility to hide a punch or not show the action. | ||
| 107 | 06:14 | Everything is captured and everything is seen. | ||
| 108 | 06:17 | The output of the capture -- | ||
| 109 | 06:21 | this is our future capture -- | ||
| 110 | 06:22 | opened our eyes for the immense capabilities. | ||
| 111 | 06:25 | It was like a full 3D scan of the entire scene. | ||
| 112 | 06:29 | We were able to move around and travel in the space. | ||
| 113 | 06:33 | The thing about this, | ||
| 114 | 06:34 | it's not anymore about perceiving the light emitted from a screen | ||
| 115 | 06:38 | but now traveling inside the light, | ||
| 116 | 06:41 | traveling inside the scene. | ||
| 117 | 06:42 | This obviously opens possibilities for an enormous amount | ||
| 118 | 06:45 | of storytelling and methodologies of creation. | ||
| 119 | 06:49 | This is the possibilities of your personal narrative, | ||
| 120 | 06:53 | the possibility of creating your own story inside, | ||
| 121 | 06:56 | or maybe following other stories. | ||
| 122 | 06:59 | Let's take a look at one of the last renders and see. | ||
| 123 | 07:01 | (Music) | ||
| 124 | 07:03 | What you're seeing here is full volumetric video, | ||
| 125 | 07:06 | and there's no physical camera in the scene. | ||
| 126 | 07:08 | (Music) | ||
| 127 | 07:10 | We have the full control | ||
| 128 | 07:11 | (Music, sounds of combat) | ||
| 129 | 07:13 | of space and time. | ||
| 130 | 07:15 | (Music, sounds of combat) | ||
| 131 | 07:26 | Now, again, no physical camera was here. | ||
| 132 | 07:29 | Everything was captured surrounding. | ||
| 133 | 07:31 | Now, this is very nice, | ||
| 134 | 07:34 | but what if we wanted to see the scene, maybe, from the eyes of the horse? | ||
| 135 | 07:38 | Well, we can do that as well. | ||
| 136 | 07:41 | (Horse galloping) | ||
| 137 | 07:42 | So what you're seeing right now is the same action, | ||
| 138 | 07:45 | but this time, we're watching exactly from the eyes of the horse. | ||
| 139 | 07:49 | The possibilities are, well, unlimited. | ||
| 140 | 07:53 | (Applause) | ||
| 141 | 07:54 | Thank you. | ||
| 142 | 07:55 | (Applause) | ||
| 143 | 07:58 | So this is all great for creators and storytellers. | ||
| 144 | 08:03 | It really opens a huge canvas | ||
| 145 | 08:05 | for a different type of storytelling and moviemaking. | ||
| 146 | 08:08 | But what about the audience? | ||
| 147 | 08:09 | How can the audience experience this differently? | ||
| 148 | 08:12 | In order to [create] our explorations, | ||
| 149 | 08:14 | we partnered with Paramount Pictures | ||
| 150 | 08:16 | in order to explore immersive media in a Hollywood movie production. | ||
| 151 | 08:21 | Together with the director Randal Kleiser, | ||
| 152 | 08:23 | we reimagined the iconic movie of 1978, | ||
| 153 | 08:28 | "Grease." | ||
| 154 | 08:29 | Some of you know it, some of you don't. | ||
| 155 | 08:31 | A 40-year-old movie, amazing experience. | ||
| 156 | 08:35 | And our goal was really to look at how we can take this iconic action and dance | ||
| 157 | 08:41 | and bring it deeper into the experience, | ||
| 158 | 08:44 | bring it deeper into the audience. | ||
| 159 | 08:46 | Imagine that you can not just watch the movie | ||
| 160 | 08:49 | but get inside it and dance with the actors | ||
| 161 | 08:52 | and dance with the performance. | ||
| 162 | 08:54 | Now we're breaking, really, the traditional 2D mindset of thinking, | ||
| 163 | 09:00 | and bringing a much richer possibility of moviemaking | ||
| 164 | 09:03 | and content creation. | ||
| 165 | 09:05 | But why watch it on the screen? | ||
| 166 | 09:08 | Let's try to bring these actors here on the stage. | ||
| 167 | 09:17 | So they're not going to really come -- | ||
| 168 | 09:19 | I'm going to use an iPad. | ||
| 169 | 09:20 | (Laughter) | ||
| 170 | 09:21 | Sorry. | ||
| 171 | 09:22 | I'm going to use an iPad in order to bring in augmented reality. | ||
| 172 | 09:27 | Now, obviously, these devices have their own limitations | ||
| 173 | 09:33 | in terms of the data-computing process, | ||
| 174 | 09:35 | so we have to reduce the amount of resolution. | ||
| 175 | 09:39 | So what I'm doing now, I'm placing here a marker, | ||
| 176 | 09:42 | so I'll be able to position exactly where I want everyone to appear. | ||
| 177 | 09:54 | OK. | ||
| 178 | 09:55 | I think we have them here. | ||
| 179 | 09:57 | (Applause) | ||
| 180 | 10:04 | John Travolta, or -- | ||
| 181 | 10:05 | (Laughter) | ||
| 182 | 10:06 | a version of him. | ||
| 183 | 10:08 | Let's take a look. | ||
| 184 | 10:10 | (Video) Female: Hey. | ||
| 185 | 10:11 | Male: And that is how it's done. | ||
| 186 | 10:13 | Female: Your turn. | ||
| 187 | 10:14 | Male: Hey, guys! Check this out. | ||
| 188 | 10:16 | (Song: "You're the one that I want") | ||
| 189 | 10:19 | Danny: Sandy! | ||
| 190 | 10:20 | Sandy: Tell me about it, stud. | ||
| 191 | 10:24 | (Singing) I got chills. They're multiplying | ||
| 192 | 10:28 | And I'm losing control | ||
| 193 | 10:32 | 'Cause the power you're supplying | ||
| 194 | 10:37 | It's electrifying! | ||
| 195 | 10:39 | (Video ends) | ||
| 196 | 10:40 | (Applause and cheers) | ||
| 197 | 10:45 | Diego Prilusky: Thank you. | ||
| 198 | 10:47 | (Applause and cheers) | ||
| 199 | 10:49 | So as you can see, | ||
| 200 | 10:52 | we can watch and experience content in the traditional way | ||
| 201 | 10:55 | or in an immersive way. | ||
| 202 | 10:58 | Really, the possibilities are open. | ||
| 203 | 11:00 | We're not trying to change or replace movies. | ||
| 204 | 11:04 | We're enhancing them. | ||
| 205 | 11:06 | The technologies enable new possibilities to start thinking beyond the flat screen. | ||
| 206 | 11:12 | We're in immersive and really exciting times in filmmaking. | ||
| 207 | 11:16 | We're at the threshold of a new era. | ||
| 208 | 11:18 | We're opening the gates for new possibilities | ||
| 209 | 11:22 | of immersive storytelling, | ||
| 210 | 11:25 | and exploration and defining what immersive media filmmaking means. | ||
| 211 | 11:30 | We're really just at the beginning, | ||
| 212 | 11:33 | and we invite you all to join us. | ||
| 213 | 11:36 | Thank you. | ||
| 214 | 11:37 | (Applause) |