Technology

Editing as 3D Art

How Editing is a 3-Dimensional (3D) Art
by Kiran Ganti

3D means three-dimensions, i.e. depth (or length), width (or breadth), and height. The screens on which we see movies are 2D. They have width and height, no depth. The only time audience experience 3D is when they see movies wearing 3D glasses. Some cinematographers light and shoot the scenes in such a way that in their frame, not only space in width and height is explored but also one can see “depth”. This is their effort to break the 2D nature of the frame. But cinematography is only restricted to the visual.

Cinema is made up of twins, Audio and Visuals. Editing explores the other twin also. One of the key aspects of cinema is exploring space and time. And editing is the place where both can be changed, manipulated, juxtaposed either independently or collectively. By exploring space in time and by using audio and images, editors are creating a third dimension to their art and craft, which is beyond the normal definition of 3D. Let me break it down by each dimension. In an edit timeline, there is the Video layer (or tracks) and the Audio layer. One can have as many layers for video and audio. It is with these audio and video layers that editors create the third dimension.

Depth (or Length) – In the Video layer, all the visuals for the scene are arranged. All the aspects of 3D that the cinematographer has shot are there for the editor to consider while editing. An editor “looks into the frame” and this is depth. He/She reads the shot; looks at the actor’s performance, the space they inhabit and the way the space has been lit. An actor’s close up would reveal the inner thoughts or emotions of the character. Their dialogue could reveal a new layer. This depth or insight into the character is explored by the editor by deciding of the placement, duration of the visual and also through the audio.

Width– The timeline of the edit is the width. For a movie which is of 2 hours duration, the rough cut can be anywhere between 3:30 hours to 2:45 hours. During the final cut, the editor gets the duration down to the required time. This is achieved by removing scenes, inserting shots between scenes, which give new meaning to the scenes, changing the position of the scenes in the timeline etc. So a timeline is constantly changing. In all other aspects of post-production, the time is fixed. A constantly changing width, or time, is only found in editing.

Height– There are multiple tracks for the audio. Other than the dialogue, there will be sound effects, background score (either temporary score that the editor has put in or the one that the music director has created) etc. If the dialogues are placed in A1&2 tracks, sound effects in A3&4 and background music in A5&6, the editor is dealing with 3 different audio parts. These three parts are sometimes complementing and at other times colliding with one another. Given the complexity of the scene, the number of tracks in the audio layer can increase, their volume adjusted, the dialogues overlapped. Any editor sometimes use as many as 30 layers. So the editor is constantly dealing with height, especially through the audio tracks.

Editing is a 3D art. I think that while editing, Cinema is explored in all the 3-Dimensions in its purest form. We need to look at 3D in a new way.

Tech Corner 2Q21

 

Avid Media Composer version 2018.12.15 is the last version of the software before it went through a major interface overhaul. A recent select survey of ACE editors about the new interface showed that only 35% of the respondents (6 out of 17) used the newer interface.

The latest release of AMC is 2021.2, so after three years of release a substantial majority of these ACE editors are still working on software that is three years older. Why? Here are some of the responses to the question, “Have you or your show declined to use the most recent version of AMC?”

“The Avid rental facility made the decision for us because they have received so many complaints about the current AMC from other projects.”

“The habit of seeking stability in tried-and-true versions, and because we are Luddites.” “Yes, it is too unstable I’m told by tech people.” “Yes, most recent sometimes means new bugs.” “…we haven’t seriously considered using it yet … I’ll wait until the rental houses recommend it.” “The newer version has never been suggested, but if it was, I would decline and stick with the latest version of 2018 which is very stable and does the job.”

Well, the Luddites are winning.

The new interface does require some re-thinking about how your system is set up, and how you like to work. And when 2018 new features are added, there are more chances of software errors popping up. Newer versions of Media Composer don’t seem particularly unstable outside of the new features. But if something doesn’t work like it did yesterday, that is certainly frustrating. And some features are constantly under development (I’m looking at you Titler+).

Most of those editors using the newer interface appear to be participants in the Avid beta program or are active in technology forums. These are people, I think, like myself, who are more interested in new toys than unerring performance. And who want to participate in the program’s growth. Personally, I like to see new features as they are being developed to comment on their usefulness and how they may be improved. And what they break.

Why did Avid make such dramatic changes in the software, when most users weren’t clamoring for a new interface? My guess is money. Today software developers survive on subscriptions, which fund development, which justify subscriptions and combat piracy. It is the world we live in.

The biggest change from the 2018 interface is the introduction of the unified single window. Rather than being a series of separate windows (Project, Composer, Timeline), all windows are joined together. Resize one, you resize all. This is how Adobe’s Premiere Pro interface defaults.

Everything now sits within a screen-filling ‘host panel.’ Windows and Premiere Pro users are already familiar with the host panel. But for macOS users, it is strange to no longer see the computer desktop while in the application. The host panel can be resized/minimized. But its size isn’t remembered with a workspace. The host panel however no longer resizes to take over the full screen when you choose another workspace. That is a welcome improvement.

I’ve been using the latest interface since its release and am on the latest beta past 2021.2. Here are some observations. First the look is modernized. Here is a comparison of the opening screens.

Note the different icons for the bins. And the project no longer has tabs on top to reach other features. Settings, for example, now has to be called up from a pull-down menu. For me, this is a regression. It moves important controls deeper into the interface. But everything looks sharper. Then there are the new Bin features.

The project bins and folders can be accessed from any bin, now called a Bin Container. Initially the Project window was eliminated. But it was restored after user complaints. Each bin also has a fast search window (see the magnifying glass icon on the upper right). A fantastic addition for quick sound or music searches.

Here is a comparison of the palette for choosing clip colors.

This is maybe the most baffling change. There is a system to how the colors are arranged in the expanded palette, it has been explained to me … and I still don’t get it. The additional colors are great. But the arrangement is, um, perplexing.

Several more substantive changes have been made under the hood. The media engine has changed, now called the Universal Media Engine. Another thing I can’t say I understand. Partially this is because QuickTime is being phased out as a media engine. ProRes still exists, but QuickTime is going away. Confused? Yep, me too.

And some terminology has changed. DNxHD codecs were numbers but are now abbreviations. DNxHD 36 for example is now DNxHD LB. A lot less intuitive than before.

The other substantive changes have to do with modernization of computer operating systems. MacOS is dropping all support for 32-bit applications. Which means that the latest macOS release called Big Sur will no longer run the classic Avid Title Tool. Titler+, the replacement for the Avid Title Tool, has been an enormous pain point for many adopters of the latest Avid Media Composer. It was released in 2018. It has been a work in progress ever since. Avid decided rather than update the Title Tool, which may not have been possible, to invent a new title tool.

It has so many non-intuitive behaviors that are just confusing. I can’t use it and have told the engineers as much. And it is a system hog. Titler+ clips must be rendered or my system slows to a crawl. One can install the old Title tool by downloading and installing what’s called MediaComposerLegacyComponents. This won’t work, however, if your computer is using Big Sur, macOS 11.

The other pain point will be the loss of QuickTime Reference creation, also a victim of 64-bit platforms. The latest export method is as a single file, MXF OP1a. It is slow and creates enormous files. A new alternative output is MP4, where you can output to H.264 and adjust the target bit rate to make smaller files, similar to Adobe’s Media Encoder. Either way ties up Media Composer longer than before with QT Reference.

There are some really cool features in the latest Media Composer. First is ‘Find and Replace,’ a bin level search-and replace feature.

It is important to note that you can work more closely to the previous interface with some tweaking. For example, rather than have the unified window interface, I do what’s called ‘Float All Panels.’ That makes every window a free agent, changeable on its own, more like the 2018 version. Cheating, I know. I do the same float-all-panels in Adobe Premiere Pro as well.

There are a huge number of new features and improvements in the latest Avid Media Composer. There are some problems not yet solved (I’m looking at you, Titler+! Again!). But I don’t want to go back to an earlier version. What’s the fun of having everything work like you expect?

So, does that falling tree make a sound? I don’t know. It’s a stupid question. I just want to try the next Avid features.

Cuts We Love – ET

An instant classic, E.T. the Extra-Terrestrial’s depiction of a friendship between an alien stranded on earth and a young boy named Elliot continues to enchant audiences.

“We knew there was one big problem when making the movie and that was the character of E.T.,” explains Carol Littleton, ACE, who earned an Oscar® nomination for the film. “It’s a puppet made of different parts and we knew that if the audience could accept this little creature as real then half of the battle of telling the story would be won.

“That’s why the ‘getting to know you’ scene is so important,” she says. “Previously, Elliot left a trail of Reese’s Pieces and he is sitting waiting for the creature to appear. There’s a sense of mystery and perhaps fear over what he will discover.

“The creature appears and each time it gets closer, Elliot’s curiosity grows. He lures the creature upstairs to his bedroom by continuing the trail of sweets. Once inside the bedroom they have their first conversation.”

Littleton points out this scene has virtually no dialogue and likens it to a silent movie. “They create a sign language almost by mistake,” she relates. “Elliot rubs his nose and the creature mimics him. Finally, Elliot releases a laugh of glee and surprise. You begin to realize E.T. has a certain power over Elliot. He has an almost telepathic way of communicating. In a way the whole movie is about gaining trust and having a true friendship. It’s almost a love affair between the little boy and E.T.

E.T. itself was a combination of puppetry, a mime artistry and a face articulated by a dozen engineers just off-set. “Steven would call out to them exactly what he wanted them to do: ‘Lift the neck’ or ‘get the eyes to blink.’ There were all sorts of issues with the movement. They shot a roll of film for each single shot. That meant that each time you needed to make a cut it had to be a sustained piece so you could feel it was a character and yet we didn’t always have the pieces. We had to comb through the material to find what we needed.

“One of the reasons the child actors and the audience could relate to E.T. as a real creature was because it existed in 3D space. That could not have been achieved if done by greenscreen.”

Littleton confirms that the film is a personal one for Spielberg, who as a child felt alienated by his parents’ divorce. “He told us he had a make-believe buddy who was a person who consoled him at this moment. So, it’s with a great deal of empathy that he created this movie. It’s a very loving portrait of childhood and the resilience of children to accept the best of any situation.”

The end of this scene depicts adults searching for E.T. with torches. It is the adults (with exception of Elliot’s mother) who are alien. “I recall Steven and DP Allen Daviau discussing lowering the camera to a child’s point of view,” Littleton remembers. “When I first saw the scene in dailies I realized he was capturing their world view where adults exert a control or exhibit indifference to small people. That feeling is in the film from the first frame.

“The power of this scene comes from the audience being shown the character bit by bit, over preceding scenes, culminating in the first face-to-face meeting in Elliot’s bedroom. We feel the trust, the magic, the trepidation which is then resolved when E.T. puts Elliot to sleep. It’s the key that unlocks the relationship between them.