As quickly as it began, FMX is over for another year. Perhaps more than at previous conferences, speakers were especially confident in discussing the current state of the animation, vfx, and digital content creation industries – including the state of virtual reality. Just as importantly, there was much discussion of the future, especially with the clear arrival of vr and interactive and immersive storytelling. Just what should filmmakers do with this new medium? Luckily, pretty much the world’s top vr filmmakers were there to weigh in.
VR capture tools, techniques for producing content, and headsets for displaying that content seem to be innovated upon almost weekly. We still don’t know quite yet what many of the virtual reality and augmented reality tech companies have in store for us (think: Magic Leap). But there are, of course, already many ways to experience vr and several studios have dived into vr content creation. The question is, how do you make that content compelling?
At FMX, an expert panel of vr and immersive filmmakers hosted by Google Spotlight Stories’ Kim Adams was on hand to discuss their relatively short but intense experiences in the field. One of those panelists was Jan Pinkava of Pixar fame (Geri’s Game, Ratatouille). Now creative director at Spotlight Stories (part of Google ATAP), Pinkava created an early interactive film for mobile devices in 2013 with Windy Day, while ATAP was still part of Motorola.
Pinkava’s film grew out of a mission at ATAP to create something emotional for the smartphone; the group recruited Pinkava, who brought aboard other Pixar veterans like Doug Sweetland and Mark Oftedal. “We wanted to do something with this power, something emotional,” Pinkava told a packed FMX audience. “Coming from Pixar, those films they make are emotional – so we thought, let’s make movies. We sat in a room and said, why don’t we make a movie and give the camera to the audience?”
After overseeing other films crafted in Google’s ongoing Spotlight series, Pinkava has formed his own views about what works in immersive and interactive films, whether they be 360 degree experiences or vr pieces that can be watched with either simple goggles or hard-wired headsets. Pinkava constantly asks, when considering vr proposals, ‘Is this the best way to watch this? Is every way I’m going to watch this a good way? As a director, how can you help that experience for the audience be a good one?’
Two other filmmakers on the panel with experience in Google Spotlight Stories were Tim Ruffle and Jason Fletcher-Bartholomew from Aardman Animations. They collaborated on the 360-degree Special Delivery. Writing Special Delivery required a re-think of the traditional short film script. “In our heads we initially wrote a linear script,” said Fletcher-Bartholomew. “We would write in background gags, but we should have concentrated on the main chase. So getting out of that linear mindset was actually quite hard for a number of weeks. Even our storyboard was very linear, but in end we cut it up on different walls. If anyone looked at it they would have thought we were mad. We went back to cardboard cutouts and moving back to theater-like sets.”
Another challenge faced on this Aardman short, the duo shared, was how to lead the viewer around in the story and not have them get lost in the 360 world. “I found it more like choreographing a play than writing a play,” said Ruffle. “We made sure there were elements of our sets that made particular characters visible. It’s a bit like video game language. People just seem to understand to look in certain directions because something is happening over ‘there.’”
This requirement to consider new ways of choreographing the action in an immersive film was a view also shared by Nexus’ co-founder and executive creative director Chris O’Reilly. Nexus is making a new Google Spotlight Story called Rain or Shine (some clips were shown here at FMX). “Directors come in with filmmaking tools, but they all need re-thinking in vr,” said O’Reilly. “We were thinking about how to control spaces. We talked to games companies. We even talked to architects who have experience in cajoling people through airports.”
Aardman and Nexus’ experience also highlights one important aspect of vr right now: it’s new. Which means everyone is experimenting. Jacquie Barnbrook, a producer with The VR Company, which is in the process of making the much-anticipated The Martian VR experience, says part of the joy and frustration of vr filmmaking right now is just getting through it. “The director of The Martian VR, Robert Stromberg, says we’re building the plane as we’re flying it – we may die,” Barnbrook related. “Shooting in vr is the most weird and awkward experience you can imagine. Things like our cameras are not yet able to sustain shooting for more than six minutes. We were putting cups of ice on the cameras to cool them down!”
For Mirada Studios’ Andy Cochrane, who has worked on everything from a Google Shop VR Tour to The Strain VR, the result of this new wave of interactive filmmaking actually provides the opportunity to serve in multiple roles. “I’m a technical supervisor, a director, a visual effects supervisor. We settled on ‘digital and interactive director’ because it seemed to be descriptive of something.”
And Cochrane suggests that this new vr/interactive storytelling world – which is heavily concentrated right now on the west coast of the U.S – just happens to come at a time when the visual effects industry on that side of the country has been partially decimated by subsidies and other benefits being offered elsewhere. That, says Cochrane, represents an opportunity to visual effects artists who once worked in Los Angeles and are highly suited to technical and artistic aspects of virtual reality. Indeed, effects artists have for years been dealing with stitching plates, making 3D scenes and characters, and solving complex lighting issues, in helping to tell compelling stories.
The future does look bright in vr and immersive and interactive entertainment, even if we don’t quite know just what it ‘is’ yet. Certainly, vr content providers are doing everything they can to, as Jan Pinkava observes, “put tools out there for their collaborators — filmmakers — to make the things they know how to do. The whole purpose is then to make all this available to you, the audience.”
The post FMX Report #2: What The VR Future Holds appeared first on Cartoon Brew.
Google's interactive film division has announced four new animation projects, and introduced a developer's kit that allows anyone to produce interactive animation.
Google's ATAP group has released a mini-featurette about the making of Glen Keane's short "Duet," which was released earlier this week in its intended interactive version.
"Duet," the directorial debut of animation legend Glen Keane, has already appeared online and is a strong contender in this year's Oscar field, but it hasn't been available in its interactive format until today.
Today we look at the work of Jon Klassen, Cartoon Brew's Artist of the Day!
Watch Glen Keane's new short "Duet" that he debuted this morning at the Google I/O developer conference.
Last week, Google’s Advanced Technology and Projects group released the second animated short, Buggy Night, in its Spotlight Stories, a series of interactive mobile-specific animated films available on Moto X phones. Like the first film in the series, Windy Day, which debuted last October, the new short relies on spatial awareness and the sensory inputs of a mobile device to create a distinctive storytelling experience. Readers would be warranted in expressing skepticism at the words ‘interactive’ and ‘animation’ being used in the same sentence. The concept has been touted often, yet rarely executed in a manner that suggests it could become a viable alternative to linear entertainment experiences. These shorts have finally proven, to me at least, that there is a promising future ahead for interactive animation and immersive worlds where multiple stories can unfold at the individual viewer’s pace, with no two viewing experiences alike. While it still requires some imagination to see where this could all go, and how it might eventually figure into our emerging augmented reality environment and mixed digital-physical world, the idea no longer seems as far-fetched and impractical as it once did. Before we imagine the possibilities, let’s look at the pathbreaking animated shorts that exist before us today. The stories of both Windy Day and Buggy Night are simple but effective ideas designed to explore the interactive concept: in one, a mouse loses his hat on a windy day, and in the other, a group of bugs attempt to hide from a hungry frog. Since most readers of this site will not have a Moto X readily available to experience these shorts, simply imagine that you are standing in the middle of an animated scene. The action takes place all around you in a 360 degree space. Anywhere you turn your phone—left, right, up or down—could potentially reveal something happening. The film’s running time depends entirely on how often you, the viewer, chooses to move your camera—the more you move it, the longer it takes to finish the story. This video gives a sense of the physicality of the viewing experience: The interactivity in these shorts not only feels natural, but adds immensely to the viewing experience. This success can partly be attributed to the amount of interactivity allowed to the audience. While control of the camera is ceded to the viewer, the overall narrative remains in the hands of filmmakers. It’s a careful balance between interactivity and linear storytelling that recognizes tried-and-true narrative structures can’t be reinvented—the only thing that changes is how we experience them. Over the years, we have moved from oral tradition to literary form, and finally, visual delivery systems like film and video. While each new mode of expression presents a distinct set of narrative possibilities, the underlying story form must remain intact, an idea heretofore not clearly acknowledged in interactive attempts. Google’s entry into interactive storytelling and immersive animation began almost accidentally with their purchase of Motorola Mobility in 2012. Eager to explore the untapped potential of phones as an experiential device, they launched an open-ended research group called Advanced Technology and Projects—ATAP for short—to foster innovation and develop next-generation concepts. Spotlight Stories is one of the ideas that has emerged out of ATAP, alongside complementary technologies like Project Tango. (Google sold Motorola a month-and-a-half ago, but as an acknowledgement of ATAP’s importance, the group was not part of the sale and remains a part of Google.) Google/Motorola also learned something that it took the computer animation industry decades to fully understand: if the creative potential of a technology is to be fully unleashed, creative people need to “challenge the technology,” as John Lasseter is fond of saying. Google hired incredibly qualified people to push the limits of interactive storytelling. The first two films have been directed by Jan Pinkava (creator and co-director of Ratataouille) and veteran animator Mark Oftedal (who animated on Toy Story and A Bug’s Life among other films). Another Pixar vet, Doug Sweetland (Presto), supervised the animation, and notable children’s book author/illustrator Jon Klassen (This is Not My Hat) styled the look of the shorts. Continuing this trend of working with A-list talents, another upcoming Spotlight short will be directed by Glen Keane. The Spotlight Stories aren’t just exploring new ways of telling stories interactively, they are also pushing forward the technological development of mobile devices. Google touts in their promotion of Spotlight Stories that mobile graphics processors now rival the capabilities of video game consoles such as the PS3 and Xbox 360, a fact that will come as a surprise to the average smartphone user who is accustomed to the primitive worlds of Candy Crush and Angry Birds. This dormant computational power is finally being used, and in turn, developed further to meet the demands of the Spotlight Stories. Among the numerous technological highlights, the shorts contain the first-ever real-time subdivision surfaces on a mobile device using Pixar’s open graphics standard, OpenSubdiv. Not so coincidentally, Windy Day’s director Jan Pinkava also directed the Oscar-winning Geri’s Game (1997), which was the first Pixar production to use subdivision. Stay tuned to Cartoon Brew, where next week we will dig more deeply into the creative and technological challenges of interactive storytelling in an interview with Jan Pinkava. (Disclosure: Google provided Cartoon Brew with Moto X phones to view the Spotlight Stories. The phones have been used for the sole purpose of viewing the shorts.)