Since Google and Facebook adopted the new technology of 360-degree videos, it is spreading throughout YouTube and Facebook, offering an unique perspective and experience. Peter Spence at Putty Studios applied this technology and produced the innovative 360-degree animated video: DO NOT Push The Red Button. The animated 360-degree Rube Goldberg Machine in 4K gives you a different experience from traditional animation where your viewpoint is set. We asked Peter Spence his insight from this creative challenge and are very pleased to be able to share his views.
Address: Vardeveien 1A, 1444 Drøbak, Norway
Founder: Peter Spence
According to Peter Spence, the production process followed a very similar structure to a normal film, which can be also seen in his making-of video. The entire movie is a MassFX physics simulation and the sequence is entirely created in 3ds Max, rendered using the Redshift 3D plugin, and put together in After Effects with the Skybox plugin from Mettle.
Overview of Production process
1. Procreate: Character concept design
2. Sculptris: 3D Sculpting
3. 3ds Max: Concept simulation test
4. Building scenes
5. Building the custom skeleton rig in Cat
6. Setting up the rig for joystick motion-capture
7. The character animation and the scene simulation
8. Redshift 3D: rendering
9. Audition: Sound effects and music
10. After Effects: Putting together the whole production
11. Mettle Skybox: Creating 360-degree video from Cube-map
Making of 360-degree animated video
2. The key difference in the production process from regular CGI animation
Spence explains the difference in the production process from regular CGI animation is “many of the usual short-cuts used in film-making were not possible”. For example, it is not common to create the entire set and animate a character if the area and the character are off-camera, but everything had to be created and animated for the entire length of the video with the audience controlling the view.
3. Technical challenge
Spence talks about one of the technical challenges he faced: “Creating a spherically-rendered animation also presented challenges in the incredible amount of frames needed”. His presented number of frames for rendering suggests how challenging this could be. As stated by Spence, “To be able to output a 4K animation, the scene was rendered as a 6-sided cubic map (Figure1 and 2). This meant that each of the 2,100 frames needed to be rendered from 6 angles, and the character animation, which was rendered as a separate pass, also needed to be rendered for each view it was visible in. It added up to about 19,000 renders”. He sees that this is a challenge for many content creators out there as there is a large amount of 360 videos that are either very low resolution or poorly rendered. Using Redshift3D (https://www.redshift3d.com/) solved these challenges. It does not only produce beautiful results, but is also fast (Figure3).
4. Challenge in a narrative development
“Thinking more like a theatre director than a film director”
360-degree video revealed some fundamental differences in the planning stages, which Spence has not had to consider on previous animation productions. When giving control of the camera view over to the audience, there was much more planning of how the narrative would work.
A new challenge with a narrative development liberated Spence from the constraints of the current trend seen in many video productions.
He says, “I found myself thinking more like a theatre director than a film director. These days we are getting used to fast-paced video editing and a lot of camera movements. I had to strip all that away to build a coherent narrative. It was actually very liberating! If I created too much action it was actually confusing rather than exciting. I could focus on the pacing of the action and use it to guide the audience”.
5. Outlook of a 360-degree video
As many of you know, real-time rendering is normally required for interactive content like a game. It is said that it must draw one frame at a time, rendering what each frame should look like based on where the user is looking, and the minimum acceptable rate to perceive smooth motion is 30 frames per second or much higher in case of virtual reality (VR) according to the article on Road to VR. Thus, it requires a high-spec computer to play and see real-time rendering naturally.
In case of traditional CGI animated films, a single frame is so complex that it could take hours to render, but we can watch a CGI animated film, it appears to run so smoothly. This is because CGI frames are pre-rendered, which mean that they are essentially saved as still photos and compiled into a video and played back at the desired frame rate. The 360-degree video made by Putty Studios is also pre-rendered so that it can be played and enjoyed on our tablets or smartphones. Yet, there could be improvement for the future. Spence thinks that there could be two creative solutions for the future of the 360-degree video where everything is still pre-rendered.
1. Stereoscopic playback
The first one could be some attempts at stereoscopic playback. Spence thinks that this is technically complicated to get working successfully. He explains that it needs more than just 2 cameras rendered side-by-side if it is going to be anything other than a panorama. Pre-rendered movies projection-mapped on to real-time rendered low-poly geometry is a possible solution.
2. Videos themselves becoming more interactive
Currently, only playing videos with a linear timeline is possible, but this could be a problem when the viewer might be looking in the wrong direction during an important plot detail for presenting the story. To tell far more complex and engaging stories, it would be ideal for the playback device to be able to detect the direction that the viewer is looking and start or replay the relevant video segment while keeping the freedom of the viewer to look around the space, according to Spence.
6. Potential of 360-degree video
Spence sees a huge potential with 360-degree videos. He says, “I really look forward to see what other creative people do with this new way of presenting their story. It’s about more than just sticking a camera in the middle of the action and I find it very exciting and fun. I’ve got a little notebook of new ideas that didn’t make it in to this video, so watch this 360 space!”
Lang, B (2014) DreamWorks Reveals Glimpse of 360 Degree ‘Super Cinema’ Rendering for VR Films. 20 November 2014. Road To VR.[Online] Available from:<http://www.roadtovr.com/dreamworks-reveals-video-360-degree-virtual-reality-super-cinema-format/>. [20 November 2015].