PipelineFX's Richard Lewis explains how working on a groundbreaking animated film helped establish Qube! a vital component in today's VFX industry
L'Atelier Animation's Ballerina makes use of Qube!
© L'Atelier Animation
PipelineFX's story stretches all the way back to beginnings of big-scale VFX. Founded by Richard Lewis in 1989 to create visuals for 3D animation, TV graphics and print, the Honolulu company found itself working on Square USA's 2001 movie Final Fantasy: The Spirits Within.
The film required a massive render farm and bespoke management software, the latter of which (Qube!) was purchased by PipelineFX. Lewis, and many of his employees, have been there ever since. Here we talk to him about the ins and outs of running render farms, and how they fit into the visual effects industry
as a whole.
ReelFX's boundary-pushing movie Book of Life needed Qube!'s power.
Qube! has gone on to become a vital component in the VFX industry, as well as the related fields of post-production, broadcast, architecture and automobile manufacturing. It's been used for a variety of recent productions, including ReelFX's Book of Life
, L'Atelier Animation's Ballerina
, Passion Pictures' Nike World Cup ad 'The Last Game', and a rebrand by MPC commercials. Even South Park
- which many people don't realize is rendered in 3D - makes use of the power of the Qube!
Qube is about to get a lot bigger, too. The Jiangye Newtown Technology Park - China's visual effects-centric equivalent of Silicon Valley - has just acquired 400 Qube licenses to ensure it can compete with western effects companies. But the question remains: why use a render farm when you could just create content locally?
"Rather than waiting a day to see a sequence rendered on a single machine, a small render farm of 10 machines can reduce that wait by 90 per cent”
"I call it the 'four Rs' of computer graphics – render, review, rethink, rework." explains Richard. "In order to see what we are working on, we have to render it. Once rendered, we can review it with our peers, managers or clients, then rethink what we are doing, rework the animation or image, and send it back for rendering again.
"Render farms reduce the time between opportunities to review and rethink graphics work. Rather than waiting a day to see a sequence rendered on a single machine, a small render farm of 10 machines can reduce that wait by 90%, potentially allowing for a review (or even multiple reviews) the same day."
Square's custom-built Linux render farm of over 1,500 servers
A day on the farm
But render farms shouldn't be thought of as autonomous one-size-fits-all solutions - rather like their agricultural namesakes they need input from the farmer to keep them running. This is where farm management software comes in.
"We built charting into our software from the beginning to provide the immediate insight into farm performance required to optimize a render pipeline”
"How do you know if you are really using all the CPU cores, memory and rendering licenses you've purchased?" asks Richard. "Are you just sending one job per host because it's easy even though your software may not be capable of using all of the resources in a given server or desktop? We built charting into our software from the beginning to provide the immediate insight into farm performance required to optimize a render pipeline."
Another assumption about render farms is that they're big, hot rooms packed with noisy server racks, but there's been a gradual shift away from this dedicated space. "I am a firm believer that architects, for example, do not need render farms - not in the traditional sense," says Richard. "The office spaces architects occupy don't lend themselves to the weight, heat, noise and specialized requirements of a data center. But on the other hand the typical architects' workstation is a very powerful machine. We have many architectural clients running 6 core desktops with 32GB of memory. The systems make great render nodes when not in use."
Passion Pictures' Qube!-rendered world cup ad, The Last Game.
© Passion Pictures
Architecting the farm
In fact, building a render farm may not need any new equipment at all. "An option for architects, for instance, is to save some of their last-generation desktops and just leave them under the designers' desks but with a switch box for display and input," says Richard. "This distributes the power and heat across the office and provides dedicated rendering machines. To manage a desktop used as a render node we created an application called QBLocker. It lets the desktop user control if and when his system is added to the render farm."
So everything's in place for you to roll your own render farm, but being able to actually use it is still an issue. University courses tend to focus either on the artistry of content creation or the minutiae of technical engineering, while ignoring the growing realities of render farm management.
"Organizations using render management don't need dedicated staff but they do need great support and help to get started”
Fortunately, PipelineFX has created a training course to get users up to speed on faster rendering. "To date we've run over 300 professionals through our Qube! Certified Administrator 2-day course," Richard says. "We also offer a 3-hour Qube! Artist Workshop to orient a new user to submitting, managing and troubleshooting their render jobs, as well as JumpStart services to get the system installed, configured and up and running in 4 hours or less. It's important to get started right. Organizations using render management don't need dedicated staff but they do need great support and help to get started."
Rocks, pebbles and sand
"The internet and the Cloud has made scaling up and down render farms to meet peaks and valleys in production a possibility”
The future for render farms, according to Richard, is granular. "The internet and the Cloud has made scaling up and down render farms to meet peaks and valleys in production a possibility," he says. "Think of it in terms of rocks, pebbles, and sand in a jar. To get it all in, you first you put in the rocks. These are your desktops and your on-premise dedicated render nodes. Next you pour in the pebbles. These are render servers in a hosted data center that you either lease or rent short-term. Lastly you pour in the sand to fill in the tiny gaps. These are the Cloud nodes you spin up and down when your rocks and pebbles are busy or you need to go very wide and get a render done quickly.”
"A few of our customers are on the bleeding edge and making good use of all 3, but this multi-level render pipeline is new to most. In the future we will see more integration between pipeline software to enable the use of these three tiers of rendering power”
Learn more about Qube!
Try Qube! for free
Learn Character Creation in Maya