We recently had an opportunity to interview Jay Grenier, Director, Technical Operations at Faceware Technologies, Inc. Faceware is a company that specializes in facial motion capture for video games as well as movies. They have have brought faces to life in AAA video game franchises like Crysis, Grand Theft Auto and Assassin’s Creed. We asked Jay about the future of the technology, the games that are utilizing Faceware and a ton of other interesting things.
Sit back and enjoy the full interview below.
Rashid Sayed: For people who are not aware of Faceware, can you please explain what this technology is all about and what you guys do?
Jay Grenier: Faceware Technologies offers hardware and software for high-quality, high-speed facial motion capture. Our technology, licensed from Image Metrics, is a marker-less, video-based system that tracks and solves facial data from any video camera to any CG character. We also rent and sell a Head-Mounted Camera which is an optional, optimized capture method you can use with our tech. Faceware is an award-winning application that has been utilized on AAA games like Crysis 2, Red Dead Redemption, Assassin’s Creed and Halo.
Rashid Sayed: How does this technology work?
Jay Grenier: Our system is completely marker-less and works from any type of video file, whether it’s from a smart phone camera or from one of our Head-Mounted Cameras. Analyzer™ accurately detects the person’s face in the video image and can track the movement of every pixel. As it learns more about the shapes and movement of the actor’s face, the work gets faster and easier. When it’s finished, a file is saved that contains all of the facial movement data for a particular video. Retargeter™, a plug-in to Maya, 3DS Max, MotionBuilder and Softimage, reads the data from Analyzer™ and with the help of an animator, it transfers that data onto any CG character.
Rashid Sayed: How different is motion capture in video game compared to a Hollywood movie?
Jay Grenier: The mocap process is very similar in both games and films. There are game developers using the exact same hardware and software that’s used in movies. The difference that the larger film budgets can make is that they have more time to spend capturing, more time to polish the data, and pre-render characters with limitless quality. Games budgets are smaller, so animators can’t spend as much time capturing and polishing facial animation. Plus, a lot of facial animation in games is rendered in real-time, which is highly dependent on processing resources. That said, hardware and games are catching up quickly. Soon enough it’ll be the same process all around.
Rashid Sayed: I was going through your projects and interestingly, you have closely worked with Rockstar on a number of AAA games like Red Dead Redemption and Grand Theft Auto 4. But in case of L.A. Noire, Rockstar implemented their own capture tech (called Mo-Cap). Was this due to the technical limitations of Faceware?
Jay Grenier: No. When Rockstar began working with Team Bondi on L.A. Noire, the game was already quite a ways into production and was already using the MotionScan technology for the facial animation. It didn’t make sense for them to start over at that point.
Rashid Sayed: Well, I have to ask this question. Rockstar are working on Grand Theft Auto 5 and undoubtedly it is one of major games of 2013. Are you guys working on it? If yes, what level of animations are we going to see in the game?
Jay Grenier: My lips are sealed about project details unfortunately. We’re just as excited about GTAV as everyone else!
Rashid Sayed: Does implementation of Faceware vary in different games? Or is it as simple as wearing the Head Mounted Cameras and letting the Analyzer do its work?
Jay Grenier: Our software workflow is pretty straight forward and the animation pipelines tend to be similar on every project. The video capture process though is different all the time. Some studios are using full performance capture, some shoot their talent in a voice-over (VO) booth, and some will use footage directly from a live-action plate. Even archival footage from old films and other videos is used in some cases. The capture process is usually the biggest variable on every project. Once studios have good video data, the tracking and solving process is identical in whatever type of project you’re working on.
Rashid Sayed: You have worked on a lot of big franchises like Crysis, Metal Gear Solid and Halo. What was the one major learning factor from these projects that led you guys to do more improvements and enhancements to Faceware?
Jay Grenier:The one resounding thing we’ve learned from doing facial animation on big titles like Crysis, GTA, Assassin’s Creed, Red Dead Redemption, and others is that making the game doesn’t ever go exactly the way it’s originally planned. Things happen that can throw everyone for a loop. Stories can change, ideas can become new ideas, and the project evolves. To be successful the developer needs to be able to adapt, deal with difficult situations, and keep the ship moving forward. Our software has been designed and battle-tested to meet these challenges. When a big script change comes around in the middle of crunch-time and you’ve got a huge amount of animation to do with a team of exhausted artists, Faceware has you covered.
Rashid Sayed: You are currently working with Bungie (probably Destiny (reference here)). Two years back Bungie claimed that by using Faceware they will have better animation quality. Since then a lot of time has passed and it is only obvious Faceware has also grown. How are the animations in Bungie’s new IP looking like now?
Working with Bungie has been a great experience for our team and our development has been influenced very positively by their feedback. It’s been great to see how much success they’ve had using Faceware and we’re looking forward to when they can share more about their new title.
Rashid Sayed: Are you guys working on any other new video game products? Any unannounced games Faceware is using?
I can’t reveal any details about upcoming titles we’re working on but I can say that we’ve had a very busy year and 2013 is looking like it’s going to be even bigger. You’ll be seeing a lot of great Faceware animation this year on some very well known and anticipated titles.
Rashid Sayed: Doing motion capture footage for a PC version of a game can be less daunting compared to consoles (PS3, Xbox 360). How do you guys work with game developers and overcome console limitations?
Jay Grenier: Console and hardware limitations have always been a huge factor for us due to the fact that historically, facial animation has been pretty low on the totem pole when it comes to divvying out game resources. Due to this, we’ve always had to work to produce the best quality possible within some pretty harsh constraints. Faceware has been built to work well with any type of character for this reason. Now that the consoles are getting better, it’s enabled higher quality assets to be built and more resources to be allocated to facial animation. All that said, we still want and try to pull the very best out of whatever limitations we’re up against.
Rashid Sayed: What’s next for Faceware? Do you guys think there is scope for this technology to improve and take it to another level?
Jay Grenier: We’ve got some huge things coming in 2013 that we’re really excited about. Working closely with Image Metrics gives us first dibs on the new tech they are developing and we’re working on a full integration of their next-gen facial tracking and animation tech into Faceware. There’s so much more room to grow and so much more to do. The sky is the limit for this technology.
Rashid Sayed: And this obviously leads to another question: How far can you go with this technology? Despite technological advancements, there has to be a threshold for motion capture. Thoughts?
Jay Grenier: Our mission isn’t just to have the highest quality capture, but to also have a solution that empowers artists and speaks to real-world production. Quality is king, but you also need a solution that’s fast, affordable, and gives you creative control all the way through the process. It’s reasonable to say that someday we’ll hit a quality threshold and motion capture will be as good as it’s going to get. When that day comes, we’ll be busy trying to make the very best quality easier, more accessible, and less expensive to produce.
Rashid Sayed: Is there anything else you want to tell our readers about Faceware?
Jay Grenier: Stay tuned for some huge advancements in facial capture this year, some big things are coming! If you’re looking to try out Faceware you can download both Retargeter and Analyzer for free on our website. We also provide free video samples and free facial rigs that you can use to evaluate the process.
Rashid Sayed: Thanks a ton for your time.
Jay Grenier: Thanks for having us!
Share Your Thoughts Below (Always follow our comments policy!)