Those still skeptical about Sledgehammer Games’ commitment to providing better visuals with Call of Duty: Advanced Warfare can take heart that the game will be using a new facial animation system for its expressions. That was briefly gleaned during the monologue Kevin Spacey’s character Jonathan Irons delivered in the reveal trailer and it’s only the start according to EDGE in a story that analyzed the game’s new facial animation tech.

“There is no question that this particular Call Of Duty throws up an unusually long list of things to talk about. First, there are its origins at Sledgehammer Games, a studio that contains many of the people who created the first – and best – Dead Space at Visceral Games, and who are convinced that one of the things they can deliver this time around is a coherent, meaningful storyline.

“Then there’s the game’s new facial animation system, powered by technology and techniques that will also be used in the production of James Cameron’s Avatar 2. There is Advanced Warfare’s audio design, whose aim is to replicate not only the sound of letting loose with heavy firearms, but the torso-invading feel of it too.

“And there is Sledgehammer’s broad goal for the game’s visual appearance – photorealism – something it achieves in places thanks to a combination of its new rendering technology, an abundance of data sourced from the real world, and the application of its artists’ expert hands.”

If that’s a true advancement of the franchise, then what is?

Call of Duty: Advanced Warfare will be out on November 4th for Xbox One, PS4, Xbox 360, PS3 and PC.

GamingBolt spoke to OC3’s chief executive officer Doug Perkowski about the benefits of FaceFX, how it came into being, facial animation on consoles vs. PC and much more.

Rashid Sayed: To begin with can you please tell us a bit about FaceFX and the initial inspiration with which the company was formed?

Doug Perkowski: FaceFX is a facial animation solution that creates a realistic talking character from an audio file. It has been used in hundreds of video games to help create some of the past decade’s most memorable video game characters and experiences.

10 years ago, games were getting more realistic with faster processors and more memory, but Moore’s law doesn’t apply to characters. You can model a high polygon face, but without realistic movement, it is lifeless.  The ability to generate convincing facial animation performances without breaking the budget became a real bottleneck for video games. It was (and still is) one of the hardest problems in 3D because humans are so good at reading into the details of the human face.

Like any creative medium, games are ultimately judged by the characters they create, the stories they tell, and the emotions they inspire. You need realistic facial animation for all of those things, so facial animation is an important problem, not just a hard one.  We founded OC3 Entertainment to help solve it.

Rashid Sayed: FaceFX have announced a new Facial Animation Runtime for the Xbox One and PS4. What kind of changes have you made to the SDK to accommodate the new consoles?

Doug Perkowski: We were able to add multi-threading support, reduce memory fragmentation, and boost the overall performance by a wide margin.  Our file loading code was completely re-written to achieve an order of magnitude increase in performance.  In general, everything was redesigned from the ground up to be more efficient and use less system resources.

"The bar for facial animation has definitely gotten higher on the new consoles. Characters are more realistic than ever and they require higher fidelity facial animation to match. So the main thing developers want is quality animation without a lot of cleanup."

Rashid Sayed: Killzone Shadow Fall is one of the early next gen benchmarks in terms of facial animation. What can you tell us about your collaboration with Guerilla Games for creating realistic facial animation?

Doug Perkowski: We worked closely with Guerrilla Games.  They were one of the first studios we visited after introducing FaceFX Studio back in 2004, and we saw their new offices and met with some of their team more recently in 2012.  They have put together an incredible game and an amazing universe.  We are thrilled that FaceFX technology contributed to it.

Rashid Sayed: Some developers are now creating their own proprietary sound technology. A good example is the Madder Technology used in Killzone Shadow Fall. How easy or difficult it is to integrate FaceFX with such custom engines/tech?

Doug Perkowski: There is a pretty loose coupling between facial animation and sound engines.  You just need to keep them in synch.  So as long as the sound engine can tell you how far along the sound is, that’s really all you need.

Rashid Sayed: Talking about lip synching, does the new tool set makes it easier for localization? [Example: a person speaking in Japanese will have a different lip syncing compared to the same person speaking in English]

Doug Perkowski: FaceFX has always handled localization very well.  Even a modest amount of dialog can require a very robust facial animation when you multiply it by all the languages that need to be supported.  FaceFX supports English, French, German, Italian, Spanish, Korean, Japanese, Mandarin Chinese, and Czech.  We are working on adding Russian, European Portuguese, and Brazilian Portuguese.  Other languages can be used if you use transliterated text or without providing any transcription of the audio.

Rashid Sayed: Given that both new consoles have a pretty high memory footprint are developers demanding more features/tools from you guys? If yes what is the most wanted next gen feature that the devs are demanding from FaceFX?

Doug Perkowski: The bar for facial animation has definitely gotten higher on the new consoles.  Characters are more realistic than ever and they require higher fidelity facial animation to match.  So the main thing developers want is quality animation without a lot of cleanup.

"FaceFX is very efficient with memory. Our animation data is highly compressed and is only expanded fully when a particular animation is playing. As a result, the underlying memory architecture of the platform does not impose any constraints for us."

 Rashid Sayed: Can you please let me know what do you mean by a ‘lot of clean up’?

Doug Perkowski: With any high volume facial animation pipeline, you need to generate lots of animations in a limited amount of time. You can use audio to generate the animation (like FaceFX does), or you can try to capture an actor’s face with mocap or video. Once you have the audio-generated data or the raw mocap or video-generated data, clean-up would entail any steps require to process the data to get it into a form suitable for driving a character in your game. As a general rule, FaceFX does not require a lot of clean-up, because the data is generated as opposed to being captured. But artists may still want to add expressions or other emotions that are difficult to generate from the audio file, so even audio-based solutions can have “clean-up”.

Rashid Sayed: Both the Xbox One and PS4 have similar architecture but the former lacks a unified memory. Many developers are apparently facing issues due to eSRAM resulting into lower resolution in certain games. As someone who has extensive hands on the Xbox One and the internal tid-bits what is your take on this issue and does it affect FaceFX tools development in anyway?

Doug Perkowski: FaceFX is very efficient with memory.  Our animation data is highly compressed and is only expanded fully when a particular animation is playing.  As a result, the underlying memory architecture of the platform does not impose any constraints for us.

Rashid Sayed: When compared to a PC with a high end GPU/CPU combination what kind of challenges does the development team behind FaceFX in developing the right tools specific to next gen consoles given that their respective CPUs haven’t kept pace?

Doug Perkowski: A high-end PC with the latest GPU/CPU gives you the most graphics horsepower, but ultimately I think iPads and tablets are a greater competitive threat/opportunity for consoles.  High end PC games will always be a niche market that appeals to the graphics enthusiast,  but the average consumer will choosing between a console game and new gaming app on their tablet, or perhaps some hybrid.

 Rashid Sayed: Actually, I was asking the question in terms of technology you guys are building on [in terms of clock speed of CPUS]. Since high end PCs have a higher clock rate, and next gen consoles still lagging behind a high end CPU, does it hold your technology back in any way? You guys could have probably done more if these next gen machines would have boasted a higher clock rate.

Doug Perkowski: Are you talking about the speed of the CPU or the GPU? It doesn’t matter because facial animation technology is not constrained by either really. But with a faster graphics card, you could potentially have higher polygon characters, but even in that case, you would probably use the same skeleton rig. And FaceFX calculates transforms for the skeleton rig, and the engine will then use the graphics card to drive the polygons with the bones, so FaceFX’s calculations would be the same. At the end of the day, you just don’t get better facial animation with faster CPUs. You can get better physics, particles, explosions, and simulations, but facial animation can only get better with more animator hours or better technology. That’s what makes it such a hard problem.

OK. And just an additional clarification, there are many facial animation methods that are definitely limited by the CPU and GPU like skin solvers and light simulations for the skin. FaceFX could even drive those systems, but most games (regardless if they are targeting high-end PC’s or consoles) will use more traditional animation techniques that are not bound by the CPU or GPU.

"The trends in the casual game development market are the ones to watch. There is a democratization that is happening as game development tools and services get more powerful and less expensive. The cost of developing a casual 3D application is dropping rapidly and more and more developers are getting involved."

Rashid Sayed: The PS4 and Xbox One consoles are here to stay. There are no doubts about that. We are probably going to see a long road map for both of these consoles. Where do you see FaceFX in the new few years as the next gen console cycle unwinds?

Doug Perkowski: The trends in the casual game development market are the ones to watch.  There is a democratization that is happening as game development tools and services get more powerful and less expensive.  The cost of developing a casual 3D application is dropping rapidly and more and more developers are getting involved.  It’s similar what happened to the World Wide Web 15 years ago.  To make a web page, you used to have to hire a “web developer”.  I think you can argue that we are all web developers today.  There are still lots of professional web developers, and there will always be professional game developers,  but I’m looking forward to seeing game development technology become more ubiquitous.  We want to make sure that FaceFX enables the next wave of 3D applications and games.

Rashid Sayed: The next big thing for the new consoles is that developers will offload some of the tasks of the CPU to the GPU. Is the team at FaceFX already doing that or is it something that the game developers have to look out for themselves?

Doug Perkowski: There is a limit to how many characters you can fit on screen at once, so unlike a particle simulation or physics calculation, there is only so much you can do with the massively parallel computational power of the GPU.  Complex skin solvers could use that horsepower of course, and FaceFX can drive skin solvers just as easily as it drives bones or normal maps.

Rashid Sayed: One last question before we wrap up.  Can you tell us what next gen games you are working on and the challenges you have faced with each of them so far?

Doug Perkowski: We can’t really talk about the specifics, but I think it is safe to say that the new runtime does an excellent job addressing the issues we have identified with next-gen facial animation pipelines.

We’ve heard about the power of the PlayStation 4 and Xbox One in creating large, open worlds and modelling realistic effects. Though the PS4 has had a leg-up over the Xbox One in terms of games that can run at 1080p/60 FPS, the power of both next gen consoles dwarfs that of current gen easily.

However, how do both consoles fare when it comes to realistically modelling facial animations and expressions, especially compared to PCs? Considering that increased realism is a hallmark of the next-generation, GamingBolt got in touch with OC3 Entertainment’s Doug Perkowski, whose team is behind FaceFX. The company’s middleware solution has been used in a number of blockbuster titles including Thief, Killzone: Shadow Fall, Battlefield 4, Grand Theft Auto V and many more.

When compared the power of next gen consoles to a high end PC, who’s hardware is constantly evolving, what kind of challenges does the team face in developing the right tools for the PS4 and Xbox One? Perkowski stated that, “A high-end PC with the latest GPU/CPU gives you the most graphics horsepower, but ultimately I think iPads and tablets are a greater competitive threat/opportunity for consoles. High end PC games will always be a niche market that appeals to the graphics enthusiast, but the average consumer will choosing between a console game and new gaming app on their tablet, or perhaps some hybrid.

But what about the overall clock speed? With CPUs logging higher clock speeds over next gen consoles, does that hold the latter back in any way with regards to FaceFX? After all, there’s no denying that so much more could have been accomplished with higher clock speeds on consoles.

Perkowski responded, “Are you talking about the speed of the CPU or the GPU? It doesn’t matter because facial animation technology is not constrained by either really. But with a faster graphics card, you could potentially have higher polygon characters, but even in that case, you would probably use the same skeleton rig. And FaceFX calculates transforms for the skeleton rig, and the engine will then use the graphics card to drive the polygons with the bones, so FaceFX’s calculations would be the same.

“At the end of the day, you just don’t get better facial animation with faster CPUs. You can get better physics, particles, explosions, and simulations, but facial animation can only get better with more animator hours or better technology. That’s what makes it such a hard problem.

“And just an additional clarification, there are many facial animation methods that are definitely limited by the CPU and GPU like skin solvers and light simulations for the skin. FaceFX could even drive those systems, but most games (regardless if they are targeting high-end PC’s or consoles) will use more traditional animation techniques that are not bound by the CPU or GPU.”

So regardless of their differences with a high-end PC, much less their differences with each other, the PS4 and Xbox One are both fairly well off in terms of facial animation. It will be interesting to see what happens when games start relying on the above mentioned skin solvers and light simulations in the distant future, which will require more CPU/GPU power but for now, all is hunky dory.

What are your thoughts? Let us know in the comments below.

One of the more intriguing aspects of next gen consoles lays not just in their overwhelming power but in their ability to channel that power for a more nuanced visual look. In other words, it explains why the facial animationfor games like Ryse: Son of Rome and Killzone: Shadow Fall are already superior to current-gen titles.

GamingBolt recently had a chance to speak to OC3 Entertainment’s Doug Perkowski on facial animation suite FaceFX, which will incorporate a new SDK especially for the PS4 and Xbox One thus taking advantage of the power of next-gen consoles.

On being asked about the new Facial Animation Runtime for next gen consoles, he responded that, “We were able to add multi-threading support, reduce memory fragmentation, and boost the overall performance by a wide margin. Our file loading code was completely re-written to achieve an order of magnitude increase in performance. In general, everything was redesigned from the ground up to be more efficient and use less system resources.

We also asked if it made for easier lip syncing when it comes to localizing specific languages (for example: the lip syncing of an English speaker will differ from that of a Japanese speaker). “FaceFX has always handled localization very well. Even a modest amount of dialog can require a very robust facial animation when you multiply it by all the languages that need to be supported. FaceFX supports English, French, German, Italian, Spanish, Korean, Japanese, Mandarin Chinese, and Czech.

“We are working on adding Russian, European Portuguese, and Brazilian Portuguese. Other languages can be used if you use transliterated text or without providing any transcription of the audio.”

One way or another, facial animation for the PS4 and Xbox One will only get better as tools like FaceFX add support. What are your thoughts on the future of facial animation in next-gen games? Let us know below.

We recently had a chance to sit down with Jake Fenske, Line Producer at CounterPunch Studios, who specializes in facial animation and has used commercial technology such as Faceware (seen in games like Crysis 3) along with CounterPunch’s own tech for producing jaw-dropping visuals and facial expressions in 3D characters.

When asked about what next gen consoles like the PS4 and Xbox One could provide in terms of visual fidelity, Fenske stated that, “We’re starting to develop technologies built around realtime applications. We are also starting to have discussions with Stereoscopic companies to project out where we think the gaming industry is going. We think one day people will be playing with holograms.”

As for the Xbox One’s 8GB DDR3 RAM versus the PlayStation 4’s 8GB GDDR5 RAM, and any advantages derived from the latter because of this discrepancy, Fenske remarked that, “Anytime there is an increase in the hardware there will be an improvement in the level of facial quality that can be achieved.

“We will be able to use higher resolution models, rigs and textures. This only increases the believability in the facial animation and the deformation quality our facial rigs. Whether the PS4 has an advantage over the Xbox One is yet to be seen on our end but both are a significant upgrade from the current generation.”

The main appeal of next generation consoles though, is their PC like architecture and tremendous amounts of power. This helps facilitate easier development while still bringing cutting edge graphics to gamers. Will a day come when next gen consoles allow for easier facial animation as well?

“No matter how much technology advances, there is still no answer for the emotion a human being brings to a character. The fidelity will increase and so will the need for people who can bring life and emotion to a character.”

As for the next generation of gaming, stay tuned as E3 2013 gets closer and we learn what developers will have in store for us in the coming months.

One of the biggest new additions, as evidenced in the gameplay trailer released from the event which looked at the next generation engine, was the facial animation Infinity Ward had employed. Despite animation lead Zack Volker being rather dodgy on what exactly make a new engine “new”, the facial animation was of a standard never before seen in a Call of Duty game.

The new engine tried to take advantage of the fact that the curvature of an object would remain constant, even as it came closer or further to the player. A similar method was seen more than a decade ago with Shiny Entertainment’s Messiah, wherein character models and environments would increase or decrease in the number of polygons being showcased depending on how near or far you got from them. This was technically supposed to help increase the efficiency of the engine, as polygons that weren’t displayed were not rendered and resources could be spent on making what was being displayed look even better.

Not that it worked in Messiah’s case but we digress.

Nowadays, the market for facial animation in video games has diversified greatly. One look at Crysis 3’s facial animation will tell you that developers aren’t playing around, using a combination of both licensed and propriety technologies and workflows to achieve the most realistic animation possible. So how well does Activision’s new engine measure up to what was shown at this year’s Game Developers Conference 2013?

Skip to 2:27 on the Xbox Reveal trailer to compare the current look of the animation to what was seen at GDC.

As stated in the comments, the GDC character trailer involved a majority of resources being devoted to lighting and animating a single head. Similar to what David Cage showcased at the PS4 Reveal event, this isn’t too difficult. But when you factor in a complete character model, other characters, environments, AI, particle effects, weather effects and a hundred other factors, it’s a given that you won’t get the exact same quality.

So it is a definite tantamount to the technology at hand that even if Activision’s engine couldn’t approach nearly the same quality shown in the GDC trailer, it’s still pretty decent in the larger scheme of things.

Does it look better than the crème of the crop in Crysis 3? Not really. But considering Infinity Ward’s approach of only including elements to increase immersion when it fits their definition of the gameplay experience, it makes sense. Right? Also, keep in mind that Ghosts is still, by all intents and purposes, a first generation title for the PS4 and Xbox One. Future games could significantly improve on what’s seen here.

Then again, Activision did recycle the same engine for years on end for the last generation of Call of Duty, only making an enhancement here and there when necessary. This resulted in Call of Duty titles that remarkably similar to each other with little improvements, to say nothing of the gameplay.

But we digress again.

What do you think about the facial animation processes in Call of Duty: Ghosts? Let us know in the comments below.