When it comes to human faces, everyone´s an expert. We´re conditioned since birth to detect both verbal and non-verbal facial cues and make judgements based those cues. Studies have shown a breakdown in communication and a negative bias towards those suffering from facial paralysis despite the best intentions of the afflicted. Thus the idea of a computer generated “digital double” passing for a real person presents a tremendous challenge not only technically but also psychologically.
Against the odds—much like our protagonist Jack Joyce—we set out to re-create convincing performances so that the player could empathize with our characters. With an array of DSLR cameras we captured their complexion, their pore details, their crow’s feet, their scars, the color of their irises, even a cast of their teeth was made to get that authentic smile. We captured ranges of motion for both their face and their body in all imaginable poses, with them in their underwear and fully clothed. We were then able to reconstruct extremely accurate models and texture maps from all the images we had amassed. We built a repeatable process and became proficient at it.
Our key finding was that instead to trying to decompose the human condition into a canon of standardized shapes and their linear activations, realism was better served by capturing their actual facial performance. This was done using a DI4D system, famous for its use in neuroscientific studies at the University of Glasgow. The system combined nice cameras at up to 60 frames per second near high definition resolution. We had a team of four people working full time “tracking” the resulting data, all five hours and 120 terabytes of it, using the face model we had constructed. We then augmented the data with wrinkle activation trained against the range of their facial motion.
This process gave us extremely fluid motion reflecting the nonlinear nature of facial deformation while sacrificing some of the extreme detail like skin pore stretching. Our tests convinced us that the nonlinear motion resulting from the Interaction of the skull, facial muscles, fascia and skin all but made up for the extreme detail. We could see that our characters were breathing as they spoke, that their faces filled up with blood as they struggled and ultimately it was apparent that it was what mattered most for us to feel them inside Quantum Break. Nothing is ever perfect, but we felt we did them no disservice.
