This new technology developed by researchers from the University of Washington and Facebook can bring still images to life! (although it is not yet fully developed, and some still images, as you can see, move awkwardly and creep you out.) They call this “Photo Wake-Up”, which I think is a really straightforward name.
On their project page, UW computing scientists Chung-Yi Weng and Brian Curless, along with Facebook’s Ira Kemelmacher-Shlizerman, describe a process that can take one single photo and create a character that walks out of the frame toward the viewer. They can also make the character run, sit, or jump.
The researchers will present their algorithm later this month at the Conference on Computer Vision and Pattern Recognition in Long Beach, California, but the software begins by analyzing a still image to detect a human form and fits a morphable body on it. From there it creates a body map, labeling each of the parts. Once the map is done, the software uses it to construct the three-dimensional mesh of the character, estimating the weight of their body so it can later apply realistic motion. Finally, it takes the original image and uses it to build a texture around the 3D body. The final step is to reconstruct the background that is occluded by the person’s form so they don’t leave a white hole behind.
(Video Credit: Chung-Yi Weng/ YouTube)