This method can be used to digitise and render human faces for various computer graphics applications, such as film visual effects, games, virtual and augmented reality applications (AR/VR), as well as applications in computer vision such as material/facial appearance recognition and analysis. This is even more important when acquiring an object or subject’s shape (geometry and surface normal maps) and reflectance
(texture maps) from multiple viewpoints which is required for complete coverage of the human face (subject) surface for digitisation and 3D reconstruction.
Current alternative methods for facial digitization and rendering have some constraints when acquiring a subject. Passive capture do not usually acquire accurate reflectance and shape information. On the other hand, state-of-the-art active capture method employing a Lightstage crucially requires the use of polarised light (in orthogonal planes), which results in capturing double the number of images of the subject, ultimately increasing the duration of capture. This is problematic for facial capture of live subjects since it can lead to artefacts in the data due to minor subject motion during capture. The use of polarised light also reduces light efficiency as 50 % of the light is lost through the polarisers, and imposes viewpoint restrictions.
This invention is a novel active illumination capture method which overcomes the issues of alternative methods (described above), requires a simpler setup and enables high quality object capture and 3D reconstruction as well as realistic rendering of the 3D object using the separated diffuse and specular reflectance and normal maps. This can be achieved by taking just 6 photographs acquired from any viewpoint.
A rendering of the inventor, Dr Abhijeet Ghosh, made using this technology
This technology employs illumination of a subject under different binary lighting conditions to estimate the surface normals of the subject. The technology is based on the observation that when a dielectric surface (e.g. a human face) is oriented towards a darker hemisphere it exhibits pure diffuse reflectance while the surface oriented towards the light hemisphere exhibits both diffuse and specular reflectance. This observation is exploited to separate diffuse and specular reflection and provide the diffuse and specular albedo of the subject. The method does not impose viewpoint restrictions, requires fewer photographs and a simpler setup compared to alternative state-of-the-art technology.
Further, an efficient two-photo capture using spectral multiplexing can be employed for dynamic capture with reflectance separation.
- High quality capture, digitsation, 3D reconstruction and rendering or human face.
- Faster capture of subject than state-of-art active capture technology.
- Simpler setup and no viewpoint restrictions compared to state-of-art technology.
- Increased light efficiency achieved by the use of unpolarised light.
- The potential for two-shot dynamic capture of subject.
Intellectual property information
A patent application for this technology has been filed in US (application no. 16/459200/US/PRV).
Link to published paper(s)
Kampouris C, Zafeiriou S, Ghosh A, 2018. Diffuse-specular separation using binary spherical gradient illumination, Eurographics Symposium on Rendering (EGSR) 2018, Publisher: The Eurographics Association, ISSN: 1727-3463
A. Lattas, M. Wang, S. Zafeiriou and A. Ghosh. Multi-view facial capture using binary spherical gradient illumination. ACM SIGGRAPH 2019 Posters, Publisher: SIGGRAPH 2019, DOI:10.1145/3306214.3338611.
Dr Abhijeet Ghosh, Reader in Graphics and Imaging, and members of the Department of Computing.
+44 (0)75 6355 5266