In today’s film industry, visual effects have become increasingly more realistic and immersive.
Lea Zhu, a PhD candidate in, Digital Media started this research with the original purpose of enhancing visual effects compositions in the film industry.
“It is about designing and developing a system that could output a high-quality sequence of depth maps for live-action characters or objects,” explained Zhu. “With our system, the occlusions (i.e., the blind spot of each camera) are effectively reduced with the two-axis camera deployment, and after using our algorithm to merge the information from each perspective, our generated depth sequence seems highly applicable to help simplify the object extraction and VFX composition process, which were tested in both virtual environments and live-action scenarios.”
Zhu’s motivation behind this research is that she is a big fan of fictional movies and finds excitement in concepts and visual effects used in these movies.
Shared Zhu: “When working on some VFX projects during my undergrad and graduate courses, I found that some of the commonly used techniques for element extraction are either hard to set up or very time-consuming while extracting elements is always required in post-production. Thus, when my supervisor mentioned this topic, I was very excited to try to make it happen.”
This research aims to solve problems in the film and visual effects industry with an emphasis on accuracy and speed. It will also create a more enhanced viewing of special effects films for the general public.
Zhu has been conducting this research in four stages. The first stage consisted of pre-validating the research idea as there has not been similar research that has built on the stereo system like the patten Zhu is using.
“The second stage is to physically build the system and apply the algorithm with real-life data,” explained Zhu. We built our system with computer vision cameras and 3D printed components; used them to capture data for live-action objects and character and generated depth sequences for each shot.”
The third and fourth stages required testing the research through quantitative comparison and the visual effects composition test.
So far, Zhu’s findings have been that their system helps improve depth quality by reducing occluded areas and patching them with information acquired from other directions effectively.
As Zhu came from ShenZhen, she had to adjust starting her masters degree in a new environment. Although, she found Carleton to be a very nice and inclusive environment.
“I got to know many new friends from the program and the professors in our school are very friendly and knowledgeable, so I quickly got to familiarize the environment and had a lot of fun taking courses, experiencing the cool devices in our school and doing interesting research,” explained Zhu.
Professor Chris Joslin, the director of the School of Information Technology has been supervising Zhu throughout this research.
“He has helped me very much from selecting the research topic to submitting it as my master’s thesis till now as we are trying to improve the research,” expressed Zhu. “This project would not have come this far without his guidance and advice in the variety of studies and research methods.”
Zhu’s goal for this research is to make it easily applicable in the cinematographic scenario and post-production pipeline.
She notes that the research still needs improvements and exploration on handling shots with a lot of motion blur and texture-less area.
“Investigating the compatibility and synthesis of small lens camera and cinematographic camera would be another thing we need to figure out,” shared Zhu. “Moreover, we are looking into the potential of integrating deep learning into the depth estimation process to help identify objects in a shot thus to improve the depth estimation reliability.”
Monday, February 8, 2021 in News, Research
Share: Twitter, Facebook
