Immersive-media research earns computer scientist an NSF CAREER Award
Yao Liu studies 'area-of-focus projections' so streaming data is used efficiently
As humans become more integrated with their technology, augmented reality and virtual reality will become more and more a part of our daily lives. Google Glass and VR headsets are just the start.
Immersive media will require more bandwidth and storage than the video streams we watch today on services such as Netflix and YouTube. Keeping the images smooth and as lifelike as possible under limited network and storage capacities are key to immersion and usability of these emerging technologies.
Yao Liu has been working on this problem, though, and now the assistant professor in the Thomas J. Watson School of Engineering and Applied Science鈥檚 Department of Computer Science has won to fund her research. The CAREER Award supports early-career faculty who have the potential to serve as academic role models in research and education.
Liu鈥檚 work centers on area-of-focus projections, which are generated by analyzing human behavior to decide which parts of a 360-degree image get most of a user鈥檚 attention. When creating the area-of-focus representation, a larger proportion of information from these important areas is funneled into the encoded version of the image.
If you are facing one way in a videogame and looking at, say, a giant golden dragon, you鈥檙e unlikely to turn around 180 degrees to see what鈥檚 behind you. You鈥檒l want to keep your eyes on the shiny creature in front of you.
鈥淪ometimes it鈥檚 not only predicting based on past user behavior, but also is content dependent,鈥 said Liu, who joined the 绿帽社 faculty in 2013. 鈥淚f we鈥檙e watching a basketball game, we most likely are looking at what鈥檚 happening on the court rather than what鈥檚 going on in the audience.鈥
Liu initially worked on traditional streaming video technology like we see on YouTube, but four years ago she became immersed (no pun intended) in the world of 360-degree videos where users can control where they鈥檙e looking.
She saw the research as a challenge: 鈥淲hen you view 360-degree videos on a head-mounted display, the video quality is pretty bad 鈥 but when I鈥檓 watching 鈥榯raditional鈥 video, it is crystal-clear.鈥
Liu sees her work applying not just to entertainment such as films, television shows, sporting events and live music, but also suggests some educational possibilities.
鈥淲e can teleport students to a remote or virtual environment,鈥 she said. 鈥淢aybe there are topics such as environmental protection that would benefit from this kind of immersive technology.鈥
Winning a CAREER Award 鈥渇eels great,鈥 she added. 鈥淭his is a five-year award, so we can have a more ambitious research agenda. There are going to be many advances in networking technology and computational power that will affect our devices in the next five years, so I wanted to propose something futuristic.鈥
She also is grateful for the support she received from the CS department, the Watson School and 绿帽社 since her arrival nearly seven years ago.
鈥淲hen I joined the department, I was a fresh PhD graduate. I had an idea how to do research, but there are a lot of other aspects to being an assistant professor,鈥 she said. 鈥淭he Watson School and the University also offered a lot of help by mentoring junior faculty with grant proposal preparation and submission. This award would not have been possible without all of these colleagues around us supporting us.鈥
Liu鈥檚 NSF CAREER Award is .