[Colloq] REMINDER (TODAY): Title: Beyond Pixels: Exploring Flexible Representations for Image Motion Estimation | Deqing Sun, NVIDIA Research | 3/2/16 1:30-2pm 366WVH

[Walker, Lashauna]

Title: Beyond Pixels: Exploring Flexible Representations for Image Motion Estimation
Speaker: Deqing Sun, NVIDIA Research
Date: 3/2/16  | Time: 1:30-2pm  | Location: 366WVH



Title:
Beyond Pixels: Exploring Flexible Representations for Image Motion Estimation

Abstract:
We live in a dynamic world where motion is ubiquitous. To make robots and other intelligent agents to understand the world, we need to give them the ability to perceive motion. Estimating image motion and segmenting the scenes into coherently moving regions are two closely related problems but are often treated separately. Motion actually provides an important cue to identify surfaces in a scene, while segmentation may provide the proper support for motion estimation. Despite decades of research efforts, current methods still tend to produce large errors especially near motion boundaries and in occlusion regions.

In this talk, I will start from a probabilistic layered model for joint motion estimation and segmentation. This model orders each moving object (layer) in depth and explicitly constructs the occlusions between layers. It explains segmentation using thresholded spatio-temporally coherent support functions, and describes motion using globally coherent but locally flexible priors. In this way, scene structures (segmentation), instead of motion, are enforced to persist over time. Our method achieves promising results on both the Middlebury optical flow benchmark and the MIT layer segmentation dataset, particularly in occlusion regions.

Noting that "global" layered models cannot deal with too many layers or capture mutual or self-occlusions, I will introduce a local layering representation. It breaks the scenes into local layers and jointly models the motion and occlusion relationship between local layers. By retaining uncertainty on both the motion and the occlusion relationship, we can avoid common local minima of motion-only or occlusion-only approaches. Our method can thus handle motion and occlusion well for both challenging synthetic and real sequences.

Finally I will introduce our recent work on semantic motion estimation. While existing methods make generic, spatially homogeneous, assumptions about the spatial structure of the motion, motion varies across an image depending on object class. Here we exploit recent advances in static semantic scene segmentation to segment the image into objects of different types. We define different models of image motion in these regions depending on the type of object. We pose the motion estimation problem using a novel formulation of localized layers, which addresses limitations of traditional layered models for dealing with complex scene motion. Our semantic motion approach achieves the lowest error of any published monocular method in the KITTI-2015 flow benchmark and produces qualitatively better flow and segmentation than recent top methods on a wide range of natural videos.

Bio:
Deqing Sun is a senior research scientist at NVIDIA Research. He was a postdoctoral research fellow in Prof. Hanspeter Pfister's Visual Computing group at Harvard University from Aug. 2012 to Oct. 2015 (and is still an affiliate researcher). He received his BEng degree in Electronic and Information Engineering from Harbin Institute of Technology, his MPhil degree in Electronic Engineering from the Chinese University of Hong Kong, and his MS and PhD degrees in Computer Science from Brown University working with Dr. Michael J. Black. He was a research intern at Microsoft Research New England from Oct. to Dec. 2010 working with Dr. Ce Liu. His research interests include computer vision, machine learning, and computational photography, particularly motion estimation and segmentation and the applications to computational video.



Thank You.

LaShauna Walker
Events and Administrative Specialist
College of Computer and Information Science
Northeastern University
617-373-2763
Facebook<https://www.facebook.com/ccisatnu?ref=hl> | Instagram<https://instagram.com/ccisatnu/> | LinkedIn<https://www.linkedin.com/groups/Northeastern-University-College-Computer-Information-1943637?gid=1943637&mostPopular=&trk=tyah&trkInfo=idx%3A1-1-1%2CtarId%3A1426606862845%2Ctas%3ANortheastern+University+College+of+Com> | Twitter<https://twitter.com/CCISatNU>