Efficient Fully-Convolutional Networks for Image Perception

Abstract

Neural architecture search is widely applied to design networks to outperform manually designed architectures. However, it is not trivial to be directly applied to challenging perception tasks such as object detection since previous methods often rely on manually designed complex operations such as RoI pooling and RCNN heads. Thus, we look for universal fully-convolutional representations for perception tasks, which are easy to optimise and deploy because of their sim ple structures. They perform well on dense prediction tasks such as semantic segmentation, where the networks consist of a backbone module for visual feature extraction and a task-specific module for result generation. Designing the task-specific modules helps us understand how these networks tackle perception tasks and is also crucial for performance and efficiency improvements. However, fully-convolutional networks fall behind two-stage approaches on instance-level tasks such as object detection and instance segmentation. To solve this problem, we focus on designing fully-convolutional frameworks for instance detection tasks and study the task-specific structures and improve their performance by devising efficient neural architecture search algorithms. Our approach starts by designing fully-convolutional models for instance detection tasks. With de- formable convolution, we tackle the local-incoherence problem for top-down instance segmentation, resulting in a fully-convolutional model with equivalent expressiveness as a typical two-stage model. We also propose BlendMask, a fully-convolutional instance segmentation network that is faster and more ac- curate than the state-of-the-art two-stage models. Then we demonstrate the benefit of having uniform representation by designing the first a panoptic segmentation network solving instance and semantic segmentation with a single branch. Targeting to improve the design of task-specific modules for fully- convolutional perception models, we devised efficient neural architecture search algorithms and applied them to video segmentation and object detection.