Exploiting convolutional neural networks for superpixel based contextual description

Abstract

Remote sensing is an important technique for acquiring consistent, repeated high resolution observations of large-scale phenomena and processes. However, the ever increasing amount of data brought by the continuing improvement in coverage and repetition rates in recent years makes the manual interpretation of the raw data in many cases impracticable. Attaining automatic procedures to extract useful semantic information is then crucial for enabling the latest remote sensing based applications. Automatic methods offer a wide gamut of benefits and are the focus of many research fields. Potential applications range from changes in vegetation and climate, ice melt, etc. to the mapping of natural disasters such as earthquakes, tsunamis, landslides or avalanches, to tracking of moving objects, e.g. for traffic monitoring or surveillance and urban planning. Due to the complexity of these phenomena a fully automatic processing is often not possible yet. However, reliable, fast and objective processing of the recorded data in all these cases is of great importance and requires advanced methods. The hike in spatial resolution of satellites has increased the scope for feature extraction leading to a rising number of land cover classes, while the underlying semantics of these classes were not explored. Pixel-based classification methods became then less effective, since the relationship between the pixel size and the dimension of the observed objects on the Earth's surface has changed significantly. Therefore object-oriented classification has become increasingly popular over the past decade. This combines segmentation and contextual classification. Segmentation divides the image into homogeneous pixel groups (segments), which are arranged into classes based on their spectral, geometric, textural and other features. However, this task is notoriously challenging and low-level descriptors usually employed ignore semantic information that may be yielded by the spatial configuration of the classes present in an image. Deep learning, a recent breakthrough in machine learning, has shed light on this problem. The aim of this work is to evaluate the applicability of a superpixel based contextual description method used for image classification to the scenario of object detection in remote sensing imagery. Such method exploits convolutional networks to compute deep contextual features from different context levels surrounding the superpixels obtained. The method is modified in order to assess the use of different pooling methods and is tested on publicly available datasets and custom datasets created to evaluate the methods suitability to different scenarios.