Going Deep into Remote Sensing Spatial Feature Learning

Abstract

A lot of information may be extracted from the Earth's surface through images acquired by airborne sensors. Even more, nowadays, enhanced information may be extracted from high spatial resolution images obtained from new sensor technologies. This information may be exploited in several tasks (including, classification and segmentation) assisting in different applications, such as urban planning, disaster relief, phenological studies, etc. Although widely used, the process of distilling this information is strongly based on efficiently encoding features, a challenging task given the distinct properties of the images. In fact, independently of the task, encoding the spatial features in an efficient and robust fashion is the key to generating good discriminative models. Even though many descriptors have been proposed and successfully used to encode spatial features of remote sensing images, most applications demand specific description techniques. Deep Learning, an emergent machine learning approach based on neural networks, is capable of learning specific features and classifiers at the same time and adjust at each step, in processing time, to better fit the need of each problem. For several tasks, it has achieved very good results, mainly boosted by the feature learning performed which allows the method to extract specific and adaptable features depending on the data. In this project, we tackled two main tasks, image and pixel classification, using Deep Learning to encode spatial features over high-resolution remote sensing images. Initially, we faced the scene classification problem, in which the final model performance is highly dependent on the quality of extracted features. Based on recently published works and after a systematic set of experiments, our first contribution was a novel Convolutional Network architecture fully trained specifically for the remote sensing domain. Experiments showed that the proposed architecture outperformed state-of-the-art algorithms, demonstrating the effectiveness of deep learning methods to encode features even for remote sensing domain. Then, based on adversities to explore deep learning-based techniques (such as small data), we proposed to evaluate different strategies to better exploit Convolutional Networks in the remote sensing domain. Specifically, three strategies were evaluated in three very distinctive datasets. Experiments demonstrate that fine-tuning a network into the specific domain is the best strategy to exploit Convolutional Networks. A second task tackled by this project is the pixelwise classification (also known as semantic segmentation). This task is of great interest to the remote sensing community since it involves the automatic creation of thematic maps from labeled samples. Such maps may help to understand events over a specific region, such as new urban areas, deforestation, etc. Considering such task, our first contribution is a new Convolutional Network architecture to perform pixel classification of remote sensing images. This technique is based on context windows, i.e., overlapping patches that carry the context of the centered pixels and help to understand the spatial patterns around them. Experiments showed that the proposed approach provides significant improvements when compared to traditional and state-of-the-art methods. Then, based on the problems of the previous technique and of other existing methods, we proposed a novel approach that employs an evolved version of the previously proposed network (as well as other architectures) to perform pixel labeling of the entire input image (instead of classifying only the centered pixel). Moreover, this new technique is able to aggregate multi-context information without increasing the number of parameters. Results showed that the proposed algorithm provides improvements in pixelwise classification accuracy when compared to state-of-the-art methods. Finally, our last contribution is a novel method that, when compared to other existing approaches, is capable of better learning some visual characteristics (such as edges and corners), which are very useful for some applications and domains, including the remote sensing one. Precisely, this method, called Deep Morphological Network (DeepMorphNet), is able to perform and optimize non-linear morphological operations, that may be able to better cope with some interesting visual properties of the objects. These DeepMorphNets can be trained and optimized end-to-end using traditional existing techniques commonly employed in the training of deep learning approaches and can be used in several applications, such as image and pixel classification, detection, and so on. A systematic evaluation of the proposed algorithm was conducted. Experimental results showed that the DeepMorphNets is a promising technique that can learn distinct features when compared to the ones learned by current deep learning methods.