New techniques for spatial resolution enhancement of hyperspectral images

Abstract

Vision plays a central role for human perception and interpretation of the world. With the beginning of the space age during late 1950s presented opportunities for remote sensing of earth resources [1]. Multispectral sensors image the earth in a few strategic areas of the electromagnetic spectrum using a small number of carefully chosen spectral bands (typically 3 to 10) spread across the visible and infrared regions of the electromagnetic spectrum. These bands are not contiguous and omit many wavelength ranges. The ability of a sensor to distinguish between wavelength intervals in the bands describes the spectral resolution. Higher the spectral resolution, narrower the wavelength range for a band. The spectral resolution determines the materials discrimination ability of the sensor. The high spectral resolution of multispectral imaging was found useful for ground-cover classification, mineral exploration, and agricultural assessment to name a few. In remote sensing, acquisition of image details helps accurate localization and correct identification of minerals and vegetation, and hence better classification of the landmass. The hardware in the remote sensing sensors limits the amount of detailed information captured (i.e., spatial resolution) whenever the spectral width of the acquired image is small. The size of the ground area expressed as meter x meter represented by a single pixel in an image defines the spatial resolution of the image. Smaller the ground area per pixel means higher spatial resolution. It depends on the sensor design and the height of the sensor above the ground. To increase the spatial resolution without affecting spectral resolution, the sensor should have a small instantaneous field of view (IFOV). But this reduces the signal power falling on the detector and hence signal to noise ratio is reduced. One can increase signal to noise ratio by widening the bandwidth of the acquired spectral band, but this reduces the spectral resolution of the image. Thus, there exists a trade-off between spatial and spectral resolutions of remotely sensed images. Multispectral images provide higher spectral resolution (of the order of 100 nanometers) but they suffer from low spatial resolution. Improved versions of these early multispectral imaging sensors known as hyperspectral imager provide spectral width of 10 nanometers for each band of hyperspectral image (HSI). Having very high spectral resolution they provide ample spectral information to identify and differentiate spectrally unique materials [1]. Hence, presently they are used in wide range of military and civilian applications that include target detection and tracking of objects, agriculture planning, forest inventory, mineral exploration, and urban planning to mention a few. Similar to the multispectral images hyperspectral images also suffer from low spatial resolution due to very small spectral width. Many times it is not feasible to capture the spatially high resolution (HR) images due to the limitation in implementation such as requirement of large memory, higher transmission bandwidth, high power requirement and higher camera cost. Since HR imaging leads to better analysis, classification and interpretation one may look for algorithmic approaches to obtain the HR images. Hence, we need to perform post processing of the hyperspectral images (HSIs) to increase their spatial resolution and hence the image details, without affecting their spectral resolution. Super-resolution enhancement refers to an algorithmic approach to increase the spatial resolution of a low spatial resolution image by using either multiple low-resolution (LR) observations or using a database of high and low-resolution images. Many satellites like WorldView-1, 2, 3, SPOT, Landsat, Quickbird, Ikonos, etc. capture two different types of images, namely, the high spectral but low spatial resolution multispectral (MS) images and high spatial but low spectral resolution registered panchromatic (PAN) image (auxiliary image). The reason behind configuring satellite sensors this way is to reduce weight, cost, bandwidth and complexity of the satellite. In this thesis, we develop different algorithms to enhance the spatial resolution of hyperspectral images. To start with, we first address the problem of enhancing the spatial resolution of MS images by merging information from PAN image, called multiresolution fusion, using two step approach. In the first step, the high-resolution edge details of the fused multispectral image are learned in the form of an initial estimate using discrete wavelet transform and compressive sensing (CS).We know that PAN and MS images are obtained from the same geographical region with the difference that PAN image is acquired with high spatial resolution. We assume panchromatic image and the multispectral bands are in registered form. This results in high spatial correlation between the MS observation and the coarser part of the PAN image. Our approach uses CS technique to obtain the detailed wavelet coefficients of the MS image by assuming the same sparseness of MS image with the coarser level, as well as detailed level of the PAN image. This way we obtain initial estimate of the fused image. To better preserve spatial homogenity, in the second step, we regularize it further to obtain the final solution. We restrict the solution space by using maximum a posteriori - Markov random field (MAP-MRF) approach that imposes smoothness constraint on the fused image by using first order neighborhood for MRF prior. We make use of the initial estimate to obtain the MRF parameter. Hyperspectral images are used for the same purpose as do MS images, but they have very high spectral dimensions that enables distinguishing the spectrally unique materials. The statistical classification (clustering) methods often used with multispectral images can also be applied to hyperspectral images by handling their high dimensionality [2]. Hyperspectral sensors like AVIRIS, Hyperion, HYMAP do not capture auxiliary HR image. In such circumstances we cannot use fusion to increase the spatial resolution of HSI. Our remaining three techniques discussed in this thesis deal with spatial resolution enhancement of HSIs using the concept of super-resolution without making use of auxiliary HR image (i.e. PAN image). The goal of super-resolution (SR) is to recover high-frequency details lost during image acquisition process which in turn increases the number of pixels in the input image. This is an inverse problem wherein the original high-resolution (HR) image has to be retrieved from the observed low-resolution data. There are large number of HR images which are consistent with the LR image. Hence, while solving such an ill-posed inverse problem, knowing the forward model alone is not sufficient to obtain a satisfactory solution. We need to add proper constrains by using priors to limit the solution space. This procedure to get a solution of the inversion problem in accordance with the prior information is called regularization. Selection of appropriate model as the prior information and use of regularization helps to obtain improved solution. In our work, we have considered different kinds of priors in regularization in order to obtain improved solution. We make use of compressive sensing theory and estimated wavelet filter coefficients to obtain SR results for HSIs. To reduce high computational load due to large number of spectral bands of HSIs, we use principal component analysis (PCA) to reduce the dimensions and work on reduced dimensional space to obtain SR results. In the first method, we use CS based approach to obtain initial SR of the most informative PCA image which represents highest spectral variance of the HSI. Here we use LR and HR raw dictionaries having large number of atoms in the CS based framework. Using the sparsity constraint, LR test patch is represented as a sparse linear combination of relevant LR dictionary elements adaptively. Assumption of same sparsity to LR and HR images with respect to their dictionaries gives SR image as an approximate. The final SR solution is obtained using a regularization in which AR prior model parameters are obtained from the initial SR estimate obtained using CS. SR results of the other significant PCA components are obtained using the same AR parameters and using the regularization framework. While regularization, decimation process is modeled as an averaging process. The decimation process modeled as the averaging process represents the aliased pixel in the low-resolution image by averaging the corresponding pixels in the high-resolution image. This means, the point spread function (PSF) of sensor considered is square and is same for all spatial and spectral region. However, in practice PSF depends on several factors like camera gain, zoom factor, and imaging hardware etc. This motivates us to estimate the PSF i.e., the aliasing and then perform SR. Here our CS based approach is further extended to obtain initial SR of all significant PCA components that represent most of the spectral variance (98 %) of the HSI where the aliasing is estimated for all the significant PCA components. Here we use jointly trained LR and HR dictionaries having very less number of atoms (i.e. 1000) using training algorithm called K-singular value decomposition (K-SVD). This is onetime and offline procedure. Regularization using our new prior i.e., Gabor prior preserves various bandpass features in the final SR image. Also the use of estimated entries of degradation matrix in the form of PSF represents imaging hardware more effectively in image observation model. This leads to better solution of final SR result. Finally, we address learning based super-resolution in wavelet domain using estimated wavelet filter coefficients. In this work, we estimate the PSF in the form of wavelet filter coefficients to take care of the degradation between LR and HR images. Here we do not consider spatially varying PSF, which is quite involved as this requires the estimation of PSF at every pixel. However, the space invariant PSF is estimated for individual spectral bands. The estimated filter coefficients are also used to learn high frequency details by using the HR training images in wavelet domain. This gives us an initial estimate of SR image for each HS band and they are used in deriving the sparse coefficients that are used as priors. The final SR image is obtained using the sparsity based regularization that also has the observation model constructed using the estimated filter coefficients. Since the cost function is differentiable, a simple gradient descent optimization is used to obtain final solution. We show the computational advantage of the proposed algorithm.