Some New Techniques for Motion-free Super-resolution

Abstract

Digital image processing has exhibited a tremendous growth during past decades in termsof both the theoretical developments and applications. At present, image processing andcomputer vision are the leading technologies in a number of areas that includes digitalcommunication, medical imaging, the Internet, multimedia, manufacturing, remote sens-ing, biometrics and robotics. The recent increase in the widespread use of cheaper digitalimaging terminals such as personal digital assistants, cellular phones, digital camera, highde nition TV and computers in consumer market has brought with it a simultaneous de-mand for higher-resolution (HR) images and video. Since the high resolution images andvideo carry more details and subtle gray level transitions, they o er pleasant views of thepictures and videos on these devices. In commercial and industrial applications the highresolution images are desired as they lead to better analysis, interpretation and classi-cation of the information in the images. High resolution images provide better detailsthat are critical in many imaging applications such as medical imaging, remote sensing,surveillance.The resolution of the image captured using a digital camera depends on the number ofthe photo detectors in the optical sensors. Increasing the density of the photo detectorsleads to high resolution images. The current hardware approach to capture images withhigh resolution relies on sensor manufacturing technology that attempts to increase thenumber of pixels per unit area by reducing the pixel size. The cost for such sensorsand related high-precision optics may be prohibitively high for consumer and commercialapplications. Further, there is a limitation to pixel size reduction due to shot noiseencountered in the sensor itself. Since the current sensor manufacturing technology hasalmost reached this limit, the hardware approach is no more helpful beyond this limit.One promising solution is to use signal processing approaches based on computational,mathematical, and statistical techniques. Because of the recent emergence of these key-relevant techniques, resolution enhancement algorithms have received a great deal ofattention. Super-resolution is an algorithmic approach to reconstruct high resolutionimage using one or more low resolution images. The main advantages of the approach arethat it costs less, it is easy to implement and the existing low resolution imaging systemscan be used without any additional expense. The application of such algorithms will xcertainly grow in situations where high quality optical imaging systems are too expensiveto utilize.The motion based super-resolution approaches produce a high resolution image usingnon-redundant information from the multiple sub-pixel shifted low resolution observa-tions. The di culty in these approaches is the estimation of motion between the lowresolution frames at a sub-pixel accuracy. Motion-free super-resolution techniques allevi-ate this problem by using cues other than motion cue. The additional observations aregenerated without introducing relative motion among them.In this thesis, we present learning based approaches for motion-free super-resolution.First we solve the super-resolution problem using zoom cue. The observations of a staticscene are captured by varying the zoom setting of a camera. The least zoomed imagecontaining the entire scene is super-resolved at the resolution of the most zoomed imagewhich contains a small area of the entire scene. Generally, the decimation process ismodeled as the averaging process and the aliased pixel in the low resolution image isobtained by averaging the corresponding pixels in the high resolution image. However,aliasing depends on several factors such as zooming and camera hardware. This motivatesus to estimate the aliasing. Since a part of the scene is available at high resolution inthe most zoomed images, we make use of the same to estimate the aliasing on the lesserzoomed observations. The aliasing is estimated using the most zoomed image and thelesser zoomed images. We represent the super-resolved image using Markov random elds(MRF) and obtain super-resolution using maximum a posteriori technique. We demon-strate the application of proposed aliasing learning technique to the fusion of remotelysensed image. While experimenting, the MRF prior model parameters were adjusted ontrial and error basis. A better solution can be obtained using the parameters estimatedfrom the observations themselves. The estimation of the parameters requires the compu-tation of the partition function. Since it is a computationally intensive technique, we useautoregressive (AR) model to represent the super-resolved image. The AR prior modelparameters are obtained from the most zoomed observation. We apply this technique tothe fusion in remotely sensed images.The spatial features of a low resolution image are related to its high resolution version.The analytical representation of the relationship of the spatial features across the scalesis di cult. This motivates us to prepare a database of low resolution images and its high resolution versions all captured using same real camera and use this database to obtainhigh frequency details of the super-resolved image. We propose wavelet based new learn-ing approach using this database and obtain close approximation to the super-resolvedimage. The close approximation is used as an initial estimate while minimizing the costfunction. We employ a prior model that can adapt to the local structure of the image andestimate the model parameters as well as the aliasing from the close approximation. Theproposed approach is extended to super-resolve color images. We learn the details of thechrominance components using wavelet based interpolation technique and super-resolvethe luminance component using the proposed approach. We show the results for graylevel images and for color images and compare the them with existing techniques.In most current image acquisition systems in handheld devices, images are compressedprior to digital storage and transmission. Since, the discrete cosine transform (DCT) isthe basis of many popular codecs such as JPEG, MPEG and H.26X, we consider the DCTfor learning. The use the DCT for learning alleviates the limitations of wavelet basedlearning approach that it can not recover the edges oriented along arbitrary directions.We propose a learning based approach in the discrete cosine transform domain to learnthe ner details of the super-resolved image from the database of low resolution (LR)images and their high resolution (HR) versions. Regularization using the homogeneousprior model imposes the smoothness constraint everywhere in the image and leads tosmooth solution. To preserve edges and ner details, we represent the super-resolvedimage using nonhomogeneous AR prior model and solve the single frame super-resolutionproblem in regularization framework.Finally, we readdress the zoom based super-resolution problem using discontinuitypreserving MRF prior in order to prevent the distortions across the edges while optimiza-tion. We obtain the close approximation for the super-resolved image using the learningbased approach and use it to estimate the model parameters and the aliasing. Since thecost function consists of a linear term and a non-linear term, it cannot be optimized usingsimple gradient descent optimization technique. The global optimization technique suchas simulated annealing can be employed. Since it is computationally taxing, we proposethe use of particle swarm optimization technique. We show the computational advantageof the proposed approach.