On large scale near-independent blind source separation

Abstract

The thesis addresses Blind Source Separation (BSS) in Large Scale (LS) and near-Independent (nI) sources scenario. The large scale in BSS imply number of unknown sources ranging from 15 to 140, so that the corresponding number of unknowns to be optimized range from 100 to 10000. The real world sources producing either an added spurious local optima or a shift of global optima or both are defined to be near-independent with respect to the used BSS contrast as an optimization criteria. The exponentially increasing solution space with linearly increasing dimensions for optimization and added complexity in the optimization landscape due to the near-independent sources make the Large Scale near-Independent BSS (LSnIBSS) to be a more difficult problem than the BSS. As a solution to the LSnIBSS problem, the thesis derives suitable optimization criteria and a Large Scale Global Optimization (LSGO) technique. Looking Probability Density Function (PDF) as a generalized multivariate differentiable function, there is derived L2-Norm of Gradient of Function Difference (GFD) as a BSS contrast, where, GFD is the difference between gradient of product of marginal PDFs and gradient of joint PDF. A nonparametric estimation of the derived contrast is achieved through ‘least squares’ based kernel method in a single stage directly, instead of a two stages indirect estimation method. The contrast estimation is a particular demonstration of a derived more general method for information field analysis through a newly introduced concept of Reference Information Potential (RIP). The performance of kernel methods depend upon the choice of kernel bandwidth parameter. There is derived Extended Rule-of-thumb (ExROT) for bandwidth parameter selection in Kernel Density Estimation (KDE). The method is based on Gram-Charlier A-Series expansion as an approximation to the unknown PDF, assuming it being near Gaussian. The ExROT is better, in terms of Integrated Mean Square Error (IMSE) criteria of performance, compare to the Silverman’s Rule-of-thumb (ROT) for unimodal density estimation with marginal increase in computational cost. The ExROT derived for multivariate density estimation and multivariate gradient of density estimation are applied to the derived BSS contrast. To accommodate near-independent scenario, there is introduced a Search for Rotation based Independent Component Analysis (SRICA) algorithm using, Genetic Algorithm (GA) like, search based global optimization technique. The BSS contrasts in simultaneous mode are proved to be nonseparable optimization functions (functions those can not be optimized componentwise), a difficult class of functions for LSGO. Towards success of GA, the schema concept is further generalized to dependency relation based Extended Forma from an existing generalization of equivalence relation based Forma. The generalization has an impact on the current debate on whether minimal (binary) alphabet or maximal (float) alphabet of representation for GA success. Taking inspiration from nature, the work in the thesis recommends use of either an intermediate level of alphabet or varying representation throughout the search. The former suggestion is empirically realized through Mendelian GA (MGA) based on the operators exploiting Extended Forma. The latter suggestion is empirically realized through newly defined the Gradual search scheme, the Spiral search scheme and others. The concepts are combined together to achieve a GA variant for LSGO of nonseparable functions. The solution is tested on the LSGO test bench functions and applied to the LSnIBSS problem using various contrasts.