A variational Bayes framework for sparse adaptive estimation
Recently, a number of $\ell_1$-norm regularized least squares (LS) type algorithms have been proposed to address the problem of sparse adaptive signal estimation and system identification. From a Bayesian perspective, this task is equivalent to maximum a posteriori (MAP) estimation under a sparsity promoting heavy tailed prior for the parameters of interest. Following a different approach, this paper develops a unifying framework of sparse variational Bayes schemes that employ heavy tailed priors in conjugate hierarchical form to facilitate posterior inference. The resulting fully automated variational schemes are first presented in a batch iterative mode. Then it is shown that by properly exploiting the structure of these batch estimation schemes, new sparse adaptive variational Bayes algorithms can be derived, which have the ability to impose and track sparsity during real-time processing in a time-varying environment. The most important feature of the proposed algorithms is that they completely eliminate the need for computationally costly parameter fine-tuning, a necessary ingredient of sparse adaptive deterministic algorithms. Simulation results are provided to demonstrate the effectiveness of the proposed sparse adaptive variational algorithms against state-of-the-art deterministic techniques for adaptive channel estimation. The results show that the proposed variational Bayes algorithms are numerically robust and have in general superior estimation performance compared to their deterministic counterparts.
A Novel Hierarchical Bayesian Approach for Sparse Semisupervised Hyperspectral Unmixing In this paper the problem of semisupervised hyper spectral unmixing is considered. More specifically, the unmixing process is formulated as a linear regression problem, where the abundance’s physical constraints are taken into account. Based on this formulation, a novel hierarchical Bayesian model is proposed and suitable priors are selected for the model parameters such that, on the one hand, they ensure the nonnegativity of the abundances, while on the other hand they favor sparse solutions for the abundances’ vector. Performing Bayesian inference based on the proposed hierarchical Bayesian model, a new low-complexity iterative method is derived, and its connection with Gibbs sampling and variational Bayesian inference is highlighted. Experimental results on both synthetic and real hyperspectral data illustrate that the proposed method converges fast, favors sparsity in the abun dances’ vector, and offers improved estimation accuracy compared to other related methods.
- Statistical signal and image processing
- Probabilistic machine learning
- Sparse and low-rank representations
- Spectral unmixing, hyperspectral image processing
- Adaptive estimation
- Image restoration, segmentation
- Astrophysics, weak gravitational lensing
- Athanasios Rontogiannis, National Observatory of Athens, Athens
- Konstantinos Koutroumbas, National Observatory of Athens, Athens
- Paris Giampouras, National Observatory of Athens, Athens
- Sergios Theodoridis, University of Athens, Athens
- Jean-Luc Starck, CEA Saclay, Paris
- Florent Sureau, CEA Saclay, Paris