Performance analysis of hyperparameter optimization in sparse Bayesian learning via Stein’s unbiased risk estimator

Sparse Bayesian Learning (SBL) is a widely-used framework for sparse signal reconstruction, yet its standard formulation optimizes model evidence rather than directly minimizing reconstruction error. In this paper, we reinterpret standard SBL as an approximate scheme for minimizing the mean squared error (MSE) in the input domain. Motivated by this insight, we derive novel hyperparameter update rules aimed at minimizing input-space MSE, and discuss the limitations of using Stein’s unbiased risk estimate in underdetermined systems. To address this issue, we propose an alternative risk minimization framework based on output-space MSE, which admits an unbiased estimator. We derive closed-form coordinatewise update rules for the regularization parameters and analyze their sparsity-promoting behavior. In particular, we identify a sufficient condition—termed the statistical orthogonality condition (SOC)—under which certain components are optimally pruned. This connects our framework to classical sparse recovery criteria. While our analysis sheds light on the emergence of sparsity via risk-based optimization, it also highlights open questions regarding the conditions under which SOC is satisfied, warranting further investigation.