Explaining f-divergence-based regularization via local curvature and sharpness-aware minimization

Divergence-based regularization and Sharpness-Aware Minimization (SAM) are two prominent ap proaches for improving generalization in deep learning, both motivated by robustness to perturba tions. However, their relationship has remained largely unexplored. Building on classical second order expansions of f-divergences, we show that the two methods are locally consistent under parameter-space perturbations: both induce curvature-sensitive penalties, with divergence regular ization yielding a Fisher-weighted quadratic form and SAM penalizing sharpness through the domi nant Hessian eigenvalue. For negative log-likelihood objectives with exponential-family output dis tributions, this correspondence becomes especially transparent, since the Fisher and Gauss-Newton matrices coincide. We further show that the same local geometric perspective extends to input-space perturbations, where divergence-based regularization is defined through transformations of the in put. In this setting, the regularizer induces a pullback quadratic form on the input space, providing a more general perturbation framework than standard SAM while preserving the same local sensitivity interpretation. To validate the analysis empirically, we use the asymmetric α-skew Jensen-Shannon divergence (JSD) family as a controlled testbed. Its local curvature coefficient scales as α(1 − α) and is maximized at the symmetric point α = 1 2, which recovers the standard JSD. Loss-landscape visualizations in the input-perturbation regime show that stronger induced curvature penalization is associated with flatter local minima. Experiments on four benchmark datasets further demonstrate that both accuracy and negative log-likelihood are consistently best near this regime of maximal curvature penalization.