The Bretagnolle-Huber Inequality provides a bound on the total variation distance between two probability distributions in terms of their Kullback-Leibler divergence, and it is better than Pinsker’s Inequality when the KL divergence is larger than two, and it is never vacuous, as shown in this figure from [Canonne2023]. The following is an elementary proof, which … More Notes on the Bretagnolle-Huber Inequality