The minimum increment of f-divergences given total variation distances

Abstract

Let (P_i, Q_i), i = 0, 1, be two pairs of probability measures defined on measurable spaces (Ω_i,F_i) respectively. Assume that the pair (P₁, Q₁) is more informative than (P₀,Q₀) for testing problems. This amounts to say that If (P₁,Q₁) ≥ If (P₀,Q₀), where If (·, ·) is an arbitrary f-divergence. We find a precise lower bound for the increment of f-divergences I_f(P₁,Q₁) − I_f(P₀,Q₀) provided that the total variation distances ||Q₁ − P₁|| and ||Q₀ − P₀|| are given. This optimization problem can be reduced to the case where P₁ and Q₁ are defined on the space consisting of four points, and P₀ and Q₀ are obtained from P₁ and Q₁ respectively by merging two of these four points. The result includes the well-known lower and upper bounds for I_f(P,Q) given ||Q − P||.