Sinkhorn: Entropic Regularization
Discrete Measures Definition: Shannon-Boltzmann entropy Let $P \in U(a, b)$ be a coupling matrix for discrete measures with vectors $a$ and $b$. Then, the Shannon-Boltzmann entropy is $$ H(P) = - \sum_{i, j} P_{i, j} \log{(P_{i, j})}, $$ where $0 = \log{0}$. Note that $$ \nabla^2 H(P) = - \diag(P_{i, j}^{-1}). $$ So, $H$ is strictly concave. Let us add a regularization term to the discrete Kantorovich problem. $\varepsilon$ will be our regularization weight and it works as a kind of “temperature”. ...