Denote this distribution by p and the other distribution by q. Assuming p, q are absolutely continuous with respect to reference measure r, the KL divergence is defined as: KL[p, q] = E_p[log(p(X)/q(X))] = -int_F p(x) log q(x) dr(x) + int_F p(x) log p(x) dr(x) = H[p, q] - H[p] where F denotes the support of the random variable X ~ p, H[., .] denotes (Shannon) cross entropy, and H[.] denotes (Shannon) entropy.

tfd_kl_divergence(distribution, other, name = "kl_divergence")



The distribution being used.


tfp$distributions$Distribution instance.


String prepended to names of ops created by this function.


self$dtype Tensor with shape [B1, ..., Bn] representing n different calculations of the Kullback-Leibler divergence.

See also


# \donttest{ d1 <- tfd_normal(loc = c(1, 2), scale = c(1, 0.5)) d2 <- tfd_normal(loc = c(1.5, 2), scale = c(1, 0.5)) d1 %>% tfd_kl_divergence(d2)
#> tf.Tensor([0.125 0. ], shape=(2,), dtype=float32)
# }