ExpRelaxedOneHotCategorical - LostTech.TensorFlow Documentation

Type ExpRelaxedOneHotCategorical

Namespace tensorflow.contrib.distributions

Interfaces IExpRelaxedOneHotCategorical

ExpRelaxedOneHotCategorical distribution with temperature and logits.

An ExpRelaxedOneHotCategorical distribution is a log-transformed RelaxedOneHotCategorical distribution. The RelaxedOneHotCategorical is a distribution over random probability vectors, vectors of positive real values that sum to one, which continuously approximates a OneHotCategorical. The degree of approximation is controlled by a temperature: as the temperature goes to 0 the RelaxedOneHotCategorical becomes discrete with a distribution described by the logits, as the temperature goes to infinity the RelaxedOneHotCategorical becomes the constant distribution that is identically the constant vector of (1/event_size,..., 1/event_size).

Because computing log-probabilities of the RelaxedOneHotCategorical can suffer from underflow issues, this class is one solution for loss functions that depend on log-probabilities, such as the KL Divergence found in the variational autoencoder loss. The KL divergence between two distributions is invariant under invertible transformations, so evaluating KL divergences of ExpRelaxedOneHotCategorical samples, which are always followed by a tf.exp op, is equivalent to evaluating KL divergences of RelaxedOneHotCategorical samples. See the appendix of Maddison et al., 2016 for more mathematical details, where this distribution is called the ExpConcrete.

#### Examples

Creates a continuous distribution, whose exp approximates a 3-class one-hot categorical distribution. The 2nd class is the most likely to be the largest component in samples drawn from this distribution. If those samples are followed by a tf.exp op, then they are distributed as a relaxed onehot categorical. Creates a continuous distribution, whose exp approximates a 3-class one-hot categorical distribution. The 2nd class is the most likely to be the largest component in samples drawn from this distribution. Creates a continuous distribution, whose exp approximates a 3-class one-hot categorical distribution. Because the temperature is very low, samples from this distribution are almost discrete, with one component almost 0 and the others very negative. The 2nd class is the most likely to be the largest component in samples drawn from this distribution. Creates a continuous distribution, whose exp approximates a 3-class one-hot categorical distribution. Because the temperature is very high, samples from this distribution are usually close to the (-log(3), -log(3), -log(3)) vector. The 2nd class is still the most likely to be the largest component in samples drawn from this distribution. Chris J. Maddison, Andriy Mnih, and Yee Whye Teh. The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables. 2016.

Show Example

temperature = 0.5
            p = [0.1, 0.5, 0.4]
            dist = ExpRelaxedOneHotCategorical(temperature, probs=p)
            samples = dist.sample()
            exp_samples = tf.exp(samples)
            # exp_samples has the same distribution as samples from
            # RelaxedOneHotCategorical(temperature, probs=p)

Properties

Public properties

object allow_nan_stats get;

object allow_nan_stats_dyn get;

TensorShape batch_shape get;

object batch_shape_dyn get;

object dtype get;

object dtype_dyn get;

TensorShape event_shape get;

object event_shape_dyn get;

Tensor event_size get;

Scalar `int32` tensor: the number of classes.

object event_size_dyn get;

Scalar `int32` tensor: the number of classes.

object logits get;

Vector of coordinatewise logits.

object logits_dyn get;

Vector of coordinatewise logits.

string name get;

object name_dyn get;

IDictionary<object, object> parameters get;

object parameters_dyn get;

Tensor probs get;

Vector of probabilities summing to one.

object probs_dyn get;