IndyLSTMCell - LostTech.TensorFlow Documentation

Type IndyLSTMCell

Namespace tensorflow.contrib.rnn

Basic IndyLSTM recurrent network cell.

Based on IndRNNs (https://arxiv.org/abs/1803.04831) and similar to BasicLSTMCell, yet with the \$U_f\$, \$U_i\$, \$U_o\$ and \$U_c\$ matrices in the regular LSTM equations replaced by diagonal matrices, i.e. a Hadamard product with a single vector:

$$f_t = \sigma_g\left(W_f x_t + u_f \circ h_{t-1} + b_f\right)$$ $$i_t = \sigma_g\left(W_i x_t + u_i \circ h_{t-1} + b_i\right)$$ $$o_t = \sigma_g\left(W_o x_t + u_o \circ h_{t-1} + b_o\right)$$ $$c_t = f_t \circ c_{t-1} + i_t \circ \sigma_c\left(W_c x_t + u_c \circ h_{t-1} + b_c\right)$$

where \$\circ\$ denotes the Hadamard operator. This means that each IndyLSTM node sees only its own state \$h\$ and \$c\$, as opposed to seeing all states in the same layer.

We add forget_bias (default: 1) to the biases of the forget gate in order to reduce the scale of forgetting in the beginning of the training.

It does not allow cell clipping, a projection layer, and does not use peep-hole connections: it is the basic baseline.

For a detailed analysis of IndyLSTMs, see https://arxiv.org/abs/1903.08023.

Methods

NewDyn

Properties

Public static methods

IndyLSTMCell NewDyn(object num_units, ImplicitContainer<T> forget_bias, object activation, object reuse, object kernel_initializer, object bias_initializer, object name, object dtype)

Initialize the IndyLSTM cell.

Parameters

object num_units: int, The number of units in the LSTM cell.
ImplicitContainer<T> forget_bias: float, The bias added to forget gates (see above). Must set to `0.0` manually when restoring from CudnnLSTM-trained checkpoints.
object activation: Activation function of the inner states. Default: `tanh`.
object reuse: (optional) Python boolean describing whether to reuse variables in an existing scope. If not `True`, and the existing scope already has the given variables, an error is raised.
object kernel_initializer: (optional) The initializer to use for the weight matrix applied to the inputs.
object bias_initializer: (optional) The initializer to use for the bias.
object name: String, the name of the layer. Layers with the same name will share weights, but to avoid mistakes we require reuse=True in such cases.
object dtype: Default dtype of the layer (default of `None` means use the type of the first input). Required when `build` is called before `call`.

Public properties

PythonFunctionContainer activity_regularizer get; set;

object activity_regularizer_dyn get; set;

bool built get; set;

object dtype get;

object dtype_dyn get;

bool dynamic get;

object dynamic_dyn get;

object graph get;

object graph_dyn get;

IList<Node> inbound_nodes get;

object inbound_nodes_dyn get;

IList<object> input get;

object input_dyn get;

object input_mask get;

object input_mask_dyn get;

IList<object> input_shape get;

object input_shape_dyn get;

InputSpec input_spec get; set;

object input_spec_dyn get; set;

IList<object> losses get;

object losses_dyn get;

IList<object> metrics get;

object metrics_dyn get;

object name get;

object name_dyn get;

object name_scope get;

object name_scope_dyn get;

IList<object> non_trainable_variables get;

object non_trainable_variables_dyn get;

IList<object> non_trainable_weights get;

object non_trainable_weights_dyn get;

IList<object> outbound_nodes get;

object outbound_nodes_dyn get;

IList<object> output get;

object output_dyn get;

object output_mask get;

object output_mask_dyn get;

object output_shape get;

object output_shape_dyn get;

object output_size get;

Integer or TensorShape: size of outputs produced by this cell.

object output_size_dyn get;