LostTech.TensorFlow : API Documentation

Type IndyLSTMCell

Namespace tensorflow.contrib.rnn

Parent LayerRNNCell

Interfaces IIndyLSTMCell

Basic IndyLSTM recurrent network cell.

Based on IndRNNs (https://arxiv.org/abs/1803.04831) and similar to BasicLSTMCell, yet with the \\(U_f\\), \\(U_i\\), \\(U_o\\) and \\(U_c\\) matrices in the regular LSTM equations replaced by diagonal matrices, i.e. a Hadamard product with a single vector:

$$f_t = \sigma_g\left(W_f x_t + u_f \circ h_{t-1} + b_f\right)$$ $$i_t = \sigma_g\left(W_i x_t + u_i \circ h_{t-1} + b_i\right)$$ $$o_t = \sigma_g\left(W_o x_t + u_o \circ h_{t-1} + b_o\right)$$ $$c_t = f_t \circ c_{t-1} + i_t \circ \sigma_c\left(W_c x_t + u_c \circ h_{t-1} + b_c\right)$$

where \\(\circ\\) denotes the Hadamard operator. This means that each IndyLSTM node sees only its own state \\(h\\) and \\(c\\), as opposed to seeing all states in the same layer.

We add forget_bias (default: 1) to the biases of the forget gate in order to reduce the scale of forgetting in the beginning of the training.

It does not allow cell clipping, a projection layer, and does not use peep-hole connections: it is the basic baseline.

For a detailed analysis of IndyLSTMs, see https://arxiv.org/abs/1903.08023.



Public static methods

IndyLSTMCell NewDyn(object num_units, ImplicitContainer<T> forget_bias, object activation, object reuse, object kernel_initializer, object bias_initializer, object name, object dtype)

Initialize the IndyLSTM cell.
object num_units
int, The number of units in the LSTM cell.
ImplicitContainer<T> forget_bias
float, The bias added to forget gates (see above). Must set to `0.0` manually when restoring from CudnnLSTM-trained checkpoints.
object activation
Activation function of the inner states. Default: `tanh`.
object reuse
(optional) Python boolean describing whether to reuse variables in an existing scope. If not `True`, and the existing scope already has the given variables, an error is raised.
object kernel_initializer
(optional) The initializer to use for the weight matrix applied to the inputs.
object bias_initializer
(optional) The initializer to use for the bias.
object name
String, the name of the layer. Layers with the same name will share weights, but to avoid mistakes we require reuse=True in such cases.
object dtype
Default dtype of the layer (default of `None` means use the type of the first input). Required when `build` is called before `call`.

Public properties

PythonFunctionContainer activity_regularizer get; set;

object activity_regularizer_dyn get; set;

bool built get; set;

object dtype get;

object dtype_dyn get;

bool dynamic get;

object dynamic_dyn get;

object graph get;

object graph_dyn get;

IList<Node> inbound_nodes get;

object inbound_nodes_dyn get;

IList<object> input get;

object input_dyn get;

object input_mask get;

object input_mask_dyn get;

IList<object> input_shape get;

object input_shape_dyn get;

InputSpec input_spec get; set;

object input_spec_dyn get; set;

IList<object> losses get;

object losses_dyn get;

IList<object> metrics get;

object metrics_dyn get;

object name get;

object name_dyn get;

object name_scope get;

object name_scope_dyn get;

IList<object> non_trainable_variables get;

object non_trainable_variables_dyn get;

IList<object> non_trainable_weights get;

object non_trainable_weights_dyn get;

IList<object> outbound_nodes get;

object outbound_nodes_dyn get;

IList<object> output get;

object output_dyn get;

object output_mask get;

object output_mask_dyn get;

object output_shape get;

object output_shape_dyn get;

object output_size get;

Integer or TensorShape: size of outputs produced by this cell.

object output_size_dyn get;

Integer or TensorShape: size of outputs produced by this cell.

object PythonObject get;

object rnncell_scope get; set;

string scope_name get;

object scope_name_dyn get;

object state_size get;

size(s) of state(s) used by this cell.

It can be represented by an Integer, a TensorShape or a tuple of Integers or TensorShapes.

object state_size_dyn get;

size(s) of state(s) used by this cell.

It can be represented by an Integer, a TensorShape or a tuple of Integers or TensorShapes.

bool stateful get; set;

ValueTuple<object> submodules get;

object submodules_dyn get;

bool supports_masking get; set;

bool trainable get; set;

object trainable_dyn get; set;

object trainable_variables get;

object trainable_variables_dyn get;

IList<object> trainable_weights get;

object trainable_weights_dyn get;

IList<object> updates get;

object updates_dyn get;

object variables get;

object variables_dyn get;

IList<object> weights get;

object weights_dyn get;