IntersectionRNNCell - LostTech.TensorFlow Documentation

Type IntersectionRNNCell

Namespace tensorflow.contrib.rnn

Parent RNNCell

Intersection Recurrent Neural Network (+RNN) cell.

Architecture with coupled recurrent gate as well as coupled depth gate, designed to improve information flow through stacked RNNs. As the architecture uses depth gating, the dimensionality of the depth output (y) also should not change through depth (input size == output size). To achieve this, the first layer of a stacked Intersection RNN projects the inputs to N (num units) dimensions. Therefore when initializing an IntersectionRNNCell, one should set `num_in_proj = N` for the first layer and use default settings for subsequent layers.

This implements the recurrent cell from the paper:

https://arxiv.org/abs/1611.09913

Jasmine Collins, Jascha Sohl-Dickstein, and David Sussillo. "Capacity and Trainability in Recurrent Neural Networks" Proc. ICLR 2017.

The Intersection RNN is built for use in deeply stacked RNNs so it may not achieve best performance with depth 1.

Methods

NewDyn

Properties

Public static methods

IntersectionRNNCell NewDyn(object num_units, object num_in_proj, object initializer, ImplicitContainer<T> forget_bias, ImplicitContainer<T> y_activation, object reuse)

Initialize the parameters for an +RNN cell.

Parameters

object num_units: int, The number of units in the +RNN cell
object num_in_proj: (optional) int, The input dimensionality for the RNN. If creating the first layer of an +RNN, this should be set to `num_units`. Otherwise, this should be set to `None` (default). If `None`, dimensionality of `inputs` should be equal to `num_units`, otherwise ValueError is thrown.
object initializer: (optional) The initializer to use for the weight matrices.
ImplicitContainer<T> forget_bias: (optional) float, default 1.0, The initial bias of the forget gates, used to reduce the scale of forgetting at the beginning of the training.
ImplicitContainer<T> y_activation: (optional) Activation function of the states passed through depth. Default is 'tf.nn.relu`.
object reuse: (optional) Python boolean describing whether to reuse variables in an existing scope. If not `True`, and the existing scope already has the given variables, an error is raised.

Public properties

PythonFunctionContainer activity_regularizer get; set;

object activity_regularizer_dyn get; set;

bool built get; set;

object dtype get;

object dtype_dyn get;

bool dynamic get;

object dynamic_dyn get;

object graph get;

object graph_dyn get;

IList<Node> inbound_nodes get;

object inbound_nodes_dyn get;

IList<object> input get;

object input_dyn get;

object input_mask get;

object input_mask_dyn get;

IList<object> input_shape get;

object input_shape_dyn get;

object input_spec get; set;

object input_spec_dyn get; set;

IList<object> losses get;

object losses_dyn get;

IList<object> metrics get;

object metrics_dyn get;

object name get;

object name_dyn get;

object name_scope get;

object name_scope_dyn get;

IList<object> non_trainable_variables get;

object non_trainable_variables_dyn get;

IList<object> non_trainable_weights get;

object non_trainable_weights_dyn get;

IList<object> outbound_nodes get;

object outbound_nodes_dyn get;

IList<object> output get;

object output_dyn get;

object output_mask get;

object output_mask_dyn get;

object output_shape get;

object output_shape_dyn get;

object output_size get;

Integer or TensorShape: size of outputs produced by this cell.

object output_size_dyn get;