LostTech.TensorFlow : API Documentation

Type LuongAttention

Namespace tensorflow.contrib.seq2seq

Parent _BaseAttentionMechanism

Interfaces ILuongAttention

Implements Luong-style (multiplicative) attention scoring.

This attention has two forms. The first is standard Luong attention, as described in:

Minh-Thang Luong, Hieu Pham, Christopher D. Manning. [Effective Approaches to Attention-based Neural Machine Translation. EMNLP 2015.](https://arxiv.org/abs/1508.04025)

The second is the scaled form inspired partly by the normalized form of Bahdanau attention.

To enable the second form, construct the object with parameter `scale=True`.

Methods

Properties

Public instance methods

object __call__(IEnumerable<IGraphNodeBase> query, IEnumerable<IGraphNodeBase> state)

Score the query based on the keys and values.
Parameters
IEnumerable<IGraphNodeBase> query
Tensor of dtype matching `self.values` and shape `[batch_size, query_depth]`.
IEnumerable<IGraphNodeBase> state
Tensor of dtype matching `self.values` and shape `[batch_size, alignments_size]` (`alignments_size` is memory's `max_time`).
Returns
object

object __call__(IEnumerable<IGraphNodeBase> query, object state)

Score the query based on the keys and values.
Parameters
IEnumerable<IGraphNodeBase> query
Tensor of dtype matching `self.values` and shape `[batch_size, query_depth]`.
object state
Tensor of dtype matching `self.values` and shape `[batch_size, alignments_size]` (`alignments_size` is memory's `max_time`).
Returns
object

object __call__(object query, IEnumerable<IGraphNodeBase> state)

Score the query based on the keys and values.
Parameters
object query
Tensor of dtype matching `self.values` and shape `[batch_size, query_depth]`.
IEnumerable<IGraphNodeBase> state
Tensor of dtype matching `self.values` and shape `[batch_size, alignments_size]` (`alignments_size` is memory's `max_time`).
Returns
object

Public static methods

LuongAttention NewDyn(object num_units, object memory, object memory_sequence_length, ImplicitContainer<T> scale, object probability_fn, object score_mask_value, object dtype, object custom_key_value_fn, ImplicitContainer<T> name)

Construct the AttentionMechanism mechanism.
Parameters
object num_units
The depth of the attention mechanism.
object memory
The memory to query; usually the output of an RNN encoder. This tensor should be shaped `[batch_size, max_time,...]`.
object memory_sequence_length
(optional) Sequence lengths for the batch entries in memory. If provided, the memory tensor rows are masked with zeros for values past the respective sequence lengths.
ImplicitContainer<T> scale
Python boolean. Whether to scale the energy term.
object probability_fn
(optional) A `callable`. Converts the score to probabilities. The default is tf.nn.softmax. Other options include tf.contrib.seq2seq.hardmax and tf.contrib.sparsemax.sparsemax. Its signature should be: `probabilities = probability_fn(score)`.
object score_mask_value
(optional) The mask value for score before passing into `probability_fn`. The default is -inf. Only used if `memory_sequence_length` is not None.
object dtype
The data type for the memory layer of the attention mechanism.
object custom_key_value_fn
(optional): The custom function for computing keys and values.
ImplicitContainer<T> name
Name to use when creating ops.

Public properties

object alignments_size get;

object alignments_size_dyn get;

object batch_size get;

object batch_size_dyn get;

object dtype get; set;

object keys get;

object keys_dyn get;

Dense memory_layer get;

object memory_layer_dyn get;

object PythonObject get;

Dense query_layer get;

object query_layer_dyn get;

object state_size get;

object state_size_dyn get;

object values get;

object values_dyn get;