LostTech.TensorFlow : API Documentation

Type BahdanauAttention

Namespace tensorflow.contrib.seq2seq

Parent _BaseAttentionMechanism

Interfaces IBahdanauAttention

Implements Bahdanau-style (additive) attention.

This attention has two forms. The first is Bahdanau attention, as described in:

Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio. "Neural Machine Translation by Jointly Learning to Align and Translate." ICLR 2015. https://arxiv.org/abs/1409.0473

The second is the normalized form. This form is inspired by the weight normalization article:

Tim Salimans, Diederik P. Kingma. "Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks." https://arxiv.org/abs/1602.07868

To enable the second form, construct the object with parameter `normalize=True`.

Properties

Public properties

object alignments_size get;

object alignments_size_dyn get;

object batch_size get;

object batch_size_dyn get;

object dtype get; set;

object keys get;

object keys_dyn get;

Dense memory_layer get;

object memory_layer_dyn get;

object PythonObject get;

Dense query_layer get;

object query_layer_dyn get;

object state_size get;

object state_size_dyn get;

object values get;

object values_dyn get;