LostTech.TensorFlow : API Documentation

Type CrossDeviceOps

Namespace tensorflow.distribute

Parent PythonObjectContainer

Interfaces ICrossDeviceOps

Public instance methods

IList<object> batch_reduce(ReduceOp reduce_op, object value_destination_pairs)

Reduce PerReplica objects in a batch.

Reduce each first element in `value_destination_pairs` to each second element which indicates the destinations.
Parameters
ReduceOp reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
object value_destination_pairs
a list or a tuple of tuples of PerReplica objects (or tensors with device set if there is one device) and destinations.
Returns
IList<object>
a list of Mirrored objects.

IList<object> batch_reduce(object reduce_op, object value_destination_pairs)

Reduce PerReplica objects in a batch.

Reduce each first element in `value_destination_pairs` to each second element which indicates the destinations.
Parameters
object reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
object value_destination_pairs
a list or a tuple of tuples of PerReplica objects (or tensors with device set if there is one device) and destinations.
Returns
IList<object>
a list of Mirrored objects.

object batch_reduce_dyn(object reduce_op, object value_destination_pairs)

Reduce PerReplica objects in a batch.

Reduce each first element in `value_destination_pairs` to each second element which indicates the destinations.
Parameters
object reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
object value_destination_pairs
a list or a tuple of tuples of PerReplica objects (or tensors with device set if there is one device) and destinations.
Returns
object
a list of Mirrored objects.

IList<object> batch_reduce_implementation(object reduce_op, IEnumerable<ValueTuple<PerReplica, object>> value_destination_pairs)

Implementation of reduce PerReplica objects in a batch.

Reduce each first element in `value_destination_pairs` to each second element which indicates the destinations.
Parameters
object reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
IEnumerable<ValueTuple<PerReplica, object>> value_destination_pairs
a list or a tuple of tuples of PerReplica objects (or tensors with device set if there is one device) and destinations.
Returns
IList<object>
a list of Mirrored objects.

IList<object> batch_reduce_implementation(object reduce_op, object value_destination_pairs)

Implementation of reduce PerReplica objects in a batch.

Reduce each first element in `value_destination_pairs` to each second element which indicates the destinations.
Parameters
object reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
object value_destination_pairs
a list or a tuple of tuples of PerReplica objects (or tensors with device set if there is one device) and destinations.
Returns
IList<object>
a list of Mirrored objects.

object batch_reduce_implementation_dyn(object reduce_op, object value_destination_pairs)

Implementation of reduce PerReplica objects in a batch.

Reduce each first element in `value_destination_pairs` to each second element which indicates the destinations.
Parameters
object reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
object value_destination_pairs
a list or a tuple of tuples of PerReplica objects (or tensors with device set if there is one device) and destinations.
Returns
object
a list of Mirrored objects.

object broadcast(ValueTuple<double, int> tensor, object destinations)

Broadcast the `tensor` to destinations.
Parameters
ValueTuple<double, int> tensor
the tensor to broadcast.
object destinations
the broadcast destinations.
Returns
object
a Mirrored object.

object broadcast(RaggedTensor tensor, object destinations)

Broadcast the `tensor` to destinations.
Parameters
RaggedTensor tensor
the tensor to broadcast.
object destinations
the broadcast destinations.
Returns
object
a Mirrored object.

object broadcast(IGraphNodeBase tensor, object destinations)

Broadcast the `tensor` to destinations.
Parameters
IGraphNodeBase tensor
the tensor to broadcast.
object destinations
the broadcast destinations.
Returns
object
a Mirrored object.

object broadcast_dyn(object tensor, object destinations)

Broadcast the `tensor` to destinations.
Parameters
object tensor
the tensor to broadcast.
object destinations
the broadcast destinations.
Returns
object
a Mirrored object.

object broadcast_implementation(ValueTuple<double, int> tensor, object destinations)

Implementation of broadcast the `tensor` to destinations.
Parameters
ValueTuple<double, int> tensor
the tensor to broadcast.
object destinations
the broadcast destinations.
Returns
object
a Mirrored object.

object broadcast_implementation(IGraphNodeBase tensor, object destinations)

Implementation of broadcast the `tensor` to destinations.
Parameters
IGraphNodeBase tensor
the tensor to broadcast.
object destinations
the broadcast destinations.
Returns
object
a Mirrored object.

object broadcast_implementation_dyn(object tensor, object destinations)

Implementation of broadcast the `tensor` to destinations.
Parameters
object tensor
the tensor to broadcast.
object destinations
the broadcast destinations.
Returns
object
a Mirrored object.

object reduce(ReduceOp reduce_op, IEnumerable<object> per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
ReduceOp reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
IEnumerable<object> per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(string reduce_op, IGraphNodeBase per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
string reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
IGraphNodeBase per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(string reduce_op, PerReplica per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
string reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
PerReplica per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(string reduce_op, ValueTuple<PythonClassContainer, PythonClassContainer> per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
string reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
ValueTuple<PythonClassContainer, PythonClassContainer> per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(string reduce_op, IEnumerable<object> per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
string reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
IEnumerable<object> per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(string reduce_op, IDictionary<object, object> per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
string reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
IDictionary<object, object> per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(object reduce_op, object per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
object reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
object per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(object reduce_op, IGraphNodeBase per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
object reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
IGraphNodeBase per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(object reduce_op, PerReplica per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
object reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
PerReplica per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(object reduce_op, ValueTuple<PythonClassContainer, PythonClassContainer> per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
object reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
ValueTuple<PythonClassContainer, PythonClassContainer> per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(object reduce_op, IEnumerable<object> per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
object reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
IEnumerable<object> per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(object reduce_op, IDictionary<object, object> per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
object reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
IDictionary<object, object> per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(ReduceOp reduce_op, object per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
ReduceOp reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
object per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(ReduceOp reduce_op, IGraphNodeBase per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
ReduceOp reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
IGraphNodeBase per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(ReduceOp reduce_op, PerReplica per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
ReduceOp reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
PerReplica per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(string reduce_op, object per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
string reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
object per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(ReduceOp reduce_op, ValueTuple<PythonClassContainer, PythonClassContainer> per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
ReduceOp reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
ValueTuple<PythonClassContainer, PythonClassContainer> per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce(ReduceOp reduce_op, IDictionary<object, object> per_replica_value, string destinations)

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
ReduceOp reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
IDictionary<object, object> per_replica_value
a PerReplica object or a tensor with device set.
string destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce_dyn(object reduce_op, object per_replica_value, object destinations)

Reduce `value` across replicas.

Given a per-replica value returned by `experimental_run_v2`, say a per-example loss, the batch will be divided across all the replicas. This function allows you to aggregate across replicas and optionally also across batch elements. For example, if you have a global batch size of 8 and 2 replicas, values for examples `[0, 1, 2, 3]` will be on replica 0 and `[4, 5, 6, 7]` will be on replica 1. By default, `reduce` will just aggregate across replicas, returning `[0+4, 1+5, 2+6, 3+7]`. This is useful when each replica is computing a scalar or some other value that doesn't have a "batch" dimension (like a gradient). More often you will want to aggregate across the global batch, which you can get by specifying the batch dimension as the `axis`, typically `axis=0`. In this case it would return a scalar `0+1+2+3+4+5+6+7`.

If there is a last partial batch, you will need to specify an axis so that the resulting shape is consistent across replicas. So if the last batch has size 6 and it is divided into [0, 1, 2, 3] and [4, 5], you would get a shape mismatch unless you specify `axis=0`. If you specify tf.distribute.ReduceOp.MEAN, using `axis=0` will use the correct denominator of 6. Contrast this with computing `reduce_mean` to get a scalar value on each replica and this function to average those means, which will weigh some values `1/8` and others `1/4`.
Parameters
object reduce_op
A tf.distribute.ReduceOp value specifying how values should be combined.
object per_replica_value
object destinations
Returns
object
A `Tensor`.

object reduce_implementation(object reduce_op, PerReplica per_replica_value, object destinations)

The implementation of reduce of `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
object reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
PerReplica per_replica_value
a PerReplica object or a tensor with device set.
object destinations
the reduction destinations.
Returns
object
a Mirrored object.

object reduce_implementation_dyn(object reduce_op, object per_replica_value, object destinations)

The implementation of reduce of `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the result on `destinations`.
Parameters
object reduce_op
Indicates how per_replica_value will be reduced. Accepted values are tf.distribute.ReduceOp.SUM, tf.distribute.ReduceOp.MEAN.
object per_replica_value
a PerReplica object or a tensor with device set.
object destinations
the reduction destinations.
Returns
object
a Mirrored object.

Public properties

object PythonObject get;