LostTech.TensorFlow : API Documentation

Set a tf.distribute.Strategy as current without `with strategy.scope()`.

``` tf.distribute.experimental_set_strategy(strategy1) f() tf.distribute.experimental_set_strategy(strategy2) g() tf.distribute.experimental_set_strategy(None) h() ```

is equivalent to:

``` with strategy1.scope(): f() with strategy2.scope(): g() h() ```

In general, you should use the `with strategy.scope():` API, but this alternative may be convenient in notebooks where you would have to put each cell in a `with strategy.scope():` block.

Note: This should only be called outside of any TensorFlow scope to avoid improper nesting.

Set a tf.distribute.Strategy as current without `with strategy.scope()`.

``` tf.distribute.experimental_set_strategy(strategy1) f() tf.distribute.experimental_set_strategy(strategy2) g() tf.distribute.experimental_set_strategy(None) h() ```

is equivalent to:

``` with strategy1.scope(): f() with strategy2.scope(): g() h() ```

In general, you should use the `with strategy.scope():` API, but this alternative may be convenient in notebooks where you would have to put each cell in a `with strategy.scope():` block.

Note: This should only be called outside of any TensorFlow scope to avoid improper nesting.

tf.distribute.ReduceOp corresponding to the last loss reduction.

This is used to decide whether loss should be scaled in optimizer (used only for estimator + v1 optimizer use case).

tf.distribute.ReduceOp corresponding to the last loss reduction.

This is used to decide whether loss should be scaled in optimizer (used only for estimator + v1 optimizer use case).

Returns the current tf.distribute.ReplicaContext or `None`.

Returns `None` if in a cross-replica context.

Note that execution:

1. starts in the default (single-replica) replica context (this function will return the default `ReplicaContext` object); 2. switches to cross-replica context (in which case this will return `None`) when entering a `with tf.distribute.Strategy.scope():` block; 3. switches to a (non-default) replica context inside `strategy.experimental_run_v2(fn,...)`; 4. if `fn` calls `get_replica_context().merge_call(merge_fn,...)`, then inside `merge_fn` you are back in the cross-replica context (and again this function will return `None`).

Most tf.distribute.Strategy methods may only be executed in a cross-replica context, in a replica context you should use the API of the tf.distribute.ReplicaContext object returned by this method instead.

``` assert tf.distribute.get_replica_context() is not None # default with strategy.scope(): assert tf.distribute.get_replica_context() is None

def f(): replica_context = tf.distribute.get_replica_context() # for strategy assert replica_context is not None tf.print("Replica id: ", replica_context.replica_id_in_sync_group, " of ", replica_context.num_replicas_in_sync)

strategy.experimental_run_v2(f) ```

Returns the current tf.distribute.ReplicaContext or `None`.

Returns `None` if in a cross-replica context.

Note that execution:

1. starts in the default (single-replica) replica context (this function will return the default `ReplicaContext` object); 2. switches to cross-replica context (in which case this will return `None`) when entering a `with tf.distribute.Strategy.scope():` block; 3. switches to a (non-default) replica context inside `strategy.experimental_run_v2(fn,...)`; 4. if `fn` calls `get_replica_context().merge_call(merge_fn,...)`, then inside `merge_fn` you are back in the cross-replica context (and again this function will return `None`).

Most tf.distribute.Strategy methods may only be executed in a cross-replica context, in a replica context you should use the API of the tf.distribute.ReplicaContext object returned by this method instead.

``` assert tf.distribute.get_replica_context() is not None # default with strategy.scope(): assert tf.distribute.get_replica_context() is None

def f(): replica_context = tf.distribute.get_replica_context() # for strategy assert replica_context is not None tf.print("Replica id: ", replica_context.replica_id_in_sync_group, " of ", replica_context.num_replicas_in_sync)

strategy.experimental_run_v2(f) ```

Returns the current tf.distribute.Strategy object.

Typically only used in a cross-replica context:

``` if tf.distribute.in_cross_replica_context(): strategy = tf.distribute.get_strategy() ... ```

Returns the current tf.distribute.Strategy object.

Typically only used in a cross-replica context:

``` if tf.distribute.in_cross_replica_context(): strategy = tf.distribute.get_strategy() ... ```

Return if there is a current non-default tf.distribute.Strategy.

``` assert not tf.distribute.has_strategy() with strategy.scope(): assert tf.distribute.has_strategy() ```

Return if there is a current non-default tf.distribute.Strategy.

``` assert not tf.distribute.has_strategy() with strategy.scope(): assert tf.distribute.has_strategy() ```

Returns `True` if in a cross-replica context.

See tf.distribute.get_replica_context for details.

``` assert not tf.distribute.in_cross_replica_context() with strategy.scope(): assert tf.distribute.in_cross_replica_context()

def f(): assert not tf.distribute.in_cross_replica_context()

strategy.experimental_run_v2(f) ```

Returns `True` if in a cross-replica context.

See tf.distribute.get_replica_context for details.

``` assert not tf.distribute.in_cross_replica_context() with strategy.scope(): assert tf.distribute.in_cross_replica_context()

def f(): assert not tf.distribute.in_cross_replica_context()

strategy.experimental_run_v2(f) ```