LostTech.TensorFlow : API Documentation

Type MetricSpec

Namespace tensorflow.contrib.learn

Parent PythonObjectContainer

Interfaces IMetricSpec

MetricSpec connects a model to metric functions.

THIS CLASS IS DEPRECATED. See [contrib/learn/README.md](https://www.tensorflow.org/code/tensorflow/contrib/learn/README.md) for general migration instructions.

The MetricSpec class contains all information necessary to connect the output of a `model_fn` to the metrics (usually, streaming metrics) that are used in evaluation.

It is passed in the `metrics` argument of `Estimator.evaluate`. The `Estimator` then knows which predictions, labels, and weight to use to call a given metric function.

When building the ops to run in evaluation, an `Estimator` will call `create_metric_ops`, which will connect the given `metric_fn` to the model as detailed in the docstring for `create_metric_ops`, and return the metric.

Example:

Assuming a model has an input function which returns inputs containing (among other things) a tensor with key "input_key", and a labels dictionary containing "label_key". Let's assume that the `model_fn` for this model returns a prediction with key "prediction_key".

In order to compute the accuracy of the "prediction_key" prediction, we would add

``` "prediction accuracy": MetricSpec(metric_fn=prediction_accuracy_fn, prediction_key="prediction_key", label_key="label_key") ```

to the metrics argument to `evaluate`. `prediction_accuracy_fn` can be either a predefined function in metric_ops (e.g., `streaming_accuracy`) or a custom function you define.

If we would like the accuracy to be weighted by "input_key", we can add that as the `weight_key` argument.

``` "prediction accuracy": MetricSpec(metric_fn=prediction_accuracy_fn, prediction_key="prediction_key", label_key="label_key", weight_key="input_key") ```

An end-to-end example is as follows:

``` estimator = tf.contrib.learn.Estimator(...) estimator.fit(...) _ = estimator.evaluate( input_fn=input_fn, steps=1, metrics={ 'prediction accuracy': metric_spec.MetricSpec( metric_fn=prediction_accuracy_fn, prediction_key="prediction_key", label_key="label_key") }) ```

Methods

Properties

Public instance methods

object create_metric_ops(IDictionary<string, string> inputs, IDictionary<object, object> labels, IDictionary<object, object> predictions)

Connect our `metric_fn` to the specified members of the given dicts.

This function will call the `metric_fn` given in our constructor as follows:

``` metric_fn(predictions[self.prediction_key], labels[self.label_key], weights=weights[self.weight_key]) ```

And returns the result. The `weights` argument is only passed if `self.weight_key` is not `None`.

`predictions` and `labels` may be single tensors as well as dicts. If `predictions` is a single tensor, `self.prediction_key` must be `None`. If `predictions` is a single element dict, `self.prediction_key` is allowed to be `None`. Conversely, if `labels` is a single tensor, `self.label_key` must be `None`. If `labels` is a single element dict, `self.label_key` is allowed to be `None`.
Parameters
IDictionary<string, string> inputs
A dict of inputs produced by the `input_fn`
IDictionary<object, object> labels
A dict of labels or a single label tensor produced by the `input_fn`.
IDictionary<object, object> predictions
A dict of predictions or a single tensor produced by the `model_fn`.
Returns
object
The result of calling `metric_fn`.

object create_metric_ops(IDictionary<string, string> inputs, IDictionary<object, object> labels, string predictions)

Connect our `metric_fn` to the specified members of the given dicts.

This function will call the `metric_fn` given in our constructor as follows:

``` metric_fn(predictions[self.prediction_key], labels[self.label_key], weights=weights[self.weight_key]) ```

And returns the result. The `weights` argument is only passed if `self.weight_key` is not `None`.

`predictions` and `labels` may be single tensors as well as dicts. If `predictions` is a single tensor, `self.prediction_key` must be `None`. If `predictions` is a single element dict, `self.prediction_key` is allowed to be `None`. Conversely, if `labels` is a single tensor, `self.label_key` must be `None`. If `labels` is a single element dict, `self.label_key` is allowed to be `None`.
Parameters
IDictionary<string, string> inputs
A dict of inputs produced by the `input_fn`
IDictionary<object, object> labels
A dict of labels or a single label tensor produced by the `input_fn`.
string predictions
A dict of predictions or a single tensor produced by the `model_fn`.
Returns
object
The result of calling `metric_fn`.

object create_metric_ops(IDictionary<string, string> inputs, string labels, IDictionary<object, object> predictions)

Connect our `metric_fn` to the specified members of the given dicts.

This function will call the `metric_fn` given in our constructor as follows:

``` metric_fn(predictions[self.prediction_key], labels[self.label_key], weights=weights[self.weight_key]) ```

And returns the result. The `weights` argument is only passed if `self.weight_key` is not `None`.

`predictions` and `labels` may be single tensors as well as dicts. If `predictions` is a single tensor, `self.prediction_key` must be `None`. If `predictions` is a single element dict, `self.prediction_key` is allowed to be `None`. Conversely, if `labels` is a single tensor, `self.label_key` must be `None`. If `labels` is a single element dict, `self.label_key` is allowed to be `None`.
Parameters
IDictionary<string, string> inputs
A dict of inputs produced by the `input_fn`
string labels
A dict of labels or a single label tensor produced by the `input_fn`.
IDictionary<object, object> predictions
A dict of predictions or a single tensor produced by the `model_fn`.
Returns
object
The result of calling `metric_fn`.

object create_metric_ops(IDictionary<string, string> inputs, string labels, string predictions)

Connect our `metric_fn` to the specified members of the given dicts.

This function will call the `metric_fn` given in our constructor as follows:

``` metric_fn(predictions[self.prediction_key], labels[self.label_key], weights=weights[self.weight_key]) ```

And returns the result. The `weights` argument is only passed if `self.weight_key` is not `None`.

`predictions` and `labels` may be single tensors as well as dicts. If `predictions` is a single tensor, `self.prediction_key` must be `None`. If `predictions` is a single element dict, `self.prediction_key` is allowed to be `None`. Conversely, if `labels` is a single tensor, `self.label_key` must be `None`. If `labels` is a single element dict, `self.label_key` is allowed to be `None`.
Parameters
IDictionary<string, string> inputs
A dict of inputs produced by the `input_fn`
string labels
A dict of labels or a single label tensor produced by the `input_fn`.
string predictions
A dict of predictions or a single tensor produced by the `model_fn`.
Returns
object
The result of calling `metric_fn`.

object create_metric_ops_dyn(object inputs, object labels, object predictions)

Connect our `metric_fn` to the specified members of the given dicts.

This function will call the `metric_fn` given in our constructor as follows:

``` metric_fn(predictions[self.prediction_key], labels[self.label_key], weights=weights[self.weight_key]) ```

And returns the result. The `weights` argument is only passed if `self.weight_key` is not `None`.

`predictions` and `labels` may be single tensors as well as dicts. If `predictions` is a single tensor, `self.prediction_key` must be `None`. If `predictions` is a single element dict, `self.prediction_key` is allowed to be `None`. Conversely, if `labels` is a single tensor, `self.label_key` must be `None`. If `labels` is a single element dict, `self.label_key` is allowed to be `None`.
Parameters
object inputs
A dict of inputs produced by the `input_fn`
object labels
A dict of labels or a single label tensor produced by the `input_fn`.
object predictions
A dict of predictions or a single tensor produced by the `model_fn`.
Returns
object
The result of calling `metric_fn`.

Public properties

string label_key get;

object label_key_dyn get;

object metric_fn get;

object metric_fn_dyn get;

Metric function.

This function accepts named args: `predictions`, `labels`, `weights`. It returns a single `Tensor` or `(value_op, update_op)` pair. See `metric_fn` constructor argument for more details.

string prediction_key get;

object prediction_key_dyn get;

object PythonObject get;

string weight_key get;

object weight_key_dyn get;