Skip to content

资源

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

A TensorFlow computation is described by a directed graph, which is composed of a set of nodes. The graph represents a dataflow computation, with extensions for allowing some kinds of nodes to maintain and update persistent state and for branching and looping control structures within the graph in a manner similar to Naiad [36]. Clients typically construct a computational graph using one of the supported front end languages (C++ or Python). An example fragment to construct and then execute a TensorFlow graph using the Python front end is shown in Figure 1, and the resulting computation graph in Figure 2.

上面段中的

with extensions for allowing some kinds of nodes to maintain and update persistent state

所指的其实就tf.Variable，其实所谓的**persistent state** 应该就是一个表示**参数**的值，**参数**所指的是需要模型进行学习的参数，比如weight和bias；正如上述所介绍的，tf.Variable对应的是node，即它是一个operation，正如tf.constant也是一个node，这个node的出边是它的值，所以tf.Variable这个node的出边就是它的值；

在这篇论文的Figure 2: Corresponding computation graph for Figure 1中，将b（tf.Variable）、W（tf.Variable）、X（tf.placeholder）都画出了node；

Graphs and Sessions

Executing v = tf.Variable(0) adds to the graph a tf.Operation that will store a writeable tensor value that persists between tf.Session.run calls. The tf.Variable object wraps this operation, and can be used like a tensor, which will read the current value of the stored value. The tf.Variable object also has methods such astf.Variable.assign and tf.Variable.assign_add that create tf.Operation objects that, when executed, update the stored value. (See Variables for more information about variables.)

其实这段话对于理解为什么使用 tf.Variable 来作为参数提供了一个非常好的解释，它有着如下的特性： tf.Variable can be used like a tensor，当它作为一个tensor来使用的时候，那么它就会读取the current value of the stored value，这样它就有值了，显然这种情况往往是出现在前馈的时候；如果从dataflow graph的角度来看的话，这种情况就相当于从node中流出tensor值；

The tf.Variable object also has methods such astf.Variable.assign and tf.Variable.assign_add that create tf.Operation objects that, when executed, update the stored value.显然这是在反馈的时候基于梯度来学习参数的时候要使用的；如果从dataflow graph的角度来看的话，这种情况就相当于tensor值流入到node中；

Variables

A TensorFlow variable is the best way to represent shared, persistent state manipulated by your program.

Variables are manipulated via the tf.Variable class. A tf.Variable represents a tensor whose value can be changed by running ops on it. Unlike tf.Tensor objects, a tf.Variable exists outside the context of a single session.run call.

Internally, a tf.Variable stores a persistent tensor. Specific ops allow you to read and modify the values of this tensor. These modifications are visible across multiple tf.Sessions, so multiple workers can see the same values for a tf.Variable.

***SUMMARY*** ： 显然tf.Varialbe的这种特性是非常适合来作为模型的参数，权重的；因为训练的过程就是不断地调整参数，并且训练的过程往往不是在一个run中就完成的，而是需要经过多次run，因此tf.Varaible的persist特性就非常重要了；

Tensors

Some types of tensors are special, and these will be covered in other units of the TensorFlow guide. The main ones are:

With the exception of tf.Variable, the value of a tensor is immutable, which means that in the context of a single execution tensors only have a single value. However, evaluating the same tensor twice can return different values; for example that tensor can be the result of reading data from disk, or generating a random number.

SUMMARY : 从computation graph的角度来看待：

它们都node， 但是它们都有一个出边，这个出边是它所维护的tensor的值；

SUMMARY

其实从不同的角度来对tf.Variable进行分析，会得出不同的结论；