Hands on with TensorFlow

Google released a new open source machine learning library called TensorFlow. I was excited to try it out not only because it is released from Google, but also because it is being used in production at Google for various products like smart response and translation. I quickly went through the tutorials and documentation, and here are my initial thoughts.

TensorFlow framework

As with any good library, building a model with TensorFlow feels like building something out of Lego blocks. The building blocks are straightforward, and the properties and functions map well to how a scientist thinks about machine learning. In TensorFlow, you represent computations as graphs. The graph is composed of inputs which are tensors and operations which produce zero or more Tensors. The library provides facilities to build this graph from the bottom up, supporting many of the deep learning techniques used by machine learning researchers.

Going through the first example on the tutorial for MNIST digit recognition for example, the code to represent a Softmax Regression model looks like this:

y = tf.nn.softmax(tf.matmul(x,W) + b)

This model will be able to predict a digit between 0 and 9, given a handwritten image of that digit. You can refer to the tutorial itself for the meanings of the variables. Note that at this point the equation is an abstraction, there is no real data populated in there yet. It's just a representation of the model.

After writing the model, in order to train it, you write a cost function and optimize the model by minimizing the loss with respect to the actual labeled truth. Lastly you run a session to feed the data through the graph you created. All together the code to build and test this model is less than 30 lines of python.

I would say that from going through this tutorial the library is very well thought out, with the right tools in place like graph visualization. I definitely need to explore further the capabilities of TensorBoard, the graph visualization library that comes with TensorFlow.

Why Python?

I am a bit confused that Python is the language of choice for the initial interface of TensorFlow. By going with Python it seems to me that the library needs to account for type safety in a way that seems awkward to the language, like this: x = tf.placeholder("float", [None, 784]). Go seems to be a much better fit, given their emphasis on performance and type safety. Plus Go was created at Google, what better synergy is there.

What this means for researchers

With it's well thought out structure and ease of use, this looks like a library that researchers will be interested in for sure, however this is by no means a silver bullet to all of your machine learning headaches. For example, I mentioned earlier that the Softmax Regression code is less than 30 lines, but there is a huge caveat. Google kindly prepared a seperate class which is much bigger (approx. 200 LOC) that will pull down data from Yann LeCun's homepage and prepare the data for training. Using lines of code as a proxy, this means that about 80% of the work is still in data preparation and 20% is actual modelling. Also, one of the selling points of the library is the ability to leverage GPUs and train large models in production environments. Unless one of the big cloud providers like AWS start offering a template for this, it is still hard for individual researchers to get access to the infrastructure required to pull off these massive models.

What this means for Google

This library is a great move for Google in that:

  1. It positions Google as an innovator in the increasingly competitive field of deep learning
  2. As more researchers utilize the library in their research, Google will become a popular destination for the leading minds in this field
  3. Open sourcing it will lead to more contributions from the public

If you liked this post, follow us on Twitter for updates on future blog posts. Also, check out our new eBooklet, Data Science Industry in Plain English

Other articles and resources on TensorFlow