Recently, Deep Learning has been getting a lot of attention. We are trying to find new applications for deep learning and we are building models of unforeseen magnitude. For this purpose, we use various libraries and tools which help us in building these frameworks. This all has made DL little complicate, hence this article will acts as guide to Theano for DL.
Theano is one such tool, more accurately, it's a mathematical computation library. It has been around since 2008 and has been used widely in development of some of the most complex model architectures for neural networks.
Today, we will discuss Theano, what is it, how does it work and how can you build your own neural network using Theano. We will also perform a brief comparison of Theano with its contemporary deep learning libraries and tools. In this beginner's guide to Theano, you will learn how it is impacting Deep Learning.
Content:
- What is Theano
- Prerequisites for using Theano
- Theano with Python
- Theano with R
- How does Theano Work?
- Theano Vs. Others
- Step by Step Guide to Build a project in Theano
- Python Refresher
- Step 1: Set the Stage
- Step 2:Extensions and Wrappers for Theano
- Step 3: Project Using Theano
- Theano Cheat sheet and Sources
- Endnote
What is Theano?
Theano is a numerical computation library that allows you to define optimize and evaluate mathematical expressions involving multi-dimensional arrays efficiently. It was developed for Python.
It runs efficiently on both GPU or CPU. The computations in Theano are expressed in a format similar to NumPy, the numerical library for Python.
Theano is an open source project primarily developed by a machine learning group at Universite De Montreal.
Generally called the Granddaddy of machine learning and deep learning, Theano will no longer receive any major upgrades as Pascal Lamblin, announced on September 28, 2017 that there will be no further development of the framework due to competing of offerings by strong industrial players. Theano is an open source project, hence it faced lot of competition.
This has resulted in migration of programmers from Theano to Tensorflow.
Major Features of Theano:
- Strong integration with NumPy
- While using a GPU, Theano performs data intensive computations much faster than on a CPU.
- Theano implements efficient symbolic differentiation by doing your derivatives for functions with one or many inputs.
- Even for a very minute value of variables Theano performs consistently. This results in speed and stability optimization.
- Theano generates dynamic C code which helps in evaluating expressions faster.
- Theano also implements extensive unit testing and self verification which results in detection and diagnosis of many types of errors.
Theano is equipped to perform large-scale computationally intensive scientific investigation it is also available for usage in classrooms.
Prerequisites for Theano:
Theano is part framework and part library for evaluating and optimising mathematical expressions it is it same to the fact that it enables you to build optimised symbolic computational graphs and that the gradients can be automatically computed.
However, before we move to the topic next in our beginner's guide to Theano, you need to have a good grasp of machine learning concepts, deep learning concepts, intermediate to advanced programming skills with Python and a good grasp of basic mathematical concepts like probability, basic linear algebra, calculus, basics of statistics, etc.
You should also be able to write at least simple neural networks in Python and train it with gradient Descent or backpropagation. It also helps if you have read the Theano documentation.
While there is not much support available for OpenCL practically, Theano supports almost all other major operating systems and GNUs.
Theano with Python
Theano was basically developed as a computational library come Framework for python and it does justice to the idea.
Before tensorflow Theano was the only framework of choice for developing deep neural networks with Python owing to its simple to use and understand features. Even after TensorFlow was released, Theano has a thriving community and ecosystem.
Theano with R
R derived its importance from its unique ability to produce high-quality data analytics and graphics. There was hardly any efficient tool for developing data visualisations on the size of data sets used for training deep neural networks.
Theano compliments R by adding a layer of abstraction on it. This helps to keep the code shorter as compared to regular R programs, which can be pretty long and complicated, considering R is a low level programming language.
However as better libraries are being produced for python to generate high quality data visualisations and corresponding graphics, there is a reduction in motivation to use R.
How does Theano work?
Theano handles mathematical expressions very efficiently including the ones which have multi dimensional arrays. These errors are also called tensors.
What does Theano bring to the table?
- Automatic differentiation: Theano automatically figured out how to calculate the gradients at various points this allows you to perform gradient Descent for model training. What it further means is that you only have to implement the prediction part of the model.
- Transparent use of a GPU: this could be called a signature benefit of using Theano. This library will not only figure out which part of the computation should be moved to the GPU, it will also let you run the same code on either a CPU or GPU.
- Speed and stability Optimisation: we have discussed this earlier as well, Theano internally reorganizes and optimizes your computations. This makes the computation run faster and be more stable numerically. Theano also tries to compile some operations is no C code so that it can speed up the computation.
Theano is more of a mathematical library then a machine learning library. It does not provide you with pre-built models that you can train on your data set, instead it provides you with tools to build your own machine learning models.
However, if you are looking for machine learning tool kits you might want to try Keras, Lasagna or PyLearn2.
Theano Vs. Others:
| Feature | TensorFlow | Theano | PyTorch | DeepLearning4j |
| License | Apache Open Source 2.0 | No license yet | No license yet | Apache Open Source 2.0 |
| Written in | Python | Python | Python | Java |
| Strengths | Ensured continued support, Visualization suit available, debugging potential, easy to spin session without having to restart programs | Optimized for both CPU and GPU, well adapted for numerical tasks, can convert just-in-time compilation to CPU or GPU code. | Easy to switch between CPU and GPU, easy to combine modular code blocks | Supports JVM languages like Java and Scala, runs parallely on CPUs and GPUs. |
| Drawbacks | Not completely open source | Relies on the mathematical side of deep learning more often and has lower number of followers today even though it was the first set of libraries launched. | Not accessible to the academic community | Becoming obsolete |
Step By Step Guide to Build a Project in Theano:
- A Brief Python Refresher
Theano is basically a Python library, it would be a good idea app that we go over some important points in Python before they get started with development using Theano.
- Considering Python is an interpreted language it is rather platform-independent. However, generally slower than some other languages like C.
- Python uses dynamic typing that is, even though each variable has a specific type during execution, these types are not required to be explicitly stated in the code.
- Instead of special characters like curly brackets Python uses indentation for block delimiting. This means that Python uses white space to separate a block of code.
- To define a list in Python we use square brackets.
- To define a key value mapping or a dictionary we use curly brackets in Python.
- Python is equipped with list comprehension which is a need shorthand for constructing lists. The following code snippet shows a loop for 5 steps with values from 0 to 4 and each time it adds +i1 to the list.
- Step 1: Set the Stage
To use Theano in any new applications, you will need to install the dependencies and install Theano itself. Theano provides support for most contemporary operating systems like Windows, MacOS, Linux, Ubuntu, CentOS 6 etc.
We recommend using the Anaconda distribution for Theano as it includes Theano, SciPy and all important libraries in one nice package.
At the Theano home page for installing Theano, you can choose your operating system and the guide will help you glide through the installation process. Once your setup is complete, you can also test how well Theano is performing with your GPU here.
If you're using the Ubuntu platform you might get away with executing just these two commands:
If it doesn’t work out as expected, check Theano Home page for instructions on installing Theano and setting up environment variables as applicable.
To import Theano in a Python program, you need to write just this:
- Step 2: Extensions and Wrappers for Theano:
Theano could be a hard tool to start working with if you are new to Deep learning.
There are many popular Python projects that make theano simpler to use. These projects also called libraries or wrappers provide data structures and behaviours written in Python.
These are specifically designed to quickly and reliability create such deep learning models which are fast and efficient as they get created an executed by Theano under the covers.
We will highly encourage you to use one of these wrapper on top of Theano as it makes code easier to understand and develop. Some of the major such libraries are Keras, Lasagne, Mariana, Passage etc. You can find a comprehensive list of libraries and wrappers which have been developed using and for usage with Python and Theano here.
- Step 3: Project Using Theano
Theano helps you build your own model from scratch instead of giving you a skeleton to train with your dataset.
The following stages encapsulate the lifecycle of a project being developed in Theano:
- Load Data
- Define Model
- Compile Model
- Fit Model
- Evaluate model
- Integrate the project
Here, a demo project on Pima Indians Diabetes Onset dataset along with all the respective code is compiled. It could be used as a stepping stone before you start with your own project.
Theano Cheat sheet
Here you will find a pdf cheat sheet for using and implementing various functions in Theano.
Here you will find a practical implementation of building blocks provided by Theano. These code blocks are explained in the context of the demo project.
The Theano Documentation is another source which you can refer to for understanding the know-how of this tool.
Endnote:
This comes to the end of our beginner's guide to Theano. It will not be receiving any serious upgrades in the near future. Some have even gone as far as calling it a ‘dead ecosystem’.
However, the strong and thriving community backing Theano has been unwavering. It is still a quite efficient and useful tool for developing skeletal to sound model architectures for your deep learning projects.
Now that we have discussed Theano and its various aspects, we hope you find Theano is a fun tool to work with. Happy Coding!