Geoffrey E. Hinton's advice for students (from Coursera)

This part is taken from an Interview of Prof. Geoffrey E. Hinton by Prof. Andrew (in Coursera course)

What is your advice to students to want to pursue a career in Deep Learning?

(I thin that the answer given by Prof. Geoffrey E. Hinton is useful to any researcher irrespective of the field of research)

Here is the answer.

"Read the literature, but don't read too much of it. So this is advice I got from my advisor, which is very unlike what most people say. Most people say you should spend several years reading the literature and then you should start working on your own ideas. And that may be true for some researchers, but for creative researchers I think what you want to do is read a little bit of the literature. And notice something that you think everybody is doing wrong, I'm contrary in that sense. You look at it and it just doesn't feel right. And then figure out how to do it right. And then when people tell you, that's no good, just keep at it. And I have a very good principle for helping people keep at it, which is either your intuitions are good or they're not. If your intuitions are good, you should follow them and you'll eventually be successful. If your intuitions are not good, it doesn't matter what you do."

[Prof. Andrew Ng: I usually advise people to not just read, but replicate published papers. And maybe that puts a natural limiter on how many you could do, because replicating results is pretty time consuming.]

"Yes, it's true that when you're trying to replicate a published you discover all over little tricks necessary to make it work. The other advice I have is, never stop programming. Because if you give a student something to do, if they're botching, they'll come back and say, it didn't work. And the reason it didn't work would be some little decision they made, that they didn't realize is crucial. And if you give it to a good student, like for example. You can give him anything and he'll come back and say, it worked. I remember doing this once, and I said, but wait a minute. Since we last talked, I realized it couldn't possibly work for the following reason. And said, yeah, I realized that right away, so I assumed you didn't mean that. "

[Prof. Andrew Ng: ny other advice for people that want to break into AI and deep learning?]

"Basically, read enough so you start developing intuitions. And then, trust your intuitions and go for it, don't be too worried if everybody else says it's nonsense."

"If you think it's a really good idea, and other people tell you it's complete nonsense, then you know you're really on to something."

I think that this advice is applicable to any technical field. Some researchers spend too much time on reading and this lead to too much bias in the published results. If researchers read just enough so that they can think and work on a problem, they can contribute something new in that field. Otherwise, they will be doing just what others have done (mostly incremental work).

Another thing I understand is that students and researchers should try to replicate seminar research papers from the scratch. This help the researchers understand the process the top researchers have undertone. This often provide the skills needed to find out something new in that field and contribute novel works in that field.

Reference:

Weblink: https://www.coursera.org/learn/neural-networks-deep-learning/lecture/dcm5r/geoffrey-hinton-interview

Machine Learning

Question: Which of them is a supervised classification problem?

Answer: Using labeled financial data to predict whether the value of a stock will go up or go down next week.

"Exactly! In this example, there are two discrete, qualitative outcomes: the stock market going up, and the stock market going down. This can be represented using a binary variable, and is an application perfectly suited for classification."

So this supervised learning can be used to predict whether something will go up or down (stock prices, temperature, number of sales, etc. To do this you need labelled data because, this is a supervised learning. This can be used whether spin is up or down kind of problem.

Where this type of model can be used in physics, chemistry, mathematics, etc? Is there such studies?

A common data sets used is Iris dataset.

This data set contain

1. petal length 
2. petal width
3. Setal length
4. Setal width

Here, in this blog, you can see how these four quantities are measured. 

Target variable?

In machine learning, a target variable is one that should be the output (after the analysis)

Three different flower species. 

          0. Setosa
          1. Versicolor
          2. Virginica
Here, we are going to give the four data of an unknown flower (to the trained model), and we are going to find out which flower it is (the model will give 0 or 1 or 2 based on the input).

iris.data.shape gives (150, 4) which means that there are 154 rows (data) with four different information (here petal length, petal width, Setal length, Setal width).  

iris.data.shape   

(150,4)

iris.target_names

(names of the target in an array)

(to be continued)

AI predicts that our Earth goes around the Sun: A prediction by neural networks

A neural network algorithm leans physical law and the result is published in Physical Review Letters, one of the leading journals in Physics.

The abstract reads as follow.

"Despite the success of neural networks at solving concrete physics problems, their use as a general-purpose tool for scientific discovery is still in its infancy. Here, we approach this problem by modelling a neural network architecture after the human physical reasoning process, which has similarities to representation learning. This allows us to make progress towards the long-term goal of machine-assisted scientific discovery from experimental data without making prior assumptions about the system. We apply this method to toy examples and show that the network finds the physically relevant parameters, exploits conservation laws to make predictions, and can help to gain conceptual insights, e.g. Copernicus’ conclusion that the solar system is heliocentric."

We all know that the earth revolves around the Sun. But, without giving this information, how can a computer algorithm can predict? It has been successfully predicted by a neural network for the first time. The authors wants to predict the laws of quantum physics using this neural network. This prediction of planetary law is one of the first step on their way.

This work has been highlighted by Nature as "AI Copernicus ‘discovers’ that Earth orbits the Sun"

References

Physical Review Letters: Discovering physical concepts with neural networks

AI Copernicus ‘discovers’ that Earth orbits the Sun

Machine Learning Tools Installations

Installations

pip install tensorflow --user
pip install Keras --user


To check the full path where TensorFlow is installed

python -c 'import os; import inspect; import tensorflow; print(os.path.dirname(inspect.getfile(tensorflow)))'

/home/full/path/tensorflow

Notes on Machine Learning

I am writing this notes on the application of machine learning in Sciences.

The applications of machine learning algorithms increases day by day. I wonder what are the specific applications of machine learning in Physics, Chemistry and Biology.

What is machine learning? Is  it possible that a machine can learn? Still we are programming. How it is different from the programming? There is a difference. In programming, you dictate the machine what to do in each step. You are cutting down a big job in to smaller junks of steps containing functions and logical steps, variables, constants, if else statements, for loops, do loops, while conditions etc. You are explicitly programming the computer to do your task. But, in machine learning, you are not programming explicitly. In most of the cases, the algorithms from standard machine learning tools learn from large data and understand about the data. If a new unknown data is given, the machine learning algorithm predict the new data correctly.

First let us see a common example.

If you are using google search engine or amazon online store, you are already benefited by Machine Learning. Because, these companies use Machine Learning algorithms to know user preferences and learned guesses to suggest what the authors want.

Here are some of the courses for learning Machine Learning techniques.

The fields which are useful are.

Even high energy physics.

Some of the chemistry database links are here. These would be useful for machine learning applied to Chemistry and biology problems.

RDKit

ChEMBL

KNIME

PubChem
AFlow library
Materials Project

See here for the full list of database of materials. 

A list of database sites for a computational science, quantum chemistry, computational materials science, theoretical condensed matter researchers

Presently, there are a number of sites where you can get the structure information of new materials. Following is a list of such websites.

1. Materials Project (LBNL) (check for its API)
2. Nomad
3. CSIRO Graphene Database (Graphene Nanoflakes)

A related post: Notes on machine learning

Notes on Machine Learning (from RandomMatrix.blogspot.com)