This part is taken from an Interview of Prof. Geoffrey E. Hinton by Prof. Andrew (in Coursera course)
What is your advice to students to want to pursue a career in Deep Learning?
(I thin that the answer given by Prof. Geoffrey E. Hinton is useful to any researcher irrespective of the field of research)
Here is the answer.
"Read the literature, but don't read too much of it. So this is advice I got from my advisor, which is very unlike what most people say. Most people say you should spend several years reading the literature and then you should start working on your own ideas. And that may be true for some researchers, but for creative researchers I think what you want to do is read a little bit of the literature. And notice something that you think everybody is doing wrong, I'm contrary in that sense. You look at it and it just doesn't feel right. And then figure out how to do it right. And then when people tell you, that's no good, just keep at it. And I have a very good principle for helping people keep at it, which is either your intuitions are good or they're not. If your intuitions are good, you should follow them and you'll eventually be successful. If your intuitions are not good, it doesn't matter what you do."
[Prof. Andrew Ng: I usually advise people to not just read, but replicate published papers. And maybe that puts a natural limiter on how many you could do, because replicating results is pretty time consuming.]
"Yes, it's true that when you're trying to replicate a published you discover all over little tricks necessary to make it work. The other advice I have is, never stop programming. Because if you give a student something to do, if they're botching, they'll come back and say, it didn't work. And the reason it didn't work would be some little decision they made, that they didn't realize is crucial. And if you give it to a good student, like for example. You can give him anything and he'll come back and say, it worked. I remember doing this once, and I said, but wait a minute. Since we last talked, I realized it couldn't possibly work for the following reason. And said, yeah, I realized that right away, so I assumed you didn't mean that. "
[Prof. Andrew Ng: ny other advice for people that want to break into AI and deep learning?]
"Basically, read enough so you start developing intuitions. And then, trust your intuitions and go for it, don't be too worried if everybody else says it's nonsense."
"If you think it's a really good idea, and other people tell you it's complete nonsense, then you know you're really on to something."
I think that this advice is applicable to any technical field. Some researchers spend too much time on reading and this lead to too much bias in the published results. If researchers read just enough so that they can think and work on a problem, they can contribute something new in that field. Otherwise, they will be doing just what others have done (mostly incremental work).
Another thing I understand is that students and researchers should try to replicate seminar research papers from the scratch. This help the researchers understand the process the top researchers have undertone. This often provide the skills needed to find out something new in that field and contribute novel works in that field.
Reference:
Weblink: https://www.coursera.org/learn/neural-networks-deep-learning/lecture/dcm5r/geoffrey-hinton-interview
