Geoffrey E. Hinton's advice for students (from Coursera)

This part is taken from an Interview of Prof. Geoffrey E. Hinton by Prof. Andrew (in Coursera course)

What is your advice to students to want to pursue a career in Deep Learning?

(I thin that the answer given by Prof. Geoffrey E. Hinton is useful to any researcher irrespective of the field of research)

Here is the answer.

"Read the literature, but don't read too much of it. So this is advice I got from my advisor, which is very unlike what most people say. Most people say you should spend several years reading the literature and then you should start working on your own ideas. And that may be true for some researchers, but for creative researchers I think what you want to do is read a little bit of the literature. And notice something that you think everybody is doing wrong, I'm contrary in that sense. You look at it and it just doesn't feel right. And then figure out how to do it right. And then when people tell you, that's no good, just keep at it. And I have a very good principle for helping people keep at it, which is either your intuitions are good or they're not. If your intuitions are good, you should follow them and you'll eventually be successful. If your intuitions are not good, it doesn't matter what you do."

[Prof. Andrew Ng: I usually advise people to not just read, but replicate published papers. And maybe that puts a natural limiter on how many you could do, because replicating results is pretty time consuming.]

"Yes, it's true that when you're trying to replicate a published you discover all over little tricks necessary to make it work. The other advice I have is, never stop programming. Because if you give a student something to do, if they're botching, they'll come back and say, it didn't work. And the reason it didn't work would be some little decision they made, that they didn't realize is crucial. And if you give it to a good student, like for example. You can give him anything and he'll come back and say, it worked. I remember doing this once, and I said, but wait a minute. Since we last talked, I realized it couldn't possibly work for the following reason. And said, yeah, I realized that right away, so I assumed you didn't mean that. "

[Prof. Andrew Ng: ny other advice for people that want to break into AI and deep learning?]

"Basically, read enough so you start developing intuitions. And then, trust your intuitions and go for it, don't be too worried if everybody else says it's nonsense."

"If you think it's a really good idea, and other people tell you it's complete nonsense, then you know you're really on to something."

I think that this advice is applicable to any technical field. Some researchers spend too much time on reading and this lead to too much bias in the published results. If researchers read just enough so that they can think and work on a problem, they can contribute something new in that field. Otherwise, they will be doing just what others have done (mostly incremental work).

Another thing I understand is that students and researchers should try to replicate seminar research papers from the scratch. This help the researchers understand the process the top researchers have undertone. This often provide the skills needed to find out something new in that field and contribute novel works in that field.

Reference:

Weblink: https://www.coursera.org/learn/neural-networks-deep-learning/lecture/dcm5r/geoffrey-hinton-interview

What is your biggest lesson after publishing in Nature Journal?

"What is your biggest lesson?" This is the question that Nature asked researchers published in Nature recently.

Christina Hicks:
"I need to focus on myself for me. It’s easy to become really busy. I run up and down the mountains in the UK Lake District for fun. My brain gets cluttered and overworked if I don’t take a break. Running helps me to slow down."

Johan van den Hoogen:
"Working on that nematode paper helped me to appreciate the value of outsourcing — I didn’t develop the models in it. You should let other people do the things that they’re good at. I understand the models in the paper, but it would have taken me a year and half to create them on my own."

"My biggest realization is that you don’t need to move up the academic career ladder to have a satisfying career in science. The moment I stopped worrying about advancing in academia marked a change for me."

Stephanie Ellis:
"You have to have people around you who can point out the weaknesses in anything you do."

Oscar Serrano:
"Some relationships are more mutually beneficial than others. You can collaborate for years with someone and then realize it’s been a one-way street. You’re sharing ideas and resources with them but getting little in return. But other people really do give back as much as they get and really help you grow your career. I want to be the person that people want to collaborate with because it’s reciprocal."

Here is the full article from Nature.com

This is how a researcher actually develop a new method and what others think of it.

When a new method is developed, how different persons involved perceive the path of the research flow? Here it is.

How much is this true? Comment on this.

Credit: Facebook page Interesting Science.

How to do research Series 2

Keep an eye on list of unsolved, challenging (but doable) and interesting problems (for you) problems always. On the one hand keep strong focus on the present problem. At the same time, if you keep a list of problems, you may solve the problem sooner or later. At present, you might not have the proper tool for doing that or necessary skill to do that (math), or necessary idea to connect the dots (the eureka moment).

Don't put all your eggs in one bucket. You might have a backup unless you have solid evidence that the current problem will be worth solving for long term.


Here is the list of other related posts on "How to do research".