Jeremy Howard: In transfer learning you start with a model trained to do something, then fine-tune it.

Patryk Kocielnik

Writing with the hope that a record of the random things people do around can save some duplication of effort -- except for fun.

More posts by Patryk Kocielnik.

Patryk Kocielnik

15 Oct 2020 • 1 min read

When you have a model trained to recognize images of size 128x128, you don't need to train another model from scratch to be able to recognize images of size 256x256.

What you can do instead, is: retrain the prevoius model for the new task.

Unfreeze the model, select a new learning rate about 5x-10x smaller than where loss starts to increase (rule of thumb). Then, run a training epoch and observe resulting loss.

Also, regarding identifying the learning rate to use - Jeremy noted he could imagine picking a learning rate being done automatically as a result of a project that maybe you could take up! He said he was sure it's possible to do it.

Source: Lesson 3: Deep Learning 2019 - Data blocks; Multi-label classification; Segmentation

Jeremy Howard: In transfer learning you start with a model trained to do something, then fine-tune it.

Patryk Kocielnik

Patryk Kocielnik

Jeremy Howard: Using smaller images in training first works great, but is still largely unknown.

Frank Nielsen: Information sciences seek methods to distill information from data to models.