Transfer learning, Multi-task learning, Meta learning, Few-shot learning

https://ruder.io/thesis/index.html

 

Transfer learning, Multi-task learning, Meta learning, Few-shot learning

헷갈리는 개념들을 정리할 때 꼭 필요한 마법의 그림, 학부 수업에서 이 그림으로 배워서 더 쉽게 느껴지는 것 같기도 하다.

 

One/Zero shot learning

When the number of samples in each class is one, it’s called one shot learning. When it’s zero, it’s called zero shot learning. One shot learning is always studied in few shot learning while zero shot learning is not the same. In few shot learning, we leverage the auxiliary dataset while in zero shot learning, the model relies on attributes or semantic information of those classes to transfer some supervision signals and make learning feasible (Wang et al., 2019b). There is a chance to borrow some methods from zero-shot learning for Few-shot learning.

Transfer learning

Transfer learning tries to transfer knowledge learned from another dataset to the target dataset. In the current setting of few shot classification, it can be viewed as a special case of transfer learning, we transfer knowledge from base dataset to novel dataset.

Multitask learning

multitask learning aims to learn several related tasks together. This is a more efficient way of learning so that we can save time and computing resources. In addition, this should also be one feature of general artificial intelligence. Meta learning can also be viewed as a special case of multitask learning in which all tasks are sampled from the same task distribution, which means all tasks share the same statistics. Also during testing, it’s tested on novel tasks. It’s not the case for regular multitask learning.

Semi-supervised learning

Semi-supervised learning has both labeled data and unlabeled data while in the classic few shot learning, we only have very limited labeled data. The amount of both labeled and unlabeled data, which semi-supervised data can use, is much more than than few shot learning. There is an intersection for these two types of learning: semi-supervised few shot learning.

Overview of few shot learning

In this section, we will discuss the taxonomy of few shot learning methods and methods under each category. There are four types of methods: optimization based methods, metric learning based methods, parameter generation based methods (or black box / model-based methods), and data augmentation based methods.

 

Machine learning with limited data  F Yao ,2021