Multitask vs Multiclass vs Multilabel¶

Multilabel:¶

a single input can have non-exclusive labels

Multiclass:¶

a single output can have a single, mutually exclusive label among >2 candidate labels

Multitask:¶

A single input has different classification problems. MTL is more like an optimization trick like pretraining, where we have multiple different tasks which would benefit from sharing low-level features.

Can be more efficient and improve performance by 1) forcing the classifier to learn more general low-level features (preventing overfitting) and 2) essentially exposing to more training data, since we backprop \(n\) times for a model with \(n\) tasks

Examples of each¶

Multilabel¶

Given an image of an animal, label 1: what is the animal? label 2: is the animal jumping? label 3: is the animal furry?

Multiclass¶

Given an image of a dog, what type of dog is it? MNIST handwritten digits is a popular example. Single output

Multitask¶

Given an image, is there a tree? a dog? a cat? In multitask, it makes sense to train multiple classifiers.