Why is attention particularly crucial in sequence-to-sequence tasks like translation?
Answer options
A
It speeds up the training process
B
It makes the model more interpretable
C
It allows the model to focus on relevant parts of the input
D
when producing an output
E
It ensures the output is of a fixed size
F
It reduces the model's size
Correct answer: It allows the model to focus on relevant parts of the input, when producing an output
Explanation
Attention is crucial in sequence-to-sequence translation because it allows the decoder to dynamically focus on the most relevant parts of the encoded source sequence when generating each output token, overcoming the bottleneck of compressing the entire source into a single fixed vector. Options [2] and [3] form the complete answer: 'It allows the model to focus on relevant parts of the input when producing an output.'