Transformer
/trænsˈfɔːrmər/trans·form·mernoun
Definition
1.[in natural language processing] a neural network architecture that relies solely on attention mechanisms, allowing it to process sequential data efficiently.
The Transformer model revolutionized sequence-to-sequence tasks by eliminating recurrence.
Formal statement
Attention(Q, K, V) = softmax(Q K^T / sqrt(d_k)) VThe core mechanism involves scaled dot-product attention.
Etymology
Coined by the authors of the seminal 2017 paper, drawing inspiration from the concept of transforming data representations.
Synonyms
- attention-based modelsense 1 · Absolute