Transformer models: an introduction and catalog — 2022 Edition

Why this post

What are Transformers

Encoder/Decoder architecture

Attention

What are Transformers used for and why are they so popular

The Transformers catalog

Pretraining Architecture

Pretraining Task

Application

Catalog table

Transformer model catalog (see original table here)

Family Tree

Transformers family treee

Chronological timeline

Catalog List

Further reading

--

--

--

Cofounder/CTO at Curai (AI for healthcare). Former Quora VP, Netflix Director. Software, Machine Learning, Data, Recsys... From Barcelona, in the Valley

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Linear Regression in Keras

Improve Your CNN With These 6 Easy Tips

Deep Learning- Backpropogation from scratch- mathematical intuition

Solving a Ledger CTF challenge with Deep Learning on esDynamic

Decision Trees with CART Algorithm

Predicting physical activity based on smartphone sensor data using CNN + LSTM

Deep CV: How do CNNs work?

Investigating Machine Learning Techniques to Improve Spec Tests — IV

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Xavier Amatriain

Xavier Amatriain

Cofounder/CTO at Curai (AI for healthcare). Former Quora VP, Netflix Director. Software, Machine Learning, Data, Recsys... From Barcelona, in the Valley

More from Medium

ML Arxiv Haul #5

Overview of explainable AI methods in NLP

Hands on Data Augmentation in NLP using NLPAUG Python Library

Nanit’s AI Development Process — Improve Your Model Quality, Time to Market and Culture