Transformer models: an introduction and catalog — 2022 Edition

Why this post

What are Transformers

Encoder/Decoder architecture

Attention

What are Transformers used for and why are they so popular

The Transformers catalog

Pretraining Architecture

Pretraining Task

Application

Catalog table

Transformer model catalog (see original table here)

Family Tree

Transformers family treee

Chronological timeline

Catalog List

Further reading

--

--

--

Cofounder/CTO at Curai (AI for healthcare). Former Quora VP, Netflix Director. Software, Machine Learning, Data, Recsys... From Barcelona, in the Valley

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Introducing TensorFlow Data Validation: Data Understanding, Validation, and Monitoring At Scale

XGBoost versus Random Forest

Stochastic Gradient Descent: Explanation and Complete Implementation from Scratch

Why Your Machine Learning Project Might Fail And How to Avoid It

Deep Learning with Tabular Data

BIG DATA BOSS- BOSS MAKERS!

70% ML. 30% FUN .100% KOLLYWOOD!

How Good is Your Model? — Intro To Machine Learning #4

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Xavier Amatriain

Xavier Amatriain

Cofounder/CTO at Curai (AI for healthcare). Former Quora VP, Netflix Director. Software, Machine Learning, Data, Recsys... From Barcelona, in the Valley

More from Medium

NLP: Building a Grammatical Error Correction model — Deep Learning Analytics

ML Arxiv Haul #5

Two minutes NLP — Explain predictions with LIME

Using Huggingface🤗 Transformers with PyTorch 🔥 for NLP tasks