NLP, Transformers & LLMs
AI · Advanced
Definition
Transformers use self-attention to understand relationships between words; Large Language Models are huge transformers trained on text.
They power chatbots, translation, code assistants and search.
Diagram
Flowchart
Example: Tokenising text
text = 'AI is the future'
tokens = text.lower().split()
print(tokens)Output: ['ai', 'is', 'the', 'future']
Key points
- Attention weighs relevant words
- Pretraining + fine-tuning
- Prompting and RAG improve answers
Interview questions
What is RAG?
Retrieval-Augmented Generation: fetch relevant documents and give them to the LLM as context.