Natural Language Processing II

• basic knowledge of programing in Python
• high school level of mathematics
• Basics of machine learning on the level of our course Introduction to machine Learning
• Knowledge on the level of our basic Natural Language Processing course

• Preprocessing
  • a few words about encoding, unicode normalization
  • traditional tokenization (simple methods, spacy, moses)
  • subword tokenization (byte-pair encoding, wordpiece, sentencepiece)
  • cleaning (deduplication, boilerplate removal)
• Word embeddings
  • universal ideas
  • skip-gram implementation
• Machine Translation with RNN
  • LSTM and GRU overview
  • RNN language translation implementation
• Transformers
  • attention is all you need
  • transformer architecture
  • GPT2
  • BERT
  • XLNET
• Examples of transfer learning in NLP
  • text classification
  • named entity recognition
  • question answering

Previous courses

Introduction to Machine Learning [MLC_INTRO]

Natural Language Processing [MLC_NLP]

Python I - introduction to programming [PYTHON_INTRO]