#60 - How to input text into your model? Understanding tokenizers.

<p>Hello everyone, in this episode I explain how tokenizers work. They are basically what enables us to input the text into a NLP algorithm like BERT or GPT. In the episode I explain 3 types of tokenizers, word based, character based and sub-word based representation.</p> <p><br></p> <p>Instagram: https://www.instagram.com/podcast.lifewithai/</p> <p>Linkedin: https://www.linkedin.com/company/life-with-ai</p> <p>Huuging Face blog about tokenizers: https://huggingface.co/docs/transformers/tokenizer_summary</p>