Natural Language Processing

3+ hour, 14+ min ago   (647+ words) An introduction to Large Language Models, what they are, how they work, and use cases. With the announcement of ChatGPT and Google Bard, more and more people are speaking about Large Language Models. It’s the new wave, and everybody wants…...

8+ hour, 6+ min ago   (681+ words) Previously, on Day 28: Explained multi-head attention as used in Transformer models, detailing how multiple attention heads enable the capture of diverse and overlapping patterns within input sequences through independent, parallel computations, culminating in a combined representation. To help Transformers understand…...

8+ hour, 6+ min ago   (597+ words) Why Sequence Models Struggled, and Why Transformers Matter But this design has major drawbacks: Because of these issues, RNNs have trouble with long texts, struggle to link distant words, and are slow to train. From Sequences to Sets: The Core…...

2+ week, 1+ day ago   (449+ words) In this blog post I give an introduction to the Transformer architecture as originally presented by Vaswani et al., in their landmark paper, “Attention Is All You Need”. My focus will be on visuals and intuition. Specifically, I intend to…...

8+ hour, 6+ min ago   (707+ words) Previously, on Day 11: Explained one-hot encoding as a method for representing words as sparse vectors, highlighted its limitations regarding similarity and efficiency, and introduced the concept of word embeddings as an improvement. Word2Vec is a technique that addresses both problems. It…...

8+ hour, 6+ min ago   (508+ words) What is One-Hot Encoding? Computers need numbers, not words. To process language, we first turn words into numbers. One of the earliest ways to do this is one-hot encoding. A one-hot encoding represents each word in your vocabulary with a…...

8+ hour, 6+ min ago   (490+ words) Previously, on Day 12: Explained how Word2Vec models generate word embeddings using skip-gram and CBOW techniques, compared their methods and trade-offs, and introduced a minimal example using Gensim. Focusing only on local context leaves gaps. If two words never appear close together…...

8+ hour, 6+ min ago   (456+ words) Previously, on Day 7: Explained part-of-speech (POS) tagging, its importance in NLP, different approaches to tagging, common tagsets, and typical challenges such as ambiguity and unknown words. Named Entity Recognition (NER) is a core task in Natural Language Processing (NLP). Its…...

8+ hour, 6+ min ago   (518+ words) A sequence-to-sequence model (often called a seq2seq model) is a type of neural network that transforms one sequence into another. In natural language processing (NLP), a "sequence" is usually a sentence, a paragraph, or a list of words. Seq2seq models excel when…...

8+ hour, 6+ min ago   (585+ words) Previously, on Day 17: Explained the structure and function of neural networks, including neurons, layers, weights, biases, and activation functions, and how neural networks are trained to solve natural language processing tasks. Human language is structured as a sequence. The meaning…...