Anix Lynch – Technical Notes & Engineering Playbooks

#transformers

Articles tagged with #transformers

NLTK vs. Hugging Face #3: Visualization, Clustering, NER, Word Clouds
NLTK Codehttps://gist.github.com/43e222f213f3f217a3d99c1912d12375.git HuggingFace Code https://gist.github.com/03fac7cf537d5e07946aabee14aa81ef.git NLTK Code Chunk 1: Importing NLTK and Displaying Corpus Information Code: import nltk from nltk.co...
Nov 13, 20248 min read3
NLTK VS HuggingFace #2 - Emoji replacement, smart quotes handling, stop word removal, stemming, lemmatization, and spell checking
Source code here: NLTK Code https://gist.github.com/a60a29d0aede72d3c9f5854bdd5d7916.git HuggingFace Code https://gist.github.com/ed6e307157cb2790285247057c25e7f0.git Table summarizing scenarios where NLTK is still preferable Compared to Hugging Face...
Nov 13, 202410 min read2
21 NLP core concepts with sample output
1. Tokenization from nltk.tokenize import word_tokenize text = "Natural Language Processing with Python is fun!" tokens = word_tokenize(text) print(tokens) Output: ['Natural', 'Language', 'Processing', 'with', 'Python', 'is', 'fun', '!'] word_toke...
Oct 11, 202416 min read8
6 NLP tasks by HuggingFace Transformers
Natural Language Processing (NLP) encompasses various tasks that enable machines to understand, interpret, and generate human language. Below are some common NLP tasks, along with sample code snippets using Python and the Hugging Face Transformers li...
Oct 9, 20248 min read8
20 Huggingface transformer concepts with Before-and-After Examples
1. Installing Hugging Face Transformers Library 📦 Boilerplate Code: pip install transformers 2. Loading a Pre-trained Model and Tokenizer 🤖 Use Case: Load a pre-trained model and its associated tokenizer. Goal: Use a pre-trained transformer model...
Oct 4, 202413 min read9

Command Palette