NLP Libraries
NLTK, spaCy, Hugging Face and friends.
This is an overview — pick a sub-topic below.
Sub-topics (11)
NLTK
The classic natural-language toolkit — great for learning NLP concepts.
spaCy
Industrial-strength NLP pipelines: tokenization, tagging, parsing, NER, with pretrained models.
Gensim
Topic modelling (LDA) and word embeddings (Word2Vec, FastText) on large corpora.
TextBlob, Stanza, Flair and AllenNLP
Simple NLP (TextBlob), Stanford's multilingual pipelines (Stanza), contextual embeddings and NER (Flair), and the research library AllenNLP (archived).
Hugging Face Transformers, Tokenizers and Datasets
Thousands of pretrained models with a common API, fast tokenizers, and datasets.
Sentence Transformers
Compute sentence embeddings for semantic search, clustering and similarity.
BERTopic, Top2Vec and KeyBERT
Embedding-based topic modelling and keyword extraction.
Presidio
Detect and anonymise personal data (names, phone numbers, IDs) in text.
fastText, Polyglot and Sacremoses
Fast text classification and language ID (fastText), older multilingual toolkit (Polyglot), and Moses tokenization (Sacremoses).
DeepPavlov and Rasa
Frameworks for building conversational agents and chatbots.
Vector Databases and Search Clients
FAISS, Annoy, hnswlib, Chroma, Qdrant, Weaviate, Milvus, Pinecone and Elasticsearch clients.
Spotted a mistake or want to add detail? Suggest an improvement — Content Contributors' edits are reviewed by editors before they go live.
Discussion (0)
Sign in or join free to ask questions and discuss this topic.