LLM Tools and SDKs
Official SDKs, open-model tooling and application frameworks.
This is an overview — pick a sub-topic below.
Sub-topics (12)
OpenAI, Anthropic and Google Gen AI SDKs
Official Python clients for hosted models (GPT, Claude, Gemini).
Hugging Face Hub and Transformers for LLMs
Download, run and share open models and datasets.
vLLM, SGLang, Ollama and llama.cpp
Serve and run open LLMs: high-throughput servers (vLLM, SGLang) and local runners (Ollama, llama-cpp-python).
LiteLLM
One API to call 100+ LLM providers, with cost tracking and fallbacks.
LangChain, LangGraph and LlamaIndex
Frameworks for building LLM apps: chains, agents, graphs of steps, and data connectors/indexes for RAG.
CrewAI, AutoGen, smolagents, Agno and Semantic Kernel
Agent frameworks: role-based crews (CrewAI), multi-agent conversations (AutoGen), lightweight code agents (smolagents), and others.
Haystack and DSPy
Pipelines for search and RAG (Haystack); programming — not prompting — LLM pipelines with automatic prompt optimisation (DSPy).
Pydantic AI, Instructor, Guidance and Outlines
Get reliable structured outputs from LLMs.
Guardrails AI and NeMo Guardrails
Add validation and safety policies around LLM inputs and outputs.
Ragas, DeepEval, Promptfoo, LangSmith, Phoenix and TruLens
Evaluate and observe LLM apps: answer quality, faithfulness to sources, regressions, traces and costs.
PEFT, TRL, bitsandbytes, DeepSpeed and Accelerate
Hugging Face's fine-tuning stack plus memory and distributed-training tools.
Axolotl, Unsloth, torchtune and Megatron-LM
Fine-tuning and large-scale training frameworks.
Spotted a mistake or want to add detail? Suggest an improvement — Content Contributors' edits are reviewed by editors before they go live.
Discussion (0)
Sign in or join free to ask questions and discuss this topic.