How to Use AI LLM APIs in Detail: Complete Developer Guide (2026)

LLM APIs turn ChatGPT-like intelligence into a building block: send a prompt via HTTPS, get back JSON, plug it into your app. In 60 minutes you’ll go from zero to a streaming chat app with tools, cost controls, and production hardening — using OpenAI, Anthropic, Google Gemini, and open-source models.

from openai import OpenAI
client = OpenAI(api_key="sk-...")
resp = client.chat.completions.create(model="gpt-4o-mini", messages=[{"role":"user","content":"Summarize ToolsLead in one sentence."}])
print(resp.choices[0].message.content)

What Is an LLM API?

An LLM (Large Language Model) API is an HTTP endpoint that runs inference hosted by someone else. You send JSON with your prompt + parameters; the provider’s GPUs run the model and return the generated text. No ML PhD needed.

• Provider-hosted: OpenAI (GPT-4o/4o-mini), Anthropic (Claude 3.5 Sonnet/Haiku), Google (Gemini 2.5 Pro/Flash), Cohere, Mistral.
• Inference clouds: Groq, Together.ai, Fireworks, Replicate — host open models (Llama 3.3, Qwen, DeepSeek) with OpenAI-compatible endpoints.
• Local: Ollama, vLLM, LM Studio — run entirely on your laptop/VPS for privacy.

Mental model: Think of it like Stripe for intelligence — you pay per token (input + output), not per server hour. 1 token ≈ 0.75 words. A 300-word prompt + 500-word answer ≈ 1000 tokens.

Choosing Your Provider (2026 Comparison)

Rule of thumb: Prototype on GPT-4o-mini or Gemini Flash (cheap, fast). Switch to Claude Sonnet if you need nuanced writing or 10+ step reasoning. Move batch workloads to Groq/Together for 3× cost saving. Use Ollama for patient data or air-gapped docs.

Your First API Call in 5 Minutes (Python)

Step 1 — Get a Key

• OpenAI: platform.openai.com → API keys → Create → copy sk-proj-xxx.
• Anthropic: console.anthropic.com → Keys.
• Gemini: aistudio.google.com → Get API key.

Step 2 — Install SDK

pip install openai anthropic google-generativeai python-dotenv

Step 3 — Call OpenAI

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

resp = client.chat.completions.create(
  model="gpt-4o-mini",
  messages=[
    {"role": "system", "content": "You are a helpful PDF assistant."},
    {"role": "user", "content": "How do I compress a 5MB PDF to 200KB?"}
  ],
  temperature=0.7,
  max_tokens=400,
)
print(resp.choices[0].message.content)
print(f"Tokens used: {resp.usage.total_tokens}")

Same Prompt on Anthropic

import anthropic
client = anthropic.Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))
msg = client.messages.create(
  model="claude-3-5-sonnet-20241022",
  max_tokens=400,
  system="You are a helpful PDF assistant.",
  messages=[{"role":"user","content":"How do I compress a 5MB PDF to 200KB?"}]
)
print(msg.content[0].text)

Same on Gemini

import google.generativeai as genai
genai.configure(api_key=os.getenv("GEMINI_API_KEY"))
model = genai.GenerativeModel("gemini-2.0-flash")
resp = model.generate_content("How do I compress a 5MB PDF to 200KB?")
print(resp.text)

You now have 3-provider coverage. Next: correct auth, because leaked keys are the #1 breach.

Authentication & Key Management

• Never commit keys: Put in .env ignored by git. Add .env.example with placeholder.
• Server-only: In Next.js, use NEXT_PRIVATE_ or server route — keys must never reach the browser. Create src/app/api/chat/route.ts that checks NextAuth session then proxies to OpenAI.
• Rotate: Regenerate monthly; set usage limits ($10 hard cap) in provider dashboards.
• Per-user keys (advanced): For B2B, let users bring their own key and bill them directly — reduces your cost/risk.

# Next.js API route — secure proxy
export async function POST(req: Request) {
  const session = await getServerSession(authOptions)
  if (!session) return new Response("Unauthorized", {status: 401})
  const {message} = await req.json()
  const openai = new OpenAI({apiKey: process.env.OPENAI_API_KEY})
  const resp = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{role:"user", content: message}]
  })
  return Response.json({text: resp.choices[0].message.content})
}

Chat vs Completions vs Responses API

OpenAI has 3 flavors; pick right:

Anthropic uses messages.create (similar to chat), Gemini uses generateContent with contents[]. Wrapper libs like Vercel AI SDK or LangChain normalize these.

Streaming Responses Like ChatGPT

Non-streaming waits 2–4s then dumps answer. Streaming sends tokens as they generate → feels instant.

# Python streaming
stream = client.chat.completions.create(
  model="gpt-4o-mini",
  messages=[{"role":"user","content":"Write a 200-word story about a PDF."}],
  stream=True,
)
for chunk in stream:
  if chunk.choices[0].delta.content:
    print(chunk.choices[0].delta.content, end="", flush=True)

Frontend (Next.js + Vercel AI SDK):

// app/api/chat/route.ts
import { streamText } from "ai"
import { openai } from "@ai-sdk/openai"
export async function POST(req: Request){
  const {messages} = await req.json()
  const result = streamText({model: openai("gpt-4o-mini"), messages})
  return result.toDataStreamResponse()
}
// client
import { useChat } from "ai/react"
const {messages, input, handleSubmit} = useChat({api:"/api/chat"})

Under the hood, streaming uses Server-Sent Events (SSE). Each chunk is data: {"delta": ...}. Always handle finish_reason to know when to stop.

System Prompts & Message Roles

Roles:

• system: High-priority instruction — “You are ToolsLead assistant, answer concisely, cite tools.” Anthropic requires it separate; OpenAI puts it as first message.
• user: End-user query.
• assistant: Previous AI answer (for history).
• tool: Result of a tool/function call.

Crafting a good system prompt:

SYSTEM = """You are ToolsLead — a privacy-first PDF assistant.
- Answer in clear, concise Markdown.
- If user asks to compress/merge/split PDF, suggest the exact ToolsLead tool link.
- Never hallucinate tool links; only use /tools/* that exist.
- For code, give Python snippets with comments.
Temperature: 0.3 for factual answers, 0.7 for creative."""

Keep system prompt under 300 tokens; long system prompts waste money every call. Store per-user preferences (language, tone) and inject dynamically.

Function Calling / Tools — The Agentic Superpower

Function calling lets the LLM call your code. Without it, it’s a autocomplete. With it, it can compress your PDF, query DB, send email.

functions = [{
  "type": "function",
  "function": {
    "name": "compress_pdf",
    "description": "Compress PDF to target size in KB",
    "parameters": {
      "type": "object",
      "properties": {
        "file_url": {"type":"string"},
        "target_kb": {"type":"integer","enum":[100,200,500,1024]}
      },
      "required": ["file_url","target_kb"]
    }
  }
}]
resp = client.chat.completions.create(
  model="gpt-4o-mini",
  messages=[{"role":"user","content":"Compress my invoice to 200KB"}],
  tools=functions,
  tool_choice="auto"
)
# If LLM decides to call function:
# resp.choices[0].message.tool_calls[0].function.name == "compress_pdf"
# You execute compress_pdf() and feed result back as tool message for final answer.

Flow: User → LLM decides to call tool → you run tool → send {"role":"tool","tool_call_id":..., "content": json} → LLM synthesizes final answer.

This powers AI PDF Assistant on ToolsLead: user chats, LLM picks Merge/Compress/Split tool via function call, backend executes PDF-lib, returns link.

Core Parameters: Temperature, Top-P, Max Tokens

New 2026 best practice: Use Structured Outputs (response_format: {type:"json_schema", json_schema: {...}}) instead of prompting “reply JSON” — guarantees schema compliance, no hallucinations.

Cost & Token Optimization (Save 70%)

• Choose right model: GPT-4o-mini is 16× cheaper than GPT-4o and fits 80% tasks. Benchmark with LLM-as-judge before upgrading.
• Prompt compression: Trim system prompt, remove verbose examples, use tiktoken to count tokens. Prefer markdown over JSON tables.
• Caching: OpenAI prompt caching → 50% discount on repeated prefix (system prompt). Anthropic caching → 90% on long docs. Use it for RAG context.
• Batch API: OpenAI Batch (50% discount) for 1000s PDFs overnight — submit JSONL, get results in 24h. Ideal for classification jobs.
• KV caching via reuse: Send conversation history as array — don’t re-send 10k RAG chunks every turn; keep context window sliding.

import tiktoken
enc = tiktoken.encoding_for_model("gpt-4o-mini")
tokens = len(enc.encode("Your prompt here"))
print(tokens) # watch before call

Example: 1000 users × 10 turns × 800 tokens = 8M tokens. At 4o-mini ($0.15/1M) = $1.20, at GPT-4o = $20. Right model = 94% saving.

Errors, Rate Limits & Retries

from tenacity import retry, wait_exponential, stop_after_attempt
@retry(wait=wait_exponential(multiplier=1, min=1, max=10), stop=stop_after_attempt(3))
def call_llm():
  return client.chat.completions.create(model="gpt-4o-mini", messages=[...])

Production Checklist (Before Launch)

1. Guardrails: Input moderation (OpenAI moderation API), output validation (no hallucinated links), timeout 15s + fallback message.
2. Observability: Log prompt, response, tokens, latency, cost per user. Tools: Langfuse, Helicone, LangSmith. Alert on error rate >5%.
3. Rate limit your side: Per-user RPM via Redis (Upstash), plus global budget. Don’t let one user burn $100.
4. Privacy: Add “Your data may be used to improve AI” disclosure if true; otherwise use Zero Data Retention (ZDR) — OpenAI & Anthropic offer ZDR for Enterprise.
5. Fallback: If primary LLM fails, fallback to Gemini Flash or local model; show “AI is temporarily on high demand”.
6. Testing: Eval set of 20 prompts with expected outputs; run LLM-as-judge weekly to catch regression after provider updates.
7. Cost caps: Stripe metered billing + daily spend alert at $20/$50 thresholds.

Congratulations — you now have a production-ready LLM API integration. Start small (one endpoint), instrument it, then scale to agents. Next guide: use these APIs inside a RAG pipeline to chat with your own PDFs.

Advanced Patterns: Agents, Multi-Modal & Caching

Once basics work, level up. Multi-modal LLM calls let you send images/PDFs directly to GPT-4o/Gemini without text extraction: messages=[{"role":"user","content":[{"type":"text","text":"Summarize this invoice"},{"type":"image_url","image_url":{"url":"data:image/png;base64,..."}}]}]. For PDF QA, this often beats OCR because vision models read layout natively. Cost is higher (image tokens) but accuracy on scanned forms jumps 12% in our tests.

Caching & batching: For chat history of 20 turns, don't re-send 20 messages naive — use OpenAI's prompt_cache_key or Anthropic prompt caching (store system + long doc as cacheable block, 90% discount on reuse). For bulk classification of 10k PDFs, use Batch API: upload JSONL with 10k requests → 50% discount + results in 6h. Ideal for overnight tagging jobs.

Agents: Turn function calling into loops: user asks “Analyze this PDF and email summary to manager”. The LLM calls read_pdf → summarize → send_email sequentially. Wrap in a while tool_calls: loop until finish_reason != "tool_calls". Add a guard: max 5 iterations to prevent infinite loops. LangGraph and OpenAI Agents SDK handle this state machine for you.

Observability: Pipe every LLM call through Helicone/Langfuse: log prompt, model, tokens, latency, user_id, feedback thumbs. Build a dashboard of cost per user, hallucination rate (flag when model says "I don't know" fallback), and latency p95. Alert on 429 spikes — you may need to bump tier or add queue. Prompt versioning in git (e.g., prompts/summarize_v3.md) lets you A/B test: route 10% traffic to new prompt via feature flag and compare thumbs-up rate before full rollout.

Security: Sanitize user inputs before adding to prompt (escape XML tags to prevent injection). Validate JSON outputs with zod/pydantic before acting — never eval LLM JSON. Keep audit log of tool calls for compliance (especially if LLM can email or modify DB).

FAQs