OpenAI API — Developer Review

The OpenAI API is the default starting point for most developers building with large language models, and that’s not just inertia — it earns the position with the broadest model lineup, the most mature SDKs, and documentation that doesn’t leave you guessing. But “default” isn’t the same as “best for your use case.” After shipping production systems against it, I have a clearer picture of where it shines, where it forces architectural compromises, and when a competitor is the smarter call.

This review is opinionated. I’ll tell you which model I actually reach for, when I switch to Anthropic or OpenRouter, and the concrete decisions that have changed how I build.

What the OpenAI API actually gives you

The OpenAI API exposes a family of models through a small set of endpoints. The ones that matter for day-to-day work:

• Chat completions / Responses API — the workhorse for text generation, reasoning, tool calling, and structured output.
• Embeddings — text-embedding-3-small and text-embedding-3-large for retrieval and semantic search.
• Audio — Whisper for transcription, plus text-to-speech voices.
• Images — generation and editing.
• Realtime — low-latency speech-to-speech for voice agents.

The breadth is the selling point. You can build a RAG pipeline, a voice assistant, and an image tool against a single provider with one API key and one billing relationship. That consolidation is worth more than most teams realize until they’ve tried to juggle three separate vendor contracts.

The developer experience backs this up. The official Python and Node SDKs are well-maintained, streaming works cleanly, and structured outputs with JSON schema enforcement actually do what they promise — the model returns valid JSON conforming to your schema instead of you regex-parsing a code fence at 2 a.m. Function calling (now “tools”) is reliable enough to build agent loops on top of without constant babysitting.

Which OpenAI model should you default to?

Here’s my honest answer, and it’s not the one the launch keynotes push: I default to GPT-5.5 for real work, and I’ll keep doing so until 5.6 ships and I’ve benchmarked it on my own tasks.

The pattern with every OpenAI release is the same — the newest model gets the marketing oxygen, but the version that’s been in the wild for a few months is the one with predictable behavior, known failure modes, and community knowledge about how to prompt it. I don’t switch on launch day. I switch when I’ve run my own evals.

The exception is data processing mode — bulk classification, extraction, summarization at scale, anything where I care more about throughput and cost than about the last 5% of output quality. There I drop down to a smaller, cheaper model. The quality delta on structured extraction tasks is often negligible, and the cost difference across millions of tokens is not.

The rule of thumb I use:

• Quality-sensitive, user-facing generation → top-tier model, no compromise.
• High-volume, structured, deterministic tasks → smallest model that passes your eval suite.

If you’re not running an eval suite before picking a model tier, you’re guessing — and you’re almost certainly overpaying for capability you don’t need on the bulk-processing path.

Pricing: what you’ll actually pay

OpenAI prices per token, split between input and output, with output tokens costing several times more than input. This has direct architectural consequences:

• Output tokens are your real cost driver. A verbose system prompt is cheap; a model that rambles in its responses is expensive. Constrain output length aggressively. For structured tasks, demand JSON and nothing else — no preamble, no “Here’s the result:”.
• Caching helps. OpenAI offers automatic prompt caching that discounts repeated input prefixes. If you have a large static system prompt or few-shot examples reused across calls, structure your requests so the stable content comes first. This can cut input costs meaningfully on agent workloads that resend the same context.
• Embeddings are nearly free relative to generation. Don’t over-optimize them. The cost of text-embedding-3-small at scale is rounding error compared to your generation spend.

The Batch API deserves a specific callout: submit jobs asynchronously with a 24-hour turnaround and get a 50% discount. For any offline processing — backfilling embeddings, bulk classification, dataset generation — the Batch API is the correct tool and most teams forget it exists. If you’re processing data overnight anyway, you’re leaving half your budget on the table by hitting the synchronous endpoint.

Rate limits and the production reality

The biggest architectural lesson from running OpenAI in production: design for rate limits and transient failures from day one, not as a retrofit.

You’ll hit two kinds of limits — requests per minute and tokens per minute — and they scale with your usage tier, which scales with your spend history. New accounts start constrained. This means a workload that runs fine in your dev environment can throttle hard the moment you flip to production traffic.

What this forces in practice:

• Exponential backoff with jitter on 429s, non-negotiable. The SDK does some of this, but you should own the retry policy.
• A queue in front of bulk workloads. Don’t fire 10,000 requests at once. Meter them. For genuinely large jobs, use the Batch API instead of trying to muscle through the synchronous endpoint.
• Idempotency. Network timeouts happen. If you retry a non-idempotent generation call, you can double-bill or produce duplicate side effects. Track request IDs.
• A fallback model or provider. When the primary model is degraded or rate-limited, you want a switch you can flip without a redeploy.

That last point is where multi-provider strategy stops being theoretical.

When I pick a competitor over OpenAI

OpenAI is my default, but it’s not automatic. Here’s exactly when I go elsewhere — and these aren’t hypotheticals, they’re decisions I’ve made on real projects.

Anthropic’s Claude for code generation and “human voice” text. When the deliverable is code, or prose that needs to read like a person wrote it rather than a model, Claude is often the better tool. The output has a less synthetic cadence, and on coding tasks the difference shows up in how few corrections I need to make. I’ve started projects in Claude Code, liked the decisions it made, and just stuck with it — running OpenAI as the fallback rather than the primary. That’s a meaningful inversion of my usual setup, and it happened because the quality on that specific axis was better. If you’re weighing this directly, the tradeoffs come down to Claude’s edge on code generation and human-sounding prose versus OpenAI’s broader model lineup and tooling.

OpenRouter when one project needs many models. If a single application has heterogeneous tasks — cheap classification here, premium reasoning there, a specialized model for one feature — routing through OpenRouter lets me hit dozens of models behind one API and one bill. The abstraction cost is real (you give up some provider-specific features), but the operational simplicity of not managing five vendor relationships often wins. It’s also the cleanest way to A/B test models from different providers without rewriting your client.

The decision tree I actually use:

1. Code or human-sounding prose is the core deliverable? → Try Claude first.
2. One app, many different model needs? → OpenRouter.
3. Everything else, especially when I want breadth from a single mature provider? → OpenAI.

For a wider comparison across the field, I keep a running breakdown in the best LLM APIs for developers.

What OpenAI does better than anyone

Credit where it’s due — these are the things that keep OpenAI as the default even when competitors win on individual axes:

• Tooling maturity. The SDKs, the docs, the playground, the dashboard for usage and limits. Everything is a step ahead of most competitors. You spend less time fighting the platform.
• Structured outputs. Schema-enforced JSON is genuinely reliable. This single feature removed an entire class of parsing bugs from my pipelines.
• The Realtime API. For voice agents, the speech-to-speech latency is class-leading. Building the same experience by stitching together separate STT, LLM, and TTS services introduces lag you can’t fully engineer away.
• Ecosystem gravity. When you hit a weird edge case, someone has already posted about it. That community knowledge compounds and saves real debugging hours.

The verdict

If you’re starting a new project and you don’t have a strong reason to do otherwise, build on the OpenAI API. The combination of breadth, tooling, and ecosystem makes it the lowest-risk default, and the consolidation of every modality behind one key has real operational value.

Default to GPT-5.5 for quality-sensitive work, drop to a smaller model for bulk data processing, and use the Batch API for anything offline to halve your costs. Architect for rate limits and retries from the first commit, not after your first production incident.

But hold your defaults loosely. If your core deliverable is code or natural-sounding prose, give Claude a serious trial — it has won that contest for me more than once. If one app needs many models, route through OpenRouter and skip the vendor-management overhead. The right answer is the model that passes your evals at the price you can sustain, and OpenAI earns its default status precisely because it’s the easiest place to find that answer — not because it’s always the answer.