Together AI review

Together AI is a platform for building and running open models, not just a hosted model API. It combines serverless inference, batch inference, reserved inference capacity, dedicated inference hardware, GPU compute, and managed fine-tuning. The pitch is to use one platform for model access and the infrastructure around it, from on-demand API calls to dedicated GPUs.

For a team weighing a switch, the key distinction is how much infrastructure it wants to manage. Serverless inference runs models through APIs without requiring you to manage infrastructure. Dedicated inference puts models on dedicated infrastructure or custom hardware; GPU compute can scale from self-serve clusters to thousands of GPUs. Batch inference is aimed at asynchronous workloads and supports up to 30 billion tokens per model, according to the published product details.

Pricing is usage-based rather than a set monthly subscription. The pricing data lists no free plan and does not publish a starting price.

The useful part is the range of deployment choices. Teams can start with serverless APIs, shift asynchronous work to batch inference, or reserve capacity through Provisioned Throughput. For workloads that need isolation or specific hardware, Dedicated Inference offers single-tenant GPU options. Fine-tuning and GPU compute extend the same platform beyond inference.

The pricing page lists text-model rates per 1 million tokens, audio transcription from $0.0015 per minute, and video models from $0.115 per video. Llama 3 8B Instruct Lite is listed at $0.14 for both input and output. Rates depend on the model and workload, so these examples are not a universal cost estimate.

Some Reddit users describe Together AI as fast and useful for sending inference workloads to hosted GPUs rather than tying up their own hardware. Another commenter contrasts its latency with OpenRouter’s broader provider selection. Those are user reports, not a guarantee; they do point to a practical trade-off: Together AI’s appeal is its own infrastructure and production-oriented options, while a multi-provider router may offer more choice.

Usage-based pricing makes costs harder to compare with a flat subscription. The pricing data gives no published starting price, free plan, or free-trial terms. Batch inference has usage-based model rates, but no separate batch discount or plan fee is published. Some models list $0.00 input and output, which does not establish that every model or workload is free.

Provisioned Throughput has a 99% uptime SLA, but the pricing page’s example is an estimate, not a standard monthly plan: 10 PTUs are estimated at $21,600 per month, assuming about 43,800 minutes. Capacity per PTU depends on model and token type. For dedicated inference, the listed on-demand rates are $5.49 per GPU-hour for NVIDIA HGX H100 and $8.99 for HGX B200. H200, B300, GB200 NVL72, GB300 NVL72, and reserved pricing require contacting sales.

Image pricing can add costs beyond its default steps, and those amounts are not published. Estimate your own model mix and traffic before switching; the listed rates alone do not settle total cost.

Together AI does not list a fixed monthly starting price, a free plan or annual billing. Serverless Inference is usage-based: the pricing page lists Llama 3 8B Instruct Lite at $0.14 per 1M input tokens and $0.14 per 1M output tokens. Audio transcription starts at $0.0015 per minute; video models start at $0.115 per video. Some models list $0.00 input and output, so check the rate for the exact model you plan to use.

Batch Inference is also usage-based, with no separate plan fee or batch discount published. For reserved capacity, the page estimates 10 PTUs at $21,600 per month, based on about 43,800 minutes; capacity per PTU varies by model and token type. That is a substantial commitment compared with on-demand use. Dedicated inference starts at $5.49 per GPU-hour for an NVIDIA HGX H100 and $8.99 for an HGX B200; other listed GPUs and reserved pricing require a sales quote. Extra image-generation steps cost more, but the amounts are not published.

Together AI combines hosted model APIs, batch jobs, reserved inference and GPU infrastructure. Before moving production traffic, inventory which of those services you use and estimate token volume, GPU hours and any reserved capacity. The pricing page’s PTU example shows why: 10 PTUs are estimated at $21,600 monthly, while dedicated GPU use is billed hourly. The platform highlights open models, but the published materials do not document migration tools, data export formats or compatibility guarantees. Confirm those details before treating a move as plug-and-play.

Moving away also means planning replacements for any Together AI inference endpoints, batch workflows, fine-tuning or GPU workloads you rely on. The published sources contain no Reddit reports, so we can’t attribute specific reasons for leaving to users. For a decision, ask Together AI how to retrieve stored data and model artifacts, and verify that your replacement supports the models and workload patterns you need. If multi-provider routing or a fixed monthly bill is essential, the stated product fit may be a reason to compare alternatives.