Begin with the job that has to finish
A ten-minute video render and a talking avatar can use similar hardware while needing very different infrastructure. The render may tolerate queueing and a retry. The avatar needs to respond while someone is waiting, then hold a session without visible interruption.
For the first workload, unused capacity is a cost to eliminate. For the second, available capacity may be part of the service being sold. Comparing them through the cheapest advertised GPU-hour misses the decision.
Write down three requirements before choosing a provider: how quickly work must start, how long it occupies the GPU, and what happens if the worker disappears. These determine whether a persistent machine, an interruptible worker pool or a serverless endpoint is a sensible starting point.
The billing unit changes the comparison
Provider documentation reviewed on September 13, 2026 describes several distinct charging models. Rates and terms can change; the linked pricing pages are the references for a purchase.
| Service | What is metered | What to include in the estimate |
|---|---|---|
| Runpod Pods | Allocated GPU compute; its pricing documentation specifies per-second billing | Time the Pod is running, plus storage and any commitment terms |
| Runpod Serverless | Worker lifetime, rounded up to seconds | Initialisation, execution and idle timeout; active workers stay running |
| SaladCloud Container Engine | Running container time, with hardware and priority determining the rate | CPU and RAM are included in the selected configuration; interrupted work may need repeating |
| Modal | GPU, CPU and memory resource time | The GPU rate alone is incomplete; warm GPU capacity can also be billed |
| Replicate | Depending on the model, hardware time or an output unit | Public-model and private-deployment billing differ; inspect the specific model |
Runpod's Pod pricing suits a familiar machine-allocation model: a quoted hourly rate accumulates while compute is allocated. Storage has its own lifecycle and can remain chargeable after compute stops. For a continuously occupied worker, that arrangement is relatively easy to budget.
Its Serverless pricing counts the worker from start to stop, including model loading and idle time before shutdown. Flex workers can scale to zero; active workers remain available and running. Serverless removes some machine management, but does not make every second outside inference free.
Salad's billing documentation excludes allocation and image-download time from charges. Billing starts when a container runs. A model download or initialisation performed by the running application can therefore have a different cost from the platform downloading the container image.
An attractive GPU rate is an incomplete quote
Modal's published L4 rate, at the time of review, is $0.000222 per second: about $0.80 per hour for the GPU. CPU and memory are priced separately. That figure is useful for estimating a configuration, but it is not the total application price or a performance-equivalent comparison with a different GPU elsewhere.
Replicate adds another distinction. Its pricing catalogue includes models charged by outputs such as images or seconds of generated video, alongside hardware-time pricing. A second of finished video is a product unit; it does not say how many compute seconds produced it.
According to Replicate's billing guide, time-priced public models generally charge for active processing, while private models and deployments normally also incur setup and idle costs while online. There are exceptions for particular model types. A successful prototype on a public endpoint is therefore not enough to estimate the economics of a private deployment.
Compare the bill for the same accepted result: the same model, output quality, duration and deadline. Faster hardware can cost less per finished job despite a higher hourly rate. A cheaper attempt that misses the deadline or produces unusable output has still consumed resources.
A simple break-even calculation
Consider two hypothetical offers delivering identical throughput. These are illustrative prices, not provider quotes.
A persistent worker at $0.40 per hour costs $288 over a 720-hour month. An elastic worker at an equivalent $0.80 per billed hour costs $96 for 120 billed hours. They reach parity at 360 billed hours; at 600 hours, the elastic option costs $480.
The important input is billed hours. If 120 hours of useful inference require another 80 hours of loading, waiting or keeping capacity warm, the elastic bill becomes $160. Add storage, network transfers and any separately priced CPU or memory before comparing totals.
This calculation assumes matching performance and excludes commitments, credits and taxes. Its purpose is to expose the utilisation threshold. A few weeks of measured demand can be more useful than a long list of advertised GPU prices.
Cold starts are an architectural choice
A cold worker may need to obtain an image, load weights and prepare its runtime before accepting useful work. The user experiences the complete delay, regardless of which parts the provider bills.
Keeping workers warm exchanges some of that delay for reserved capacity. Modal's cold-start guide makes this explicit: minimum containers, spare buffers and longer scale-down windows affect the balance, and idle GPU reservations can incur charges. Similar reasoning applies when choosing active capacity on Runpod.
For occasional batch jobs, a queue can absorb startup delay. For a live talking-head application, a warm session pool may be necessary. Measure the first usable response as well as the duration of inference. Increasing a concurrency limit should not be treated as a capacity reservation.
Reliability starts at the interruption boundary
Salad's architecture documentation describes distributed consumer machines with potentially unannounced interruptions. Local container storage is ephemeral. Automatic replacement restores a worker; the application still needs a way to recover its job and output.
A suitable batch design records the request durably before dispatch, assigns an idempotency key and writes outputs to durable storage. The worker acknowledges completion only after the result is committed. If it disappears after uploading but before acknowledging, a replacement can recognise the completed request instead of publishing a duplicate. Long jobs may also need checkpoints so recovery does not always restart from zero.
Runpod offers Secure Cloud and Community Cloud infrastructure, with different hosting arrangements. Evaluate the selected offer and any contractual service commitment rather than attaching one assumed uptime figure to the entire provider. For every platform, distinguish an available endpoint from a successful model response within your deadline.
Interactive sessions need a different recovery plan. Replaying a video job is possible; replaying half a conversation is usually unacceptable. Admission control, spare warm capacity, reconnect behaviour and a voice-only fallback may matter more than shaving the last few cents from the GPU rate.
What a useful pilot should measure
Run a small workload sample that includes quiet periods, bursts and an interrupted worker. Record queue delay, first-response latency, completed outputs, rejected outputs, retries and the resulting bill. Test with the actual image and weights; a lightweight demonstration container can conceal most of the startup cost.
Place storage deliberately. Repeatedly moving large inputs or weights across regions can add latency and network charges. Check where the selected GPU is available, how the application reaches it, and which data must remain within a particular location.
The decision should emerge from those measurements. Persistent capacity can suit steady demand. An elastic endpoint can suit intermittent work. Interruptible capacity can be valuable when jobs recover cleanly. The best fit is the offer whose failure behaviour and billing model match the application you are prepared to operate.
Cover photograph: Taylor Vick / Unsplash.