The constraint is the whole conversation
A talking avatar has to make several things convincing at once: the words, the voice, the face and the timing between them. A short generated clip can demonstrate visual quality while revealing little about whether the same model could sustain a live conversation.
Open models are making real progress. Some now report throughput comfortably above normal video frame rates on powerful consumer GPUs. The narrower requirement remains difficult: convincing animation, low response latency and sustained output on a GPU with perhaps 4–8 GB of memory. Those are separate constraints, and a model satisfying one may miss the others.
When building the interface, model selection becomes a decision about the entire conversation pipeline. The practical work is deciding how much quality the presentation needs, which stages share hardware, and where the application can tolerate waiting. A convincing prerecorded clip settles none of those decisions.
Several technologies sit behind the same label
“Talking head” can describe quite different systems. Lip synchronisation adjusts a mouth region to match speech, often preserving an existing video. Portrait animation transfers movements to a source face. Audio-driven generation attempts to produce the facial motion itself from speech. An application may combine these approaches.
LivePortrait illustrates the distinction. Its official implementation animates a source portrait using driving video or motion templates. That is useful machinery for a digital face, but a voice application still needs a way to turn incoming speech into the motion that drives it. A fast portrait renderer does not establish the speed of that additional stage. See the LivePortrait implementation.
For a fixed presenter reading prepared text, an existing video with new lip movements may be sufficient. An interactive assistant has a harder job: pauses, turn-taking, changes of expression and interrupted sentences all need to remain believable. The intended interaction should determine the model family before a benchmark determines the hardware.
Low memory and real-time output are different achievements
MuseTalk makes this particularly clear. Its repository reports more than 30 FPS on an NVIDIA Tesla V100. It also describes a low-memory example on an RTX 3050 Ti Laptop GPU with 4 GB of VRAM: under that Windows configuration, an eight-second video takes about five minutes to produce. These are different configurations, not a controlled hardware comparison. They nevertheless show why the memory claim and the speed claim cannot be combined. See the MuseTalk performance notes.
The same documentation describes a 256 × 256 face region and a real-time workflow that prepares avatar material in advance. Those details matter when interpreting an output video: a large final frame need not mean every pixel was generated at that resolution, and prepared assets can move work outside the live path.
A deployment budget therefore needs two measurements. First, does the chosen configuration fit, including temporary allocations? Second, can it keep producing frames quickly enough after the rest of the application is added? Loading weights successfully answers only the first question.
Recent models move the trade-off
SoulX-FlashHead, released in February 2026, provides a more recent example. Its authors report 96 FPS for the Lite variant on one RTX 4090, or three concurrent streams above 25 FPS. The Pro variant reports 10.8 FPS on one RTX 4090 and more than 25 FPS on two RTX 5090s. The quality-oriented variant has a substantially different hardware requirement. See the SoulX-FlashHead benchmarks.
| Implementation | Published result | What the result establishes |
|---|---|---|
| MuseTalk | 30+ FPS on a Tesla V100; a separate 4 GB laptop example runs much slower | Fast inference and low-memory execution are both possible under different conditions |
| SoulX-FlashHead Lite | 96 FPS on one RTX 4090 | Considerable real-time headroom on the reported hardware |
| SoulX-FlashHead Pro | 10.8 FPS on one RTX 4090; 25+ FPS on two RTX 5090s | The higher-quality configuration changes the deployment budget |
These are author-reported results from the repositories reviewed on September 13, 2026, rather than Quandelia measurements or a like-for-like ranking. They do not establish that the same throughput is available within a 4–8 GB memory budget. Checkpoint, precision, resolution, preprocessing and concurrent workloads must accompany any reproduced result.
Frame rate hides the beginning of the conversation
At 25 FPS, a stream needs a new frame roughly every 40 milliseconds. At 30 FPS, that interval is about 33 milliseconds. But throughput describes how fast the system keeps working once it has started. A renderer can sustain 30 FPS and still make someone wait two seconds before the first visible response if its pipeline buffers that much audio.
The full path includes speech generation, any audio lookahead required by the animation model, rendering, video encoding and transport. Some stages can overlap. Others wait for enough input to become available. The relevant delay is what the person experiences from the end of their turn to the first useful response.
Interruptions expose another weakness that a demo clip can hide. When someone speaks over the avatar, queued speech and associated frames need to be cancelled together. Continuing to animate a sentence after its audio has stopped is a coordination failure even if every individual frame looks good.
Measure first-response latency separately from sustained delivered FPS. Also inspect stalls: an average of 30 FPS can conceal bursts of fast rendering separated by visible freezes.
The application competes for the same GPU
A modest GPU may also be expected to run speech recognition, voice synthesis or a language model. A memory figure measured for the avatar alone leaves that competition unresolved. Two sessions can add another set of buffers and state even when some model weights are shared.
This changes what “cheap to run” means. A lightweight renderer paired with a separate voice service may be a better initial design than placing the entire conversation stack on one small card. Conversely, transferring audio and video between services introduces latency and operational dependencies of its own.
The useful comparison is cost per concurrent session meeting a defined quality and latency target. GPU price per hour is only an input to that calculation. The cloud GPU operating model becomes part of the product: a warm conversational session has different economics from a video job that can wait in a queue.
What to evaluate before choosing a stack
Start with the actual presentation: a small face beside a workspace, a full-screen presenter or a video-call participant. Test at that displayed size. A defect that is invisible in a compact assistant may be distracting in a close-up.
Run complete conversations rather than a collection of favourable sentences. Include silence, numbers, different speaking speeds, unfamiliar pronunciations and interrupted turns. Record peak memory, first-frame delay, sustained delivered FPS and the slowest periods across a longer session. Repeat with the intended concurrency and record the exact hardware and software configuration.
Finally, separate availability from permission to ship. Review the code, checkpoint and dependency licences individually; a public repository alone is not a commercial-use guarantee. Use a face and voice for which the application has the necessary rights.
The strongest near-term approach is to define a restrained interaction and validate it end to end. The frontier is moving quickly, but the deciding question is already concrete: can this particular face keep up with this particular conversation, at the cost and hardware budget the product can sustain?
Cover: AI-generated illustration for Quandelia, not output from the models discussed.