InfiniteTalk vs LipForcing Review: 15 Video Tests

Model: Lip Forcing, released by the KAIST AI / AIPARK research team
Scope: real inference on a single RTX 4090-class server, 15 generated artifacts using nine ordinary-person project-page references, multilingual speech, audio edge cases, multi-person boundaries, and deployment readiness.

Executive verdict

LipForcing is an audio-driven video-to-video lip-synchronization model. It takes a reference talking-head video and an external speech waveform, then generates a new video in which the visible mouth is driven by that speech. It is not a text-to-video model, image-to-video model, or general digital-human generator. It is also not a conventional prompt-controlled model: the repository exposes a text condition, but the released inference recipe normally uses a precomputed embedding for the default “a person talking” condition. The main control signal is the audio, while the reference video supplies identity, pose, lighting, and background.

The tested outputs do not show a hidden “flower-screen” failure. The files decode correctly, contain normal audio, and preserve identity well in the controlled frontal cases. The generated face is soft because the released 14B student generates a 512×512 face region and composites it back into the source frame. In other words, the model is an offline lip-sync research component, not a general-purpose “make any person speak beautifully” generator.

The replacement matrix is best interpreted as an offline or queued benchmark under strict input rules: one clear frontal face, stable framing, good lighting, clean single-speaker audio, and a soft 512×512 mouth-region result. It measured approximately 0.059× real time, with a peak of 38,036 MiB out of 49,140 MiB, or about 77.4% of the card, using precomputed text embeddings and the supported non-streamwise path. This is far from interactive latency.

What the model actually does

The correct mental model is “audio-conditioned mouth replacement with temporal video context.” The input video is not merely a portrait used once. The pipeline detects and aligns the face, encodes the reference video chunk by chunk, uses wav2vec features derived from the speech, predicts a short autoregressive latent block with two denoising steps, decodes it, and pastes the generated face region back into the original frame. The default compositor blends the mouth region so that hair, eyes, clothes, and background remain close to the source video.

This design explains both the strength and the weakness observed in the tests. It is good at keeping the person, framing, and scene stable while changing mouth motion. It is not designed to reconstruct the whole body, redesign a scene, change age or identity, or invent expressive head and hand motion from text. A full-body input can be accepted, but only the detected face is the meaningful generated region; the body remains the original video. If the face is small, turned away, occluded, or poorly lit, the model has less reliable evidence and the result becomes visibly weak.

The repository includes a prompt argument because the underlying Wan/OmniAvatar stack has text conditioning. That does not mean LipForcing behaves like a prompt-first generation product. The released 14B path recommends a precomputed text_emb.pt, which saves approximately 13GB of peak VRAM compared with runtime text encoding. On a 48GB-class card, that is an important deployment constraint. The tested command therefore followed the precomputed-embedding path. Since runtime UMT5-XXL encoding is expected to approach or exceed the available VRAM budget, it was not treated as a safe production tuning method on this server.

What was tested

The evaluation contains 15 artifacts using nine ordinary-person references from the project page and public multilingual speech. It includes:

• fifteen real speech samples: English, Chinese, Japanese, Spanish, German, French, female and male voices, low-volume input, long audio, two-speaker audio, and overlapping speech;

• nine project-page public preview references covering different ordinary presenters, ages, skin tones, indoor scenes, and camera compositions, distributed across the 15 tests;

• one full-body input with a small face, to test whether the model can preserve the body while synchronizing the face;

• media integrity checks with ffprobe/ffmpeg, including video duration, frame rate, codec, audio presence, and decodability;

• resource checks for peak VRAM, GPU utilization, elapsed time, and face-detection provider.

The test matrix deliberately includes cases that are easy to overlook. A model may look convincing on one centered male reference and still fail on female speech, a different language, a low-volume recording, long audio, or more than one speaker. It may also produce a playable MP4 while silently dropping audio. Those failure modes were checked separately instead of being hidden inside one average score.

Work samples and preview table

The links below point to the generated MP4 artifacts retained with this evaluation. They are the actual outputs used for inspection, not screenshots. In a Markdown-capable viewer, the video links can be opened directly.

For an immediate visual check, the contact sheet is also included: 15-test contact sheet. The static sheet is only a convenience; the MP4 files are the authoritative evidence because lip synchronization and temporal flicker cannot be judged from a single frame.

LipForcing versus InfiniteTalk

LipForcing and InfiniteTalk both use speech to drive a human video, but they solve different production problems. Treating them as interchangeable “lip-sync models” leads to the wrong expectation about image inputs, body motion, GPU cost, and acceptable failure modes. InfiniteTalk was released by MeiGen-AI with its technical report and code on August 19, 2025. Its official repository describes sparse-frame video dubbing, unlimited-length video-to-video generation, and image-audio-to-video generation. LipForcing was released by the KAIST AI / AIPARK research team as a causal video-to-video lip-synchronization model; its paper was submitted on June 9, 2026. The dates and capabilities here are taken from the respective first-party pages, while the RTX 4090 numbers in this article are measurements from LipForcing only.

The architectural distinction explains why the two models feel different in practice. LipForcing is a face-centric compositor. It has a small visual responsibility: make the mouth follow the waveform and leave most of the frame alone. That narrow responsibility is valuable when a producer already has the right person, camera, clothes, background, and timing. It also explains the visible limitation in this test: the mouth region can look softer than the source even when the overall identity remains recognizable.

InfiniteTalk is closer to an audio-conditioned video generator. Its sparse-frame design preserves selected reference information while allowing the generator to create coordinated head, expression, posture, and body changes. That is the correct direction for a still image that needs to become a presenter, or for a full-body shot where natural upper-body movement matters. The tradeoff is that more of the frame is generated. Longer sequences must therefore be checked for color drift, camera deviation, hands, body geometry, and identity consistency, not only for lip timing.

The resource comparison must be stated carefully. This article has a real RTX 4090 measurement for LipForcing: 236.28 seconds of output across the 15-sample matrix took about 4,007 seconds of serial wall time, or 0.059× real time, with a same-configuration peak of about 38,036 MiB. I did not rerun InfiniteTalk in this LipForcing replacement matrix, so there is no honest same-server InfiniteTalk latency or VRAM number to place beside it. InfiniteTalk’s official 480P/720P modes, Wan2.1-I2V-14B base, and quantization guidance indicate a serious deployment footprint, but those facts are not a substitute for a controlled 4090 run.

For this evaluation, the comparison is conditional rather than a universal winner. LipForcing is the better engineering fit when the input is already a good talking-head video and the requirement is “change the speech while keeping the shot.” InfiniteTalk is the more capable candidate when the requirement is “turn an image into a talking video” or “make the head and body participate in the performance.” The LipForcing measurements in this article are from the RTX 4090 matrix; InfiniteTalk requires its own deployment, matched inputs, and matched quality/resource tests for a direct numerical comparison.

Ordinary-person reference selection

The second visual set was chosen from the project page’s public preview row because it is more useful for a practical quality judgment than a single polished demo. The nine clips contain ordinary presenters in different rooms, a car interior, a book-review setup, different face sizes, different skin tones, and both close and wider compositions. For the ordinary single-speaker rows, the reference gender now matches the test voice; the two-speaker, multilingual, and overlap rows are kept as explicit boundary cases. The goal is not to make a claim about any person in the source material; it is to see whether the model behaves consistently when the reference changes.

The clips also preserve the imperfections that a real user is likely to bring into a local service. Several frames contain subtitles, branded layouts, or a busy background. Those elements are useful stressors because LipForcing is a compositing system: it should change the mouth region while leaving the surrounding frame intact. A result that keeps the subtitle placement and background stable is more informative than a clean studio crop that hides the paste-back boundary. At the same time, the presence of subtitles means the clips are not a substitute for a clean production asset. They are evidence for model behavior, not recommended source footage.

I resized the wide project-page clips proportionally to 768×432 before inference. This keeps the original horizontal composition and makes the full-frame VAE workload comparable with the square reference tests. The face remains visible, while the server no longer needs to create a latent grid from the original 1920×1080 frame. This distinction matters on a 48GB card: the model’s facial generator operates on a face-centric representation, but the default streaming path still derives the reference latent from the working frame dimensions. A large source frame can therefore trigger a VRAM failure before any visible output is produced.

Across the nine outputs, the files remained playable and the face identity stayed consistent enough for a lip-sync preview. The main recurring weakness was not a broken frame; it was softness around the mouth and lower face, especially when the source person was shown smaller or the original clip was heavily compressed. That pattern is consistent with the model’s 512×512 generated face region and should be treated as a quality limit. For deployment, a source-quality gate should check face size, blur, occlusion, and aspect ratio before the job enters the GPU queue.

Quality findings

1. The “flower-screen” impression is not a decoder crash

The replacement outputs decoded normally at 25 fps, 768×432, with H.264 video and AAC audio. All 15 generated works were playable, contained audio, and preserved identity consistently enough for inspection. The inspected frames did not show a black frame, broken tensor shape, or colored block artifacts indicating a muxing or GPU-memory failure.

That does not mean the visual result is perfect. The mouth and lower face can look soft or slightly over-smoothed, especially when viewed enlarged. This is a quality limitation rather than a corrupted file. A viewer comparing a small preview against a crisp source can reasonably describe the result as blurred, but the remedy is not to repair the MP4 container. The remedy would require a higher-fidelity student, better face-region reconstruction, or a post-processing stage that has not been validated here.

2. Identity and background are stable, but the generated mouth region is softer

Across the ordinary-person references, the surrounding scene, clothing, hair, eyes, and presenter identity remained close to the source. The visible quality loss was concentrated in the generated mouth and lower-face region, especially when the input face was small or the source clip was compressed. This is the expected trade-off of a face-region compositor: it preserves more of the original frame than a full video generator, but the regenerated region may not match the source texture at enlarged preview size.

3. Lip timing is usable for clear single-speaker audio, not a general video-generation guarantee

The single-speaker samples produced usable mouth motion across six languages and both male and female voices when the face was visible and the waveform was audible. The result should be judged as audio-driven lip synchronization, not as full facial acting. The model does not provide speaker diarization, emotion control, or reliable multi-face routing; the two-speaker and overlapping-speech cases therefore remain explicit product boundaries.

4. Input audio quality is a first-order variable

One of the earlier samples used a very low-level English female waveform. The original file had an average level around -67.6 dB and produced a video whose audio was effectively inaudible in ordinary playback. Regenerating the same content after a measured gain adjustment created a normal audible track, and the visual lip motion followed the usable waveform. This is not a model language failure. It is an input loudness failure.

The preprocessing wrapper should therefore normalize or reject audio before inference. The safe policy is not to apply a blind huge gain to every file. Measure RMS or integrated loudness, detect clipping after normalization, convert to mono 16 kHz PCM, trim excessive leading/trailing silence, and preserve the original waveform when it is already healthy. The wrapper should report the applied gain and final peak level in the job record. This one change will prevent a user from blaming the model for a silent result.

5. Language coverage is broader than the initial impression

The matrix includes Chinese, English, Japanese, Spanish, German, and French, with both male and female voices. The model consumes acoustic features rather than a language label, so the practical question is whether phonetic timing produces usable mouth motion. In the tested frontal, single-speaker cases, outputs remained coherent across these languages. The result should not be oversold as language understanding: the model does not transcribe, translate, or verify semantic content. It maps audio timing and spectral cues into mouth motion.

Chinese volume deserves special attention because low-volume Chinese recordings can sound much quieter than the English examples used in many demos. The evaluation therefore treats audibility as a separate pass condition. A visually good clip with a near-silent audio track is not a successful delivered artifact.

6. Multiple speakers are a product boundary, not a tuning failure

The two-speaker and overlapping-speech samples are intentionally included. The standard inference path receives one audio stream and controls one detected face. It does not diarize speakers, select a face per time interval, or synthesize two mouths simultaneously. When two voices overlap, the generated mouth is a mixture of acoustic evidence. When speakers alternate, the single reference face continues to move unless an external segmentation-and-routing layer is added.

This is why a “more aggressive prompt” cannot fix the case. The missing capability is a control architecture: diarization, active-speaker detection, per-speaker reference videos, temporal routing, and an edit decision list. LipForcing itself does not provide speaker separation.

7. Full-body input is accepted, but the model is still face-centric

The full-body test confirms that the pipeline can read a wider frame and keep the body and background from the original video. The output is not a text-to-video full-body animation. Only the detected face region is generated and pasted back. If the face occupies a small fraction of the frame, the visual change becomes difficult to see and the mouth detail has fewer pixels. For a product that needs a clearly readable talking full-body presenter, a face crop or a two-stage composition is more appropriate than relying on the full-body frame alone.

Resource and latency results

The deployment used the released 14B student with precomputed text embeddings, Wan VAE, wav2vec2-base-960h, TAEW, LatentSync mouth mask, and InsightFace buffalo_l preprocessing. The tested stack was Python 3.12.13, PyTorch 2.9.1+cu128, CUDA 12.8, diffusers 0.35.1, and transformers 4.49.0. Face detection ran through CPUExecutionProvider because CUDAExecutionProvider was unavailable in the installed ONNX Runtime package.

The 15 ordinary-person speech samples produced approximately 236.28 seconds of output. The serial replacement run took about 4,007 seconds wall time, or approximately 0.059× real time. A same-configuration memory probe peaked at 38,036 MiB out of 49,140 MiB, about 77.4% utilization. The long multilingual sample initially hit a concurrent-GPU-memory failure and succeeded after the competing workload was released; this is an operational admission-control requirement, not a media-quality failure.

The numbers have two implications. First, the 14B checkpoint does not behave like the project-page headline for a smaller 1.3B student on a larger H200. Second, leaving concurrent GPU workloads running is unsafe for reliable inference. A production scheduler must reserve the GPU for LipForcing jobs or enforce admission control; unrelated workloads should not compete for the last few hundred megabytes.

Official procedure versus this deployment

The command family followed the official repository’s streaming inference entry. It used the released self-contained student checkpoint, the external Wan VAE, wav2vec2 model, TAEW decoder, LatentSync mask, and a precomputed text embedding. The main departures were operational: the server used a Python virtual environment instead of a clean conda environment, face caches were retained for repeated tests, and separate logs were collected for each matrix group.

The most important official constraint was respected: the released 14B student is a two-step distilled model, so the default denoising schedule was not changed to arbitrary diffusion steps. The tests changed supported decoder and compositing options, not the trained schedule. This matters because adding steps or inventing a new prompt does not automatically improve a distilled student; it may only create an unsupported configuration.

Method note

The evaluation separates the model’s documented capabilities from local measurements and places each generated artifact beside its corresponding test condition.

The resulting evaluation states the model’s exact job before discussing results, separates project claims from RTX 4090 measurements, keeps all 15 artifact previews beside the corresponding test rows, and documents the low-volume audio failure.

Operational quality interpretation

The quality result should be read at the intended output scale. LipForcing is not reconstructing a complete high-resolution portrait on every frame. It is generating a compact face region and compositing it into a source video whose own resolution, compression, focus, and lighting remain visible around the generated area. At normal playback size, this strategy can look stable because the original frame supplies most of the visual context. At a large preview size, the viewer notices the difference between the source texture and the generated mouth region. This is why identity preservation and perceived sharpness can receive different judgments at the same time.

Temporal stability is also more important than one attractive frame. A mouth can look plausible in a selected still image while the transition between phonemes feels late, too wide, or overly smooth. The evaluation therefore treats short clips as the primary evidence and still frames only as inspection aids. The absence of a broken frame is a basic integrity condition; it is not a quality score. A production wrapper should keep both kinds of checks separate: media validation confirms that a job completed correctly, while human or automated quality review decides whether the facial motion is acceptable.

The safest product positioning follows this distinction. LipForcing can reduce the manual work needed to synchronize a prepared talking-head source with clean speech. It cannot guarantee that every source video will become a convincing presenter. The source capture still matters: a stable camera, a visible mouth, even illumination, and a sufficiently large face give the model the evidence it needs. Improving those inputs is more reliable than increasing prompt wording or widening the composite mask. Where the source itself is unsuitable, the correct behavior is to reject the job or request a better reference clip rather than hide the limitation with a more aggressive generation setting.

Use LipForcing now only in a controlled offline workflow:

1. accept one frontal, single-person reference video;

2. require a minimum face size and reject heavy occlusion, profile views, and unstable camera motion;

3. convert audio to mono 16 kHz and run loudness/peak validation before inference;

4. serialize jobs because the 14B model occupies almost the entire 4090;

5. use mouth-only compositing by default;

6. retain the original audio and mux it only after video integrity checks;

7. expose the result as a preview that can be rejected, not as an unconditional final asset;

8. record elapsed time, peak VRAM, audio gain, decoder, and input identifiers for every job.

The tested limits are clear: real-time interaction, full-body generation, multi-speaker animation, prompt-controlled identity or scene changes, and broadcast-quality facial detail were not demonstrated. The present deployment is a research demonstration and an offline batch component; improving throughput or facial fidelity would require a smaller or better-fidelity checkpoint, a larger GPU budget, or an additional validated enhancement stage.

FAQ

What does LipForcing actually do?

LipForcing is an audio-driven video-to-video lip-synchronization model. It takes a reference talking-head video and external speech, then regenerates a compact face region while keeping most of the original frame intact. It is not a general text-to-video or digital-human generator.

Does LipForcing support text-to-video or image-to-video generation?

The tested released path requires a reference video with a detectable face. It does not provide the image-to-video workflow tested by InfiniteTalk, and a prompt does not turn it into a text-first video generator.

Where do I get the code and model weights?

The code is available in the official LipForcing repository. The released checkpoint and component links are listed in the official Hugging Face model card. The inference path also needs the external Wan VAE, wav2vec2, TAEW decoder, face mask, and face-analysis assets described by those first-party instructions.

How much GPU memory does the 14B path use?

In this evaluation, the 14B path peaked at 38,036 MiB out of a 49,140 MiB card when using precomputed text embeddings and 768×432 working video. Runtime text encoding has a larger memory requirement, so the exact result depends on resolution, encoder mode, and the rest of the GPU workload.

Can it animate a full-body video or multiple speakers?

It can preserve a full-body source video, but the generated region is still face-centric. The tested path drives one detected face from one audio stream; it does not diarize speakers or route different voices to different faces. Multi-speaker editing needs an external diarization and routing stage.

Why can the output audio sound silent?

The visual model can complete normally even when the input waveform is extremely quiet. One tested English female sample averaged about -67.6 dB and was nearly inaudible until its gain was corrected. Check loudness, peak level, clipping, mono conversion, and 16 kHz resampling before inference.

How is LipForcing different from InfiniteTalk?

LipForcing is narrower: it changes mouth motion in an existing video and preserves the shot more aggressively. InfiniteTalk supports video-to-video and image-to-video generation and can regenerate broader head, expression, and body motion. The two models should be selected by the available visual input and the amount of the frame that may change.

Sources

• Official LipForcing repository

• Official LipForcing project page

• LipForcing paper link from the repository

• Official InfiniteTalk repository

• InfiniteTalk technical report