Back to @hjbhjh's timeline
hjbhjh shared this post · 2h ago
Web
nvlabs.github.io
nvlabs.github.io

Open-H3-Regenerate-2K · SoL-Engine

3 steps Diffusion steps at full resolution

4 K Native output, from 768P to 4K

5–15 s Supported refine length

5.0 × Faster than the official API · single B200

Abstract

Open-H3-Regenerate-2K keeps the H3 transformer and video VAE frozen and adds one refiner LoRA, trained like Stage I of SoL-Refiner on real high-resolution clips paired with degraded copies of themselves. At inference the draft is encoded once; H3-Latent-Upscaler, a community 3D-convolutional network, enlarges the latent straight onto the target grid, with no decode and re-encode; the enlarged latent is re-noised to σ = 0.909 and denoised in three Euler steps (0.909 → 0.857 → 0.667 → 0) with classifier-free guidance off, so each step is one transformer call at full resolution. For these few steps the refiner is merged with LightX2V ’s FL2VA Turbo and the LMS Sharp adapter into a single LoRA.

The official API only accepts unmodified H3 768P output and returns 2K. Open-H3-Regenerate-2K takes a draft at any low resolution, such as 384P, 512P, 576P or 768P, made by H3 or by another video model, and regenerates it natively at 768P, 1080P, 2K (2560 × 1440) or 4K (3840 × 2160), for clips of 5 to 15 seconds in one pass. With SoL-Engine (Sol-Attn sparse attention, MXFP8 linear layers and CUDA graphs) and the fast H3 VAE decoder of [StreamRender-H3](https://nvlabs.github.io/Sana/Sol-Engine/StreamRender-H3/#3-2-fast-causal-h3-vae-self-improving-the-streaming-decoder-with-agent-loops), latent upscaling, the three steps and VAE decoding for a 5-second 2K clip take 22.45 s on one B200 instead of 87.26 s.

On RefinerBench-150, with both systems at 2560 × 1440, ours scores higher than the official API on all three UniPercept axes and on VBench aesthetic quality, imaging quality and subject consistency. In a blinded user study, ours was preferred in 87% of 200 judgments.

Why an Open Regenerate Model?

H3 makes a video in two steps. The base model settles content and motion at low resolution, where sampling is cheap. H3-Regenerate-2K then redraws the draft at 2K, so faces, fabric, fur and foliage hold up at full size.

Only the first step is open. H3-Regenerate-2K is a hosted API without released weights, so people building on the open H3 base model cannot run it on their own GPUs, adapt it, or go beyond what the service offers.

Open-H3-Regenerate-2K fills that gap with an open model for the same step: it starts from the H3 draft, keeps its content and motion, and regenerates the detail at full resolution.

Hands, input
1 / 4

Same frame, same window.

From a Low-Resolution Draft to 2K

Open-H3-Regenerate-2K wraps a frozen H3 transformer and video VAE. It takes the draft and its prompt and returns the same scene at the target resolution.

  1. Encode. The H3 VAE encodes the draft once.
  2. Enlarge. H3-Latent-Upscaler maps the latent onto the target grid.
  3. Regenerate. The enlarged latent is partly re-noised and denoised in three steps.
  4. Decode. The H3 VAE decoder turns the result back into frames; the fast path uses the 24-layer fast H3 VAE decoder from [StreamRender-H3](https://nvlabs.github.io/Sana/Sol-Engine/StreamRender-H3/#3-2-fast-causal-h3-vae-self-improving-the-streaming-decoder-with-agent-loops). Upscaling alone adds no detail. It carries layout, color and motion onto the larger grid; the three diffusion steps draw the surfaces at that size.

2.1Visual Results

The input on the left; the official H3-Regenerate-2K output and ours, both 2560 × 1440, in the middle, with ours always below; on the right, the region under the magnifier in each output. The first group starts from drafts made by the H3 base model, the second from drafts made by other video models (LTX-2 and SANA-Video), prepared at 1024 × 576.

H3 Input

Hover over a video to move the magnifier

H3 base drafts.

Other-Model Input

Hover over a video to move the magnifier

Drafts from LTX-2 and SANA-Video.

2.2From Any Resolution to 4K

Open-H3-Regenerate-2K is not tied to one input or output size. It takes a draft at any low resolution, such as 384P, 512P or 576P, and regenerates it natively at 768P, 1080P, 2K or 4K: the latent upscaler enlarges the draft latent to the target grid, and the three diffusion steps run at that size. Below, a dance draft resized to 4K and the same draft regenerated natively at 4K.

Hover over a video to move the magnifier

Input resized to 4K (top) and native 4K (bottom).

2.35 to 15 Seconds

Clips from 5 to 15 seconds are refined in a single pass. This 15-second clip, 360 frames, is regenerated at 2K in one run.

Hover over a video to move the magnifier

15 s, one pass.

Key Technologies

3.1Training

We train the refiner following Stage I of SoL-Refiner, its high-resolution continual training, with the same recipe and data but with H3 in place of SoL-Refiner’s original base model. Each training pair is built from a real, high-quality clip at 1280 × 704 or 2560 × 1440. The target is the clip itself; the condition is a degraded copy, shrunk by 8× to 64× with an antialiased bilinear filter and resized back. Both are encoded by the H3 VAE.

The H3 transformer stays frozen and only a refiner LoRA is trained, with a flow-matching velocity loss on noise levels from σ = 0.91 down, the same level the three-step inference starts from (Section 3.3). SoL-Refiner’s later stages, RL post-training with ReFL and DMD distillation to a one-step refiner, are not used yet. For inference the refiner LoRA is merged with a community distillation LoRA.

3.2Latent Upscaling

The refiner works on the target grid, so the draft first has to reach the output size. Done in pixels, that is a round trip: decode the low-resolution latent with the VAE, resize every frame, and encode the frames again at 2K or 4K. At these sizes the VAE is slow. H3-Latent-Upscaler, a community model, skips the round trip by enlarging the latent directly with a learned 3D convolutional network. We use it as released. At 2K it maps a 36 × 64 latent grid to 90 × 160, 2.5× per side, leaving the 24 channels and 37 latent frames unchanged; for 4K it is applied twice, each time within its trained range.

3.3Three-Step Regeneration

For speed we build on another community work: LightX2V ’s FL2VA Turbo, a LoRA that lets H3 generate in four steps instead of dozens. We merge it as released, together with the community LMS Sharp adapter by Alissonerdx. A four-step run starts from pure noise, but our refiner LoRA (rank 384, 5,000-step EMA) is trained to start from the upscaled draft partly re-noised at σ₀ = 0.909. The draft already fixes layout and motion, so three steps are enough: 0.909 → 0.857 → 0.667 → 0. Classifier-free guidance is off, so each step is a single transformer call at the target resolution.

3.4Inference Optimization

Starting from a 1344 × 768 H3 latent, latent upscaling to 2560 × 1440, the three refine steps and VAE decoding take 87.26 s without and 22.45 s with SoL-Engine and the fast H3 VAE decoder of [StreamRender-H3](https://nvlabs.github.io/Sana/Sol-Engine/StreamRender-H3/#3-2-fast-causal-h3-vae-self-improving-the-streaming-decoder-with-agent-loops) on a single B200, 3.9× faster.

Sparse attention saves the most: Sol-Attn removes 39.99 s. The fast H3 VAE decoder, the released 36-layer decoder pruned to 24 layers and distilled from it, cuts decoding from 15.0 s to 1.6 s. Lossless kernels, MXFP8 linear layers with a fused SwiGLU quantizer and per-step CUDA graphs account for the rest.

Warm inference on one GPU, median of three runs. Generating the 768P draft, model loading, text encoding and MP4 writing are not included. The DiT modules are measured one at a time and scaled to the measured total; the VAE row is measured directly.

Experiments

We compare against the official H3-Regenerate-2K API with two automatic metrics on RefinerBench-150 and with a user study. RefinerBench-150, from the SoL-Refiner benchmark, has 150 videos covering everyday scenes, sports and dance, animals, vehicles, landscapes, portraits and fast camera motion. Both systems receive the same input and output 2560 × 1440.

4.1VBench

VBench rates six dimensions on a 0–1 scale. Subject and background consistency check that identity and scene stay stable over time, motion smoothness and dynamic degree describe the motion, and aesthetic and imaging quality judge how each frame looks. Following SoL-Refiner, the average is the unweighted mean of the six, not the leaderboard total.

Ours is higher on aesthetic quality, imaging quality and subject consistency. The official API is slightly ahead on motion smoothness and dynamic degree, and background consistency is level.

4.2UniPercept

UniPercept scores perceived quality from 0 to 100 along three axes: aesthetics (IAA), technical image quality (IQA), and structure and texture (ISTA). The average is the mean of the three.

Ours is higher on all three axes, by about eight points on aesthetics and image quality and three points on structure and texture.

4.3User Study

The blinded pairwise study uses 50 cases: 25 RefinerBench-150 videos and 25 prompts shared online, mostly on X. Each trial shows the input as a reference and the two 2560 × 1440 results as A and B in random order. Participants pick the one that looks better, weighing clarity, detail, naturalness, artifacts and temporal stability. Each session covers ten cases; 20 completed sessions give 200 judgments.

Acknowledgements

  • Equal contribution

Citations

If Open-H3-Regenerate-2K helps your work, please cite the SoL-Refiner paper together with this blog.

BibTeX · SoL-Refiner + this blog

@misc{liu2026solrefiner,
title = {{SoL-Refiner}: Speed-of-Light One-Step Refinement for High-Resolution Video},
author = {Haozhe Liu and Tian Ye and Shuchen Xue and Yitong Li and
Junsong Chen and Haopeng Li and Jincheng Yu and Duomin Wang and
Ruihua Zhang and Lei Zhu and Song Han and Enze Xie},
year = {2026},
eprint = {2609.37969},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2609.37969}
}

@misc{gao2026openh3regenerate2k,
title = {Open-H3-Regenerate-2K: High-Resolution Video Refinement with Few-Step Diffusion},
author = {Sensen Gao and Haozhe Liu and Yitong Li and Haopeng Li and Tian Ye and Jia-Wang Bian and Song Han and Enze Xie},
year = {2026},
howpublished = {\url{https://nvlabs.github.io/Sana/Open-H3-Regenerate-2K/}},
note = {Project page}
}
```