Streaming Generative Video Compression with Embedding Alignment and In-Context Propagation

Zhitao Wang1 Hengyu Man1 Han Liu1 Wenrui Li1 Debin Zhao1 Xiaopeng Fan1,2,3
1 Harbin Institute of Technology 2 Harbin Institute of Technology Suzhou Research Institute 3 Peng Cheng Laboratory

01 / Reconstructions

Video comparisons

Choose a reference. Drag the divider to compare it with SGVC.

SGVC first P-frame preview Reference first P-frame preview
Reference
SGVC
Loading videos…
0:00 / 0:00

02 / The idea

Motivation

From latent Video DiT
to frame-by-frame Pixel DiT.

Image diffusion models have limited native temporal modeling. Clip-wise video DiTs capture richer spatiotemporal dependencies, but require multi-frame buffering and a VAE decoder to reconstruct pixels.

SGVC adapts a pretrained latent video DiT into a causal pixel-space refiner. It aligns codec features with the DiT's input embeddings, and reconstructs each frame directly through a lightweight pixel reconstruction path.

03 / The method

Framework

04 / The results

BD-rate comparison

BD-rate (%) relative to S²VC. Negative values indicate rate savings; bold marks the best result in each column.

BD-rate (%) comparison with generative video codecs
MethodHEVC-BUVGMCL-JCV
LPIPSDISTSFIDLPIPSDISTSFIDLPIPSDISTSFID
S²VC (Anchor)+0.00+0.00+0.00+0.00+0.00+0.00+0.00+0.00+0.00
GLC-Video+16.76−15.23−39.05+45.26−42.52−40.29+14.85−23.08−38.28
YODA+22.05−83.87−83.70+43.86−84.18−84.48+16.12−75.89−80.57
GVC-1D−43.40−61.27−83.09−28.68−58.21−73.09−48.54−61.82−76.41
GVC-RT−34.28−48.97−53.87−43.76−54.46−61.33−33.74−29.84−24.22
SGVC−28.46−66.48−88.23−50.96−85.41−91.62−28.40−64.02−87.61

Scroll horizontally to view all datasets

Evaluation: first 96 frames per sequence · Intra period 32

05 / Visual results

Qualitative comparison

View full resolution