VicEdit supports three types of visual references, each addressing a critical limitation of text-only instructions.
Problem solved: Text-only baselines suffer from stylistic drift. The single image reference precisely anchors the target texture, enabling coherent style transfer.
Problem solved: Linguistic instructions specify what but not where to edit. The image pair reference intuitively defines spatial anchors that text-only methods fail to exploit.
Problem solved: Complex motion dynamics cannot be described by text. The video pair reference provides temporal motion cues that enable vibrant, dynamic edits.
VicEdit architecture. Visual references (single image, image pair, video pair) are processed by Modality-Adaptive Semantic Distillation (MASD) to extract modality-specific semantic tokens. These tokens are then synergistically integrated with textual instructions through Dual-Context Injection (DCI) via cross-attention into the video DiT.
Data construction pipeline comprising four stages: Source Data Filtering, Instruction Synthesis, Visual Reference Generation, and Quality Verification.
Statistics of VicEdit-400K: theme analysis, prompt length distribution, frame distribution, editing task distribution, and prompt word cloud.
VicEdit-Bench comprehensively evaluates visual in-context editing across three reference modalities and diverse task types.
VicEdit handles diverse editing tasks across both basic instruction editing and visual in-context editing. Each demo shows the source video, the visual in-context reference, and VicEdit's output.
[background_change] Edit prompt: Replace the background with a dynamic tropical beach scene...
[local_change] Edit prompt: Replace the man with an elderly gentleman with silver hair and gentle wrinkles...
[creative_edit] Edit prompt: Transform the water bottle to a glowing miniature galaxy.
[subtitle_edit] Edit prompt: Remove the subtitles at the center of the video.
[inpainting] Edit prompt: The sun is setting over a field of wheat.
[outpainting] Edit prompt: A view of a train track and a road.
VicEdit outperforms existing methods on both visual in-context editing (VIC) and basic instruction editing tasks.
Quantitative comparison on basic instruction editing tasks. VicEdit achieves competitive or superior performance across all metrics.
Quantitative comparison on VicEdit-Bench for visual in-context editing. VicEdit significantly outperforms all baselines across three reference modalities.
@misc{vicedit2026,
title={VicEdit: Learning to Edit Videos from Visual In-Context Examples},
author={Wang, Yuji and Hu, Teng and Chen, Yuheng and Yi, Ran and Feng, Han and Cao, Weijian and Wang, Chengjie and Ma, Lizhuang and Zhang, Jiangning},
year={2026},
eprint={arXiv:XXXX.XXXXX},
archivePrefix={arXiv},
primaryClass={cs.CV}
}