MovingColor: Seamless Fusion of Fine-grained Video Color Enhancement

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-09 11:08:01

Abstract: Fine-grained video color enhancement delivers superior visual results by making precise adjustments to specific areas of the frame, maintaining more natural color relationships compared to global enhancement techniques. However, dynamically applying these specific enhancements can lead to flickering artifacts and unsatisfactory color blending at object boundaries, issues caused by the coarse and unstable masks produced by current video segmentation algorithms. To overcome these challenges, we introduce MovingColor, featuring a novel self-supervised training approach that leverages large-scale video datasets. This approach redefines color fusion as a generation process using original full-frame textures and color editing information from non-edge areas. We address spatio-temporal inconsistencies with a spectral-spatial hybrid encoder that captures multi-scale spatial and frequency features, thus enhancing color adjustments in complex scenes. Additionally, our global-local feature propagation module, incorporating Transformer blocks, consolidates spatio-temporal contexts to ensure consistency among frames. Both quantitative and subjective evaluations validate the effectiveness of MovingColor in delivering state-of-the-art spatio-temporal consistency for video color enhancements, adhering closely to the intended color editing operations. These results demonstrate that MovingColor can effectively enhance fine-grained video color grading, making it more efficient and accessible to a wider range of users. We will release the code to support further research and practical applications.

Main paper: 
 introduction: 
 Video color enhancement aims to impro of videos through targeted color adjustm enables more engaging and vibrant vid ous applications such as post-productio entertainment. While photo enhancem advanced rapidly, video color enhancem challenge, requiring both spatial and temA key di culty lies in performing sele color modifications while maintaining coh Prior arts have explored global color tran enable localized edits [1] . Segmentation like Mask2Former [2] can indicate reg stability for consistent video modificatio manual user inputs for tracking provid scalability and e ciency. Video color editing aims to enhance the visual quality of vide through targeted regional color adjustments. However, applyin edits dynamically across frames often causes flickering artifacts d to mask instability. Furthermore, coarse masks struggle to blen colors naturally across object boundaries. This paper presen MovingColor to address these challenges and enable seamless fin grained video color editing. We propose a self-supervised trainin strategy that synthesizes color transformations and object mas on videos to teach a network to fuse edits. A key innovation formulating color blending as a fills generation task, rather tha binary completion, better retaining details. Explicit global an local processing branches allow multi-scale representation. Furth a temporal transformer propagates relevant features across fram for coherence. Quantitative and subjective evaluations demonstra MovingColor's capabilities in versatile video color manipulatio with state-of-the-art spatial and temporal consistency. The syste significantly advances controllable, professional-grade video colPermission to make digital or hard copies of all or part of this wo for personal or classroom use is granted without fee provided th copies are not made or distributed for profit or commercial advanta and that copies bear this notice and the full citation on the first pa Copyrights for components of this work owned by others than t author(s) must be honored. Abstracting with credit is permitted. T copy otherwise, or republish, to post on servers or to redistribute to lis requires prior specific permission and/or a fee. Request permissio from permissions@acm.org. © 2024 Copyright held by the owner/author(s). Publication righ licensed to ACM. ACM 0730-0301/2024/8-ART111 https://doi.org/XXXXXXX.XXXXXXX AC Video color editing aims to enhance the visual quality of videos through targeted regional color adjustments. However, applying edits dynamically across frames often causes flickering artifacts due to mask instability. Furthermore, coarse masks struggle to blend colors naturally across object boundaries. This paper presents MovingColor to address these challenges and enable seamless finegrained video color editing. We propose a self-supervised training strategy that synthesizes color transformations and object masks on videos to teach a network to fuse edits. A key innovation is formulating color blending as a fills generation task, rather than binary completion, better retaining details. Explicit global and local processing branches allow multi-scale representation. Further, a temporal transformer propagates relevant features across frames for coherence. Quantitative and subjective evaluations demonstrate MovingColor's capabilities in versatile video color manipulations with state-of-the-art spatial and temporal consistency. The system significantly advances controllable, professional-grade video colorVideo color enhancement aims to of videos through targeted color ad enables more engaging and vibra ous applications such as post-pro entertainment. While photo enha advanced rapidly, video color enh challenge, requiring both spatial a A key di culty lies in performin color modifications while maintaini Prior arts have explored global col enable localized edits [1] . Segmen like Mask2Former [2] can indica stability for consistent video mod manual user inputs for tracking p scalability and e ciency. Fine-grained video color enhancement delivers superior visual results by making precise adjustments to specific areas of the frame, maintaining more natural color relationships compared to global enhancement techniques. However, dynamically applying these specific enhancements can lead to flickering artifacts and unsatisfactory color blending at object boundaries, issues caused by the coarse and unstable masks produced by current video segmentation algorithms. To overcome these challenges, we introduce MovingColor, featuring a novel self-supervised training approach that leverages large-scale video datasets. This approach redefines color fusion as a generation process using original full-frame textures and color editing information from non-edge areas. We address spatio-temporal inconsistencies with a spectral-spatial hybrid encoder that captures multi-scale spatial and frequency features, thus enhancing color adjustments in complex scenes. Additionally, our global-local feature and tracking techniques, resulting in imprecise and flickering masks that lead to spatial and temporal inconsistencies (Figure 1 and 2 ).Existing approaches for video color enhancement, such as Harmonizer [14] and DeepLPF [24] , attempt to address these challenges by adapting image-based techniques or employing localized filters. However, these methods struggle to maintain consistency across video frames and often fail to preserve the intended color edits. Additionally, video matting techniques like MODNet [15] are specialized for specific tasks and do not provide generalizable solutions for color fusion.To address these issues, we propose MovingColor, a novel selfsupervised learning method for video color fusion that effectively ensures both spatial and temporal consistency. MovingColor formulates color fusion as generating natural fusion results given the original full texture information, color editing information in the non-edge area, and the mask edge, leveraging vast amounts of unlabeled video data. MovingColor employs a spectral-spatial hybrid encoder, combining convolutional and Fast Fourier Convolutional networks to capture multi-scale spatial features and frequency information. A global-local feature propagation module with Transformer blocks aggregates spatio-temporal contexts across frames for consistency. MovingColor captures meaningful spatial and temporal features from the video data, encoding both global and local information across frames to enable consistent color fusion.To comprehensively evaluate color fusion performance, we introduce the D5 dataset, synthesized using 3D software. It features diverse 4K video clips from various scenes, with ground truth mattings for all objects. Extensive experiments demonstrate that Mov-ingColor achieves state-of-the-art results, outperforming existing color manipulation, harmonization, and video consistency methods on the D5, DAVIS [26] , and YouTube-VOS [35] datasets. Furthermore, MovingColor exhibits robustness to varying edge ratios, resolutions, and color adjustments, highlighting its adaptability to diverse real-world scenarios.The main contributions are: 1) A novel self-supervised learning approach for spatially and temporally consistent video color fusion;2) The design of MovingColor with the spectral-spatial hybrid encoder and the global-local feature propagation module for capturing multi-scale features and aggregating spatio-temporal contexts; 3) The D5 dataset for comprehensive evaluation of color fusion performance; and 4) State-of-the-art results on 3 datasets, outperforming existing methods in quantitative metrics and user studies.Video color enhancement aims to improve the visual quality of videos through targeted color adjustments. This capability enables more engaging and vibrant video content for various applications such as post-production, social media, and entertainment. While photo enhancement techniques have advanced rapidly, video color enhancement remains an open challenge, requiring both spatial and temporal constraints.A key di culty lies in performing selective, region-specific color modifications while maintaining coherence across frames. Prior arts have explored global color transforms, which fail to enable localized edits [1] . Segmentation masks from models like Mask2Former [2] can indicate regions, but often lack stability for consistent video modifications [3] . Alternatively, manual user inputs for tracking provide accuracy yet limit scalability and e ciency. 2 : Fine-grained color enhancement is crucial for maintaining accurate color relationships, but it often introduces inconsistencies. MovingColor effectively resolves these issues in natural fusion of fine-grained color edits.better than global enhancement methods, which can introduce unnatural hues such as blue in fields or pink in skies. However, as Figure 2b highlights, this approach introduces challenges related to mask precision. Current video matting technologies do not yet provide accurate tracking, leading to spatial inconsistencies from imprecise masks-evident from stark texture variations where highfrequency image components are improperly altered-and temporal inconsistencies from flickering masks, as shown by the non-smooth temporal profiles created by stacking horizontal pixel rows from consecutive frames. Our method, MovingColor, addresses these issues effectively, ensuring both spatial and temporal consistency as evidenced in the figures.Video color enhancement aims to improve the visual of videos through targeted color adjustments. This cap enables more engaging and vibrant video content fo ous applications such as post-production, social medi entertainment. While photo enhancement technique advanced rapidly, video color enhancement remains a challenge, requiring both spatial and temporal constrA key di culty lies in performing selective, region-s color modifications while maintaining coherence across Prior arts have explored global color transforms, which enable localized edits [1] . Segmentation masks from m like Mask2Former [2] can indicate regions, but ofte stability for consistent video modifications [3] . Altern manual user inputs for tracking provide accuracy ye scalability and e ciency. A P E G L F n P e 1 G o A 0 A z s h E U o = " > A A A C H n i c b V D L S g M x F M 3 U V 6 2 v q k s 3 w S K 4 K j M + q s u i m 7 q r Y h / Q q S W T p m 1 o J j M k d 4 Q y z J e 4 8 V f c u F B E c K V / Y 6 a t o K 0 X Q g 7 n 3 M u 9 5 3 i h 4 B p s + 8 v K L C w u L a 9 k V 3 N r 6 x u b W / n t n b o O I k V Z j Q Y i U E 2 P a C a 4 Z D X g I F g z V I z 4 n m A N b 3 i Z 6 o 1 7 p j Q P 5 C 2 M Q t b 2 S V / y H q c E D N X J n 7 p e I L p 6 5 J s v v k o 6 g F 0 u s e s T G H h e f J P c x R X s A v e Z x o 0 f c J x 0 8 g W 7 a I 8 L z w N n C g p o W t V O / s P t B j T y m Q Q q i N Y t x w 6 h H R M F n A q W 5 N x Ip Z F E v E h g C n G a F u 1 w x C m J k A K G K m 1 s x H R B F K J h E c y Y E Z 9 b y P K g f F Z 1 S s X R 9 U i h f T O P I o j 2 0 j w 6 R g 8 5 Q G V V Q F d U Q R Q / o C b 2 g V + vR e r b e r P d J a 8 a a z u y i P 2 V 9 f g N a 0 a K a < / l a t e x i t > I t 2 R H⇥W ⇥3 < l a t e x i t s h a 1 _ b a s e 6 4 = " 2 L m N V O 3 D u l c O S U j 4 S o R h x X E i c 0 I = " > A A A C J n i c b V D L S g M x F M 3 4 r P V V d e k m W A R X Z U a l u h G K b u q u i n 1 A p 5 Z M J m 1 D M 5 k h u S O U Y b 7 G j b / i x k V F x J 2 f Y q a t o N U L I Y d z 7 u X e c 7 x I c A 2 2 / W E t L C 4 t r 6 z m 1 v L r G 5 t b 2 4 W d 3 Y Y O Y 0 V Z n Y Y i V C 2 P a C a 4 Z H X g I F g r U o w E n m B N b 3 i V 6 c 0 H p j Q P 5 R 2 M I t Y J S F / y H q c E D N U t X L j A h c 8 S 1 w u F r 0 e B + Z L r N O 0 C d r n E b k B g 4 H n J b X q f V L F p D Z j G z W 9 w k n Y L R b t k T w r / B c 4 M F N G s a t 3 C 2 P V D G g d M A h V E 6 7 Z j R 9 B J i A J O B U v z b q x Z R O i Q 9 F n b Q E n M n k 4 y s Z n i Q 8 P 4 u B c q 8 y T g C f t z I i G B z i y Y z u x u P a 9 l 5 H 9 a O 4 b e e S f h M o q B S T p d 1 I s F h h B n m W G f K 0 Z B j A w g V H F z K 6 Y D o g g F k 2 z e h O D M W / 4 L G s c l p 1 w q 3 5 w W K 5 e z O H J o H x 2 g I + S g M 1 R B V V R D d U T R I 3 p G Y / R q P V k v 1 p v 1 P m 1 d s G Y z e + h X W Z 9 f a + i m U A = = < / l a t e x i t > Ĩt 2 R H⇥W ⇥3 < l a t e x i t s h a 1 _ b a s e 6 4 = " l l Z A e R f / T e U N N S u 0 f 5 2 M u i V J 1 S 8 = " > A A A C H n i c b V D L S g M x F M 3 4 r P U 1 6 t J N s A i u y o x o d V l 0 0 4 1 Q x T 6 g U 0 s m T d v Q T G Z I 7 g h l m C 9 x 4 6 + 4 c a G I 4 E r / x k x b Q V s v h B z O u Z d 7 z / E j w T U 4 z p e 1 s L i 0 v L K a W 8 u v b 2 x u b d s 7 u 3 U d x o q y G g 1 F q J o + 0 U x w y W r A Q b B m p B g J f M E a / v A y 0 x v 3 T G k e y l s Y R a w d k L 7 k P U 4 J G K p j n 3 p + K L p 6 F J g v u U o 7 g D 0 u s R c Q G P h + c p P e J R X s A Q + Y x o 0 f 4 K Y d u + A U n X H h e e B O Q Q F N q 9 q x P 7 x u S O O A S a C C a N 1 y n Q j a C V H A q W B p 3 o s 1 i w g d k j 5 r G S i J 2 d N O x v Z S f G i Y L u 6 F y j w J e M z + n k h I o D M H p j O 7 W 8 9 q G f m f 1 o q h d 9 5 O u I x i Y J J O F v V i g S H E W V a 4 y x W j I E Y G E K q 4 u R X T A V G E g k k 0 b 0 J w Z y 3 P g / p x 0 S 0 V S 9 c n h f L F N I 4 c 2 k c H 6 A i 5 6 A y V U Q V V U Q 1 R 9 I C e 0 A+ i 8 s b E A f t s S R + s = " > A A A C H n i c b V D L S g M x F M 3 4 r P U 1 6 t J N s A i u y o x o 6 7 L o p s s q 9 g G d W j J p 2 o Z m M k N y R y j D f I k b f 8 W N C 0 U E V / o 3 Z t o K 2 n o h 5 H D O v d x 7 j h 8 J r s F x v q y l 5 Z X V t f X c R n 5 z a 3 t n 1 9 7 b b + g w V p T V a S h C 1 f K J Z o J L V g c O g r U i x U j g C 9 b 0 R 1 e Z 3 r x n S v N Q 3 s I 4 Y p 2 A D C T v c 0 r A U F 3 7 3 P N D 0 d P j w H x J K + 0 C 9 r j E X k B g 6 P v J T X q X V L E H P G A a N 3 9 A O e 3 a B a f o T A o v A n c G C m h W t a 7 9 4 f V C G g d M A h V E 6 7 b r R N B J i A J O B U v z X q x Z R O i I D F j b Q E n M n k 4 y s Z f i Y 8 P 0 c D 9 U 5 k n A E / b 3 R E I C n T k w n d n d e l 7 L y P + 0 d g z 9 i 0 7 C Z R Q D k 3 S 6 q B 8 L D C H O s s I 9 r h g F M T a A U M X N r Z g O i S I U T K J 5 E 4 I 7 b 3 k R N E 6 L b q l Y u j 4 r V C 5 n c e T Q I T p C J 8 h F Z V R B V V R D d U T R A 3 p C L + j V e r S e r T f r f d q 6 Z M 1 m D t C f s j 6 / A X n 5 o q 0 = < / l a t e x i t > X t 2 R H⇥W ⇥7 < l a t e x i t s h a 1 _ b a s e 6 4 = " T k Z d I w C I 9 D T c 9 1 P u e Y I Z a O a 1 E m w = " > A A A C J H i c b V D L S g M x F M 3 U V 6 2 v q k s 3 w S K 4 K j M q V X B T d F N 3 V e w D O r V k 0 r Q N z W S G 5 I 5 Q h v k Y N / 6 K G x c + c O H G b z H T V t D W C y G H c + 7 l 3 n O 8 U H A N t v 1 p Z R Y W l 5 Z X s q u 5 t f W N z a 3 8 9 k 5 d B 5 G i r E Y D E a i m R z Q T X L I a c B C s G S p G f E + w h j e 8 T P X G P V O a B / I W R i F r + 6 Q v e Y 9 T A o b q 5 M / d A Y H Y 9 Q L R 1 S P f f P F V k n Q A u 1 x i 1 y c w 8 L z 4 J r m L K 9 g F 7 j O N G z / g O O n k C 3 b R H h e e B 8 4 U F N C 0 q p 3 8 m 9 s N a O Q z C V Q Q r V u O H U I 7 J g o 4 F S z J u Z F m I a F D 0 m c t A y U x e 9 r x 2 G S C D w z T x b 1 A m S c B j 9 n f E z H x d W r B d K Z 3 6 1 k t J f / T W h H 0 z t o x l 2 E E T N L J o l 4 k M A Q 4 T Q x 3 u W I U x M g A Q h U 3 t 2 I 6 I I p Q M L n m T A j O r O V 5 U D 8 q O q V i 6 f q k U L 6 Y x p F F e 2 g f H S I H n a I y q q A q q i G K H t A T e k G v 1 q P 1 b L 1 b H 5 P W j D W d 2 U V / y v r 6 B q x 1 p W c = < / l a t e x i t > Ît 2 R H⇥W ⇥3 < l a t e x i t s h a 1 _ b a s e 6 4 = " 2 L m N V O 3 D u l c O S U j 4 S o R h x X E i c 0 I = " > A A A C J n i c b V D L S g M x F M 3 4 r P V V d e k m W A R X Z U a l u h G K b u q u i n 1 A p 5 Z M J m 1 D M 5 k h u S O U Y b 7 G j b / i x k V F x J 2 f Y q a t o N U L I Y d z 7 u X e c 7 x I c A 2 2 / W E t L C 4 t r 6 z m 1 v L r G 5 t b 2 4 W d 3 Y Y O Y 0 V Z n Y Y i V C 2 P a C a 4 Z H X g I F g r U o w E n m B N b 3 i V 6 c 0 H p j Q P 5 R 2 M I t Y J S F / y H q c E D N U t X L j A h c 8 S 1 w u F r 0 e B + Z L r N O 0 C d r n E b k B g 4 H n J b X q f V L F p D Z j G z W 9 w k n Y L R b t k T w r / B c 4 M F N G s a t 3 C 2 P V D G g d M A h V E 6 7 Z j R 9 B J i A J O B U v z b q x Z R O i Q 9 F n b Q E n Mu i V J 1 S 8 = " > A A A C H n i c b V D L S g M x F M 3 4 r P U 1 6 t J N s A i u y o x o d V l 0 0 4 1 Q x T 6 g U 0 s m T d v Q T G Z I 7 g h l m C 9 x 4 6 + 4 c a G I 4 E r / x k x b Q V s v h B z O u Z d 7 z / E j w T U 4 z p e 1 s L i 0 v L K a W 8 u v b 2 x u b d s 7 u 3 U d x o q y G g 1 F q J o + 0 U x w y W r A Q b B m p B g J f M E a / v A y 0 x v 3 T G k e y l s Y R a w d k L 7 k P U 4 J G K p j n 3 p + K L p 6 F J g v u U o 7 g D 0 u s R c Q G P h + c p P e J R X s A Q + Y x o 0 f 4 K Y d u + A U n X H h e e B O Q Q F N q 9 q x P 7 x u S O O A S a C C a N 1 y n Q j a C V H A q W B p 3 o s 1 i w g d k j 5 r G S i J 2 d N O x v Z S f G i Y L u 6 F y j w J e M z + n k h I o D M H p j O 7 W 8 9 q G f m f 1 o q h d 9 5 O u I x i Y J J O F v V i g S H E W V a 4 y x W j I E Y G E K q 4 u R X T A V G E g k k 0 b 0 J w Z y 3 P g / p x 0 S 0 V S 9 c n h f L F N I 4 c 2 k c H 6 A i 5 6 A y V U Q V V U Q 1 R 9 I C e 0 A t= " > A A A C G n i c b V D L S s N A F J 3 U V 6 2 v q k s 3 g 0 V w V R K R 6 r L o R h d C B f u A J p T J Z N o O n W T C z I 1 Q Q r 7 D j b / i x o U i 7 s S N f + O 0 z a K 2 H h j m c M 6 9 3 H u P H w u u w b Z / r M L K 6 t r 6 R n G z t L W 9 s 7 t X 3 j 9 o a Z k o y p p U C q k 6 P t F M 8 I g 1 g Y N g n V g x E v q C t f 3 R 9 c R v P z K l u Y w e Y B w z L y S D i P c 5 J W C k X t l x g Y u A p a 4 v R a D H o f n S 2 y z r A X Y l 8 J B p P O / c G a N X r t h V e w q 8 T J y c V F C O R q / 8 5 Q a S J i G L g A q i d d e x Y / B S o o B T w b K S m 2 g W E z o i A 9 Y 1 N C J m q p d O T 8 v w i V E C 3 J f K v A j w V J 3 v S E m o J 8 u Z y p D A U C 9 6 E / E / r 5 t A / 9 J L e R Q n w C I 6 G 9 R P B A a J J z n h g C t G Q Y w N I V R x s y u m Q 6 I I B Z N m y Y T g L J 6 8 T F p n V a d W r d 2 f V + p X e R x F d I S O 0 S l y 0 A W q o x v U Q E 1 E 0 R N 6 Q W / o 3 X q 2 X q 0 P 6 3 N W WP T x q G 5 V r y g K q h N L d i B g m u G Q B c h S s m 2 l G 0 k i w T j S + n f u d R 6 Y N V / I B J x k L U z K U P O G U o J W C v o o V D q o 1r + 4 t 4 P 4 l f k F q U K A 1 q H 7 2 Y 0 X z l E m k g h j T 8 7 0 M w y n R y K l g s 0 o / N y w j d E y G r G e p J C k z 4 X R x 7 M w 9 s 0 r s J k r b k u g u 1 J 8 T U 5 I a M 0 k j 2 5 k S H J l V b y 7 + 5 / V y T K 7 D K Z d Z j k z S 5 a I k F y 4 q d / 6 5 G 3 P N K I q J J Y R q b m 9 1 6 Y h o Q t H m U 7 E h + K s v / y X t i 7 r f q D f u L 2 v N m y K O M p z A K Z y D D 1 f Q h D t o Q Q A U O D z B C= " > A A A C H X i c b V D L S g M x F M 3 U V 6 2 v q k s 3 w S L U h W V G S n V Z d K M L o Y J 9 Q K e U T C Z t Q z O T I b k j l K E / 4 s Z f c e N C E R d u x L 8 x 0 3 Z R W w + E H M 6 5 l 3 v v 8 S L B N d j 2 j 5 V Z W Vl + w J 8 D J x Z q S A Z q h 1 8 1 + u L 2 k c s B C o I F q 3 H T u C T k I U c C r Y O O f G m k W E D k m f t Q 0 N S c B 0 J 5 l c N 8 Y n R v F x T y r z Q s A T d b 4 j I Y F O 1 z O V A Y G B X v R S 8 T + v H U P v s p P w M I q B h X Q 6 q B c L D B K n U W G f K 0 Z B j A w h V H G z K 6 Y D o g g F E 2 g a g r N 4 8 j J p n J e c S q l y X y 5 U r 2 Z x Z N E R O k Z F 5 K A L V E U 3 q I b q i K I n 9 I L e 0 L v 1 b L 1 a H 9 b n t D R j z X o O 0 R 9 Y 3 7 8 V Y a K B < / l a t e x i t > Ĩt (1 M t )Video color enhancement aims to improve the visual qua of videos through targeted color adjustments. This capabi enables more engaging and vibrant video content for v ous applications such as post-production, social media, a entertainment. While photo enhancement techniques h advanced rapidly, video color enhancement remains an op challenge, requiring both spatial and temporal constraint A key di culty lies in performing selective, region-spec color modifications while maintaining coherence across fram Prior arts have explored global color transforms, which fai enable localized edits [1] . Segmentation masks from mod like Mask2Former [2] can indicate regions, but often l stability for consistent video modifications [3] . Alternativ manual user inputs for tracking provide accuracy yet li scalability and e ciency. 2024-04-13 19:53. Page 8 of 1-10. 
related works 2.1 fine-grained video color enhancement: 
 Video color enhancement aims to improve the aesthetic appeal of videos by adjusting colors. Techniques such as Harmonizer [14] and PSENet [33] , initially developed for image enhancement, are adapted for video by ensuring color consistency within shots. Similarly, methods using global 3D-LUTs, like 3DLUT [39] and AdaInt [38] , also maintain consistent LUT settings across clips. However, they lack the ability to make detailed, region-specific color adjustments.Traditional tools such as Adobe After Effects and DaVinci Resolve, which allow detailed regional color modifications, inspire spatial mask-based techniques like DeepLPF [24] , DCCF [36] , and LED-Net [42] . These methods employ localized filters for precise color grading but struggle with maintaining consistency across video frames. RSFNet [25] attempts to address this by using segmentationbased masks (e.g., Mask2Former [5] , SegFormer [1] , SAM [17] ) for color enhancement. Nonetheless, these masks often fail to ensure smooth color transitions, particularly during substantial edits, leading to performance issues.Our approach, MovingColor, targets these challenges by improving both spatial and temporal color consistency, especially focusing on issues related to the instability of segmentation masks.Spatial inconsistency in video color editing involves refining masks for precise segmentation and employing post-processing techniques like harmonization and color matching to preserve original edits and ensure uniformity.Mask refinement methods like HQ-SAM [13] and Seg-Refiner [32] improve the accuracy of masks for finely detailed objects but often struggle with creating natural transitions at complex edges. In image harmonization, approaches such as Harmonizer [14] , S2CRNet [21] , and DCCF [37] effectively merge foreground and background but may alter intended color edits and are less effective in video due to mask inconsistencies.Style transfer and inpainting techniques, although they adjust an image's style or fill in missing areas, frequently alter colors and textures inappropriately for video tasks. Techniques like ReCoRo [34] focus on localized lightness adjustments but are limited by their exclusive focus on this aspect and reliance on specific datasets.Our proposed solution effectively manages these challenges, even with the coarse and unstable masks from other tracking techniques, ensuring smooth color transitions and consistency.Temporal inconsistencies arise when image processing algorithms are directly applied to videos frame-by-frame. Existing methods like Blind Video Temporal Consistency [2] and Learning Blind Video Temporal Consistency [18] aim to reduce these discrepancies. While they enhance temporal stability, they often degrade the video's quality and diminish the efficacy of the original processing methods.Video style transfer methods attempt to maintain visual consistency over time by aligning frames with a reference style, as seen in Stylizing Video [12] and Interactive Video Stylization [30] . However, they generally fall short in maintaining user edits and have limited applicability to real-world photography.MovingColor, our proposed method, effectively minimizes temporal inconsistencies while preserving the desired color qualities of the edited areas, suitable for various contexts including artistic and photographic enhancement. Video color editing aims to enhance the visual quality of videos through targeted regional color adjustments. However, applying edits dynamically across frames often causes flickering artifacts due to mask instability. Furthermore, coarse masks struggle to blend colors naturally across object boundaries. This paper presents MovingColor to address these challenges and enable seamless finegrained video color editing. We propose a self-supervised training strategy that synthesizes color transformations and object masks on videos to teach a network to fuse edits. A key innovation is formulating color blending as a fills generation task, rather than binary completion, better retaining details. Explicit global and local processing branches allow multi-scale representation. Further, a temporal transformer propagates relevant features across frames for coherence. Quantitative and subjective evaluations demonstrate MovingColor's capabilities in versatile video color manipulations with state-of-the-art spatial and temporal consistency. The system significantly advances controllable, professional-grade video color 
gaps in video matting and edge refinement: 
 For a given input frame 𝑰 𝑡 , color transformation 𝑇 , and binary mask 𝑨 𝑡 , our objective is to achieve realistic color fusion. Typically, existing methods compute a blending matte 𝑩 𝑡 as follows:EQUATIONHowever, obtaining accurate 𝑩 𝑡 for video sequences is challenging, as video matting datasets primarily include only foreground elements without the corresponding (𝑰 𝑡 , 𝑩 𝑡 ) pairs. Consequently, methods are often trained on synthetic datasets, which may not perform well in practical applications. Current video matting solutions are typically specialized, such as portrait video matting [15] , and do not provide generalizable solutions. Consequently, colorists frequently resort to semi-automatic tools like RotoBrush V3 in Adobe After Effects, which require extensive manual adjustments and are both labor-intensive and time-consuming. v + + v 0 x r 7 4 = " > A A A C G n i c b V D L S g M x F M 3 4 t r 6 q L t 0 E i + B C y o z 4 2 g i i G 5 c V r A q d Y c h k M m 0 w k 4 T k j l C G f o c b f 8 W N C 0 X c i R v / x k z b h a 8 D I S f n 3 E v u P Y k W 3 I L v f 3 o T k 1 P T M 7 N z 8 7 W F x a X l l f r q 2 p V V h a G s T Z V Q 5 i Y h l g k u W R s 4 C H a j D S N 5 I t h 1 c n t W + d d 3 z F i u 5 C X 0 N Y t y 0 p U 8 4 5 S A k + J 6 E C Z K p L a f u 6 v U g + N Q s A w 6 O g 5 2 s I 5 3 d 3 A o U g W 2 e s j Q 8 G 4 P o r j e 8 J v + E P g v C c a k g c Z o x f X 3 M F W 0 y J k E K o i 1 n c D X E J X E A K e C D W p h Y Z k m 9 J Z 0 W c d R S X J m o 3 K 4 2 g B v O S X F m T L u S M B D 9 X t H S X J b T e 8 q c w I 9 + 9 u r x P + 8 T g H Z U V R y q Q t g k o 4 + y g q B Q e E q J 5 x y w y i I v i O E G u 5 m x b R H D K H g 0 q y 5 E I L f K / 8 l V 7 v N 4 K C 5 f 7 H X O D k d x z G H N t A m 2 k Y B O k Q n 6 B y 1 U B tK o A U r E v A q / 1 M l 8 O T e j N B p A l X E = " > A A A C A 3 i c b V D L S s N A F J 3 4 r P U V d a e b Y B F c l U R 8 L Y t u X L i o 0 B c 0 o U y m k 3 b o J B N m b s Q S A m 7 8 F T c u F H H r T 7 j z b 5 y 0 W W j r g Q u H c + 7 l 3 n v 8 m D M F t v 1 t L C w u L a + s l t b K 6 x u b W 9 v m z m 5 L i U Q S 2 i S C C 9 n x s a K c R b Q J D D j t x J L i 0 O e 0 7 Y + u c 7 9 9 T 6 V i I m r A O K Z e i A c R C x j B o K W e u e + G G I Y E 8 / Q 2 6 6 U u 0 A d I G w I w z 7 K e W b G r 9 g T W P H E K U k E F 6 j 3 z y + 0 L k o Q 0 A s K x U l 3 H j s F L s Q R G O M 3 K b q J o j M k I D 2 h X 0 w i H V H n p 5 I f M O t J K 3 w q E 1 B W B N V F / T 6 Q 4 V G o c + r o z v 1 j N e r n 4 n 9 d N I L j 0 U h b F C d C I T B c F C b d A W H k g V p 9 J S o C P N c F E M n 2 r R Y Z Y Y g I 6 t r I O w Z l 9 e Z 6 0 T q r O e f X s 7 r R S u y r i K K E D d I i O k Y M u U A 3 d o D p q I o I e 0 T N 6 R W / G k / F i v B s f 0 9 Y F o 5 j Z Q 3 9 g f P 4 A Y L G Y p w = = < / l a t e x i t > L TotalInput Frame Regional Color Edit (Boundary masked out)Natural Fusion of Regional Color Edits Video color editing aims to enhance the visual quality of videos through targeted regional color adjustments. However, applying edits dynamically across frames often causes flickering artifacts due to mask instability. Furthermore, coarse masks struggle to blend colors naturally across object boundaries. This paper presents MovingColor to address these challenges and enable seamless finegrained video color editing. We propose a self-supervised training strategy that synthesizes color transformations and object masks on videos to teach a network to fuse edits. A key innovation is formulating color blending as a fills generation task, rather than binary completion, better retaining details. Explicit global and local processing branches allow multi-scale representation. Further, a temporal transformer propagates relevant features across frames for coherence. Quantitative and subjective evaluations demonstrate MovingColor's capabilities in versatile video color manipulations with state-of-the-art spatial and temporal consistency. The system significantly advances controllable, professional-grade video color Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. © 2024 Copyright held by the owner/author(s). Video color editing aims to enhance the visual quality of videos through targeted regional color adjustments. However, applying edits dynamically across frames often causes flickering artifacts due to mask instability. Furthermore, coarse masks struggle to blend colors naturally across object boundaries. This paper presents MovingColor to address these challenges and enable seamless finegrained video color editing. We propose a self-supervised training strategy that synthesizes color transformations and object masks on videos to teach a network to fuse edits. A key innovation is formulating color blending as a fills generation task, rather than binary completion, better retaining details. Explicit global and local processing branches allow multi-scale representation. Further, a temporal transformer propagates relevant features across frames for coherence. Quantitative and subjective evaluations demonstrate MovingColor's capabilities in versatile video color manipulations with state-of-the-art spatial and temporal consistency. The system significantly advances controllable, professional-grade video colorTo enhance the robustness of the pretext task learning, we introduce randomness in color adjustment and mask generation, leveraging the diverse content and color distributions in the large-scale dataset. A random parametric Look-Up Table (LUT) generator, parameterized by 𝒑, modifies video frame batches to create color-enhanced versions Ĩ 𝑡 . Randomly generated masks 𝑴 𝑡 , incorporating both static and dynamic elements, are obtained using approaches similar to [16] . The training input 𝑿 𝑡 = Concate[𝑰 𝑡 , Ĩ 𝑡 ⊙ 𝑴 𝑡 , 𝑴 𝑡 ] is formed in R 𝐻 ×𝑊 ×7 , with Ĩ 𝑡 serving as the ground truth.During the inference stage for video color enhancement, the system processes input frames, masks, and coloradjusted frames, addressing spatio-temporal inconsistency at mask edges, such as where the sky meets the field, as shown in the figure 3 . Inference is performed in a sliding window fashion over G L A I q i F J 9 2 0 r A z Y g E T g X L q 0 6 q W E L o m A x Z X 9 O I h E y 5 2 T R A j k + 0 M s B B L P W L A E / V 3 x s Z C Z W a h L 6 e L M 5 V 8 1 4 h / u f 1 U w g u 3 Y x H S Q o s o r O P g l R g i H H R B h 5 w H R b E R B N C J d e 3 Y j o i k l D Q n V V 1 C f Z 8 5 E X S O 
method 4.1 architecture overview: 
 As shown in Figure 4 , the proposed MovingColor employs a spectralspatial hybrid encoder with convolutional and Fast Fourier Convolutional networks to capture multi-scale spatial features and frequency information, followed by a global-local feature propagation module with Transformer blocks to aggregate spatio-temporal contexts across frames for spatial and temporal consistency. Due to space constraints, we briefly introduce the key components here, with more comprehensive details available in the supplementary materials. Additionally, we will make our code available for reproduction and application purposes.Seeking to combine the benefits of both convolutional and Fourier techniques [6] , we propose chaining a convolutional encoder with a Fast Fourier Convolutional encoder into one hybrid architecture. This spectral-spatial design allows efficient learning of multi-scale spatial features while also capturing frequency information, providing a rich yet compact input representation for color fusion.Encoder. We use a lightweight encoder that is the same as that in FuseFormer [23] , which is adapted from STTN [40] with deeper layers, more channel growth in the same time, using the group convolutions in the later layers to limit model complexity. Features from earlier layers are concatenated to later ones using grouped convolutions to aggregate multi-scale representations. The encoder outputs a compact yet rich representation of the input for further processing. The module for global-local frame feature propagation is designed to improve temporal consistency. It is architecturally composed of 𝑁 temporal Transformer blocks.Each temporal transformer block undergoes a series of transformations, beginning with layer normalization applied to input token features. This step stabilizes the learning process and normalizes feature distributions. The normalized features are then processed through the Self-Attention module for aggregating spatio-temporal contextual information. Following the attention mechanism, we integrate a residual connection, allowing the addition of the attention outputs to the original input token features. This approach ensures the retention of critical information through the network layers. Another layer normalization is then applied to these combined features, setting the stage for the subsequent Fusion Feed-Forward Network [23] . This network, with its series of linear transformations and GELU activations, further refines the representations, endowing the model with the capacity to capture complex, high-level abstractions from the data.Frame Feature Propagation. Reflecting insights from prior studies [8, 20] , the exclusive reliance on local temporal neighbors is acknowledged to be informationally constraining. Incorporating global-level information can enhance the synthesis of naturalistic color and textures. To this end, non-local frames are integrated to introduce a broader context. The process involves a soft split operation for embedding overlapping patches from both local and non-local temporal features. At the end of the feature propagation module, we use a soft composite operator to composite the embedded tokens to features. This architecture, encompassing temporal Transformer blocks along with global-local frame features, is adept at forging a detailed and layered representation of spatio-temporal dynamics. It aggregates features across spatial and temporal scales effectively. The resulting token features offer a holistic view of the input video data, capturing the nuanced interplay of spatial and temporal elements.In each iteration, the model processes input frames, masked out frames, and masks. The generator, which encompasses feature encoding, propagation and decoding is responsible for predicting the output frames. The model utilizes a combination of loss functions to optimize video frame generation. Reconstruction Loss: To measure the discrepancy between the output video sequence Ŷ and the ground-truth sequence 𝑌 , an L1 loss is employed, defined asL rec = Î 𝑡 -Ĩ 𝑡 1 .Adversarial Loss: Furthermore, we introduce an adversarial training procedure with a T-PatchGAN based discriminator 𝐷 [3] to enhance the quality and coherence of generated videos.The discriminator's loss function, L 𝐷 , optimizes it to differentiate between real and generated frames and is defined asL 𝐷 = E Ĩ 𝑡 [log 𝐷 ( Ĩ 𝑡 )] + E Î 𝑡 [1 -log 𝐷 ( Î 𝑡 )].For the generator, the adversarial loss is calcu-lated as L 𝐺 = -E Î 𝑡 [log 𝐷 ( Î 𝑡 )].Perceptual Loss:The perceptual loss L perc , calculated between the VGG-19 [27] feature maps of the output and ground truth, facilitates similarity assessments within the semantic feature space. Total Loss: The total loss is a weighted sum of these components: L total = L rec +𝜆 1 L 𝐺 +𝜆 2 L perc , with the weight 𝜆 1 = 0.01, 𝜆 2 = 0.5. 
experiments 5.1 datasets: 
 5.1.1 DAVIS and YouTube-VOS. DAVIS [26] and YouTube-VOS [35] are two widely used video object segmentation datasets. DAVIS contains 50 video sequences with pixel-level annotations. YouTube-VOS is a large-scale dataset with 3,471 video sequences. We use the training set of YouTube-VOS to train MovingColor. The referencebased similarity metrics for the DAVIS and YouTube-VOS datasets are calculated exclusively in non-edge regions due to the absence of ground truth for evaluating color fusion on edge areas, as these datasets provide only coarse, unrefined masks.We propose the D5 dataset to address the limitations of existing video object segmentation datasets, which lack precise masks for evaluation, and video matting datasets that only contain foreground object mattings. Synthesizing results by compositing foreground mattings with background videos often yields unnatural outputs that do not accurately reflect lighting and color interactions. Using D5 Render, a 3D software, we generate 121 diverse 4K video clips from 12 scenes, featuring various camera movements and moving subjects such as people, animals, and architecture. Each video includes material ID maps serving as ground truth masks for all objects. This dataset is suitable for evaluating color fusion and related tasks, e.g. matting and composition.5.2.1 Implementation Details. We use the training set of YouTube-VOS [35] with 3471 video sequences for training and did not finetuned on other datasets. We use the Adam optimizer with a learning rate of 0.0001. The batch size is set to 4. The training process takes about 2 days on 4 NVIDIA V100 GPUs. For more implementation settings, please refer to the supplementary materials.To the best of our knowledge, we are the first to introduce a color fusion method, and thus lack direct comparative methods in this area. Instead, we compare our approach with related color manipulation techniques: the color matching method Color Matcher [11] , image harmonization methods (Harmonizer [14] , S2CRNet [22] , and PCTNet [9] ), and the style transfer method StyA2K [43] and FSPBT [30] . We also compare with the video consistency method All-In-One-Deflicker [19] .We employ quantitative metrics to evaluate video color fusion performance, measuring reference-based similarity, texture preservation, temporal consistency, and efficiency. Reference-based Difference: We use PSNR, SSIM, ΔE, and the perceptually-aligned DreamSim [7] between result and ground truth frames. Higher PSNR, SSIM, and DreamSim values and lower ΔE indicate better similarity and color accuracy. For the D5 dataset, we apply these metrics to full frames, while for DAVIS and YouTube-VOS, we compare only non-edge areas due to the lack of ground truth for edge regions. Texture Preservation: We introduce the texture difference (TD) metric to assess the preservation of texture details, inspired by digital art practices [4] and image processing workflows [16] . TD measures the difference in high-frequency texture components between result and input frames, with lower values indicating better preservation without unwanted textures. Temporal Consistency: We employ flow warping error (𝐸 Warp ) [18] , Patch Consistency (PC) [10] , and Perceptual Video Clip Similarity (PVCS) [29] to assess visual steadiness across frames. Lower PVCS and 𝐸 Warp values and higher PC scores indicate better temporal coherence. For brevity, we report 𝐸 Warp × 10 3 , denoted as 𝐸 * Warp . Efficiency: We measure FLOPs and inference time per frame.As shown in Table 1 , MovingColor achieves the best performance on the D5-Material dataset in terms of reference-based similarity, with the highest PSNR (26.88), SSIM (0.88), and DreamSim (0.10) scores, and the lowest ΔE (4.35). For texture preservation, Moving-Color and Color Matcher both achieve the lowest TD (1.03), demonstrating their effectiveness in maintaining texture details without introducing artifacts. In terms of temporal consistency, Moving-Color performs competitively, with the highest PC SSIM (0.97) and the second-lowest PVCS (0.54) and 𝐸 * Warp (0.29) scores. MovingColor maintains a good balance between performance and efficiency, with a runtime of 0.08s per frame, which is comparable to the fastest methods (Harmonizer and S2CRNet) while achieving significantly better results. Although MovingColor has higher FLOPs (26.51G) compared to some baselines, it is still more efficient than FSPBT and the combination of StyA2K and Deflicker. These results demonstrate that MovingColor achieves state-of-the-art performance in video color fusion, successfully balancing referencebased similarity, texture preservation, temporal consistency, and efficiency across multiple datasets and evaluation metrics. Additional results on DAVIS and YouTube datasets are provided in the supplementary materials.Comparison with Adapted Inpainting Methods. Figure 5 shows that simply adapting existing inpainting methods to the color fusion setting yields unsatisfactory results. We compare MovingColor to LAMA [28] and Propainter [41] , both adapted to take the same input as our method. Since LAMA is an image inpainting method that performs poorly in terms of temporal consistency, we add the same temporal module as MovingColor for a fair comparison. Despite this modification, MovingColor significantly outperforms both LAMA and Propainter on all datasets. Due to space constraints, we present results on the D5 dataset; however, similar trends are observed on other datasets. These comparisons, along with the ablation studies, further validate the effectiveness of MovingColor's design choices for color fusion tasks.We conduct ablation studies to investigate the effectiveness of key components in MovingColor, including the Fast Fourier Convolution (FFC) encoder and the global-local feature propagation. As shown in Table 2 , the most substantial improvement is observed when integrating the FFC encoder along with both local and global features, achieving the lowest ΔE, TD, and 𝐸 * Warp scores across all datasets (D5, DAVIS, and YouTube-VOS). Comparing the variants with and without the FFC encoder, we observe a significant reduction in ΔE when FFC is included, highlighting the importance of capturing spectral information for accurate color fusion. Similarly, the inclusion of global features consistently improves performance across all metrics and datasets, underscoring the benefits of incorporating non-local temporal information. These ablation results conclusively demonstrate the efficacy of the FFC encoder and the synergistic effect of global-local feature propagation in enhancing MovingColor's performance. In addition, the effectiveness of the loss functions has been validated, with detailed loss ablation studies provided in the supplementary material.To evaluate MovingColor's adaptability and consistency, we perform an extensive robustness test, considering various edge ratios, input resolutions, and color adjustments. Figure 6 showcases the results, demonstrating the model's capability to deliver high-quality output across diverse and demanding conditions. 5.5.1 Robustness to Varying Edge Ratios. Figure 6a evaluates Mov-ingColor's robustness to edge ratios ranging from 5% to 25% on the D5, DAVIS, and YouTube datasets, covering typical real-world scenarios. Across all datasets, ΔE increases with larger edge ratios due to less available color information for fusion, while texture difference (TD) improves as larger fusion areas allow for smoother transitions and more natural textures.On D5 and YouTube, MovingColor maintains stable performance, with slight increases in mean ΔE (3.09 to 3.55) and consistent mean TD (around 1.1). DAVIS, containing fast-moving objects and more occlusions, presents a more challenging scenario. Still, MovingColor outperforms all baselines and maintains stable performance, with mean ΔE rising from 2.8 to 3.48 and mean TD improving from 2.06 to 1.81 as edge ratio increases, demonstrating its effectiveness in preserving original color adjustments and textures for better spatial consistency and harmony. Evaluation metrics based solely on reference-based similarities have limitations, as there can be multiple reasonable results for color fusion tasks. Therefore, we conducted a user study with 68 participants. The supplementary material contains comprehensive information on the user study settings and an interview with a professional colorist. The baselines we selected cover key capabilities needed for video color fusion: handling spatial inconsistencies (Color Matcher, StyA2K), temporal inconsistencies (Deflicker), and combinations thereof (Deflicker+StyA2K). For the study, we cropped areas with noticeable artifacts from 3 videos to better showcase details. As shown in Figure 7 , the user study results demonstrate Mov-ingColor's superior performance over the baselines across three aspects: spatial consistency, temporal consistency, and color accuracy. Specifically, MovingColor substantially outperforms the baselines in enhancing spatial consistency, with Color Matcher being a reasonably strong baseline by maintaining color harmony. Although StyA2K can perform good style transfers, it still suffers from unrealistic color transitions and inconsistencies. Deflicker alone does little to improve spatial consistency. The Deflicker+StyA2K combination, while combining the strengths of both methods, still falls short of MovingColor. Regarding temporal consistency, MovingColor performs comparably to the strong ColorMatcher baseline and better than StyA2K and Deflicker. This shows MovingColor's ability to maintain consistent colors over time. Finally, MovingColor has the lowest mean ΔE score, indicating it most accurately preserves target colors with minimal deviation. Together, these user study findings demonstrate MovingColor's state-of-the-art performance in video color fusion across key criteria.  
limitations and future work: 
 MovingColor struggles to accurately process long, thin objects, such as sticks, and transparent objects. This limitation stems from the difficulty in segmenting long, thin objects and transparent objects, which remains a challenging problem in computer vision. Addressing this issue and improving the method's ability to handle these complex objects will be a focus of our future work. Despite their limitations in generating high-resolution, long videos, Diffusionbased models show promise for enhanced color fusion, which we aim to explore in the future. Video color editing aims to enhance the visual quality of videos through targeted regional color adjustments. However, applying edits dynamically across frames often causes flickering artifacts due to mask instability. Furthermore, coarse masks struggle to blend colors naturally across object boundaries. This paper presents MovingColor to address these challenges and enable seamless finegrained video color editing. We propose a self-supervised training strategy that synthesizes color transformations and object masks on videos to teach a network to fuse edits. A key innovation is formulating color blending as a fills generation task, rather than binary completion, better retaining details. Explicit global and local processing branches allow multi-scale representation. Further, a temporal transformer propagates relevant features across frames for coherence. Quantitative and subjective evaluations demonstrate MovingColor's capabilities in versatile video color manipulations with state-of-the-art spatial and temporal consistency. The system significantly advances controllable, professional-grade video color 
: 
 Video color editing aims to enhance the visual quality of videos through targeted regional color adjustments. However, applying edits dynamically across frames often causes flickering artifacts due to mask instability. Furthermore, coarse masks struggle to blend colors naturally across object boundaries. This paper presents MovingColor to address these challenges and enable seamless finegrained video color editing. We propose a self-supervised training strategy that synthesizes color transformations and object masks on videos to teach a network to fuse edits. A key innovation is formulating color blending as a fills generation task, rather than binary completion, better retaining details. Explicit global and local processing branches allow multi-scale representation. Further, a temporal transformer propagates relevant features across frames for coherence. Quantitative and subjective evaluations demonstrate MovingColor's capabilities in versatile video color manipulations with state-of-the-art spatial and temporal consistency. The system significantly advances controllable, professional-grade video color 
 
 Appendix:  


Decision: The authors in this paper propose "MovingColor" for fine-grained color enhancement in videos through a self supervised training approach. The proposed method generates high quality results and promising performance based on several quantitative metrics.  The reviewers agree that there are several strengths - paper is well written, experimental results show good performance, dataset, and the propsed method is novel, etc. There are also some limitations in the paper that the reviewers elaborate below. Overall the reviewers agree that the paper can be accepted.  Please address the editorial and other issues that the reviewers identify in the camera ready version.

Reviews: review: paper_summary: paper_summary: This paper introduces MovingColor, a novel self-supervised training approach for fine-grained video color enhancement. This approach tackles flickering artifacts and difficulties of color blending at object boundaries due to unstable video segmentation. It incorporates full-frame textures and color for color fusion that takes into account spatiotemporal contexts to ensure consistency among frames. This proposed method demonstrates promising performance in terms of several evaluation metrics. However, additional clarification and comparisons may be needed for better understanding and evaluation.
strength_weakness: strength_weakness: 1. D5 dataset is generated using D5 Render to address the lack of precise masks for evaluating video color enhancement.
2. A self-supervised training methodology is proposed for color fusion in video editing. A short video that vividly demonstrates this contributions is provided.
3. Multiple evaluation metrics are involved to evaluate the quality of video color fusion from different perspectives.
limitations: limitations: 1. MovingColor is designed to “address spatio-temporal inconsistencies” according to the abstract and introduction, but the method only “performs competitively” in terms of temporal consistency compared with existing video editing methods according to Table 1 and Section 5.3. The framework seems to be similar to existing work such as video inpainting. 
2. At the beginning of the second paragraph in the introduction section, “Harmonizer [14] and DeepLPF [24]” are mentioned as existing approaches for “video color enhancement”, but their main focus is to address image enhancement. The authors may want to improve the accuracy of description and cover comparisons between video color enhancement and related video-based techniques such as video inpainting and video harmonization.
3. What do the red and blue colors refer to in the third sub-figure of Figure 2 (a)?
4. In Section 5.2.2, MovingColor is mentioned as “the first to introduce a color fusion method”, but automated color fusion is also explored in ChromaFusionNet (AAAI 24) and other work. The authors may want to improve the accuracy of description and consider additional comparisons.
paper_summary: paper_summary: This paper introduces a novel video colour enhancement method, MovingColor,  that offers superior visual results by making precise adjustments to specific frame areas, avoiding flickering artifacts and unnatural colour blending at object boundaries due to coarse masks from current video segmentation algorithms. To achieve spatio-temporal inconsistencies, MovingColor utilizes a spectral-spatial hybrid encoder that captures multi-scale spatial and frequency features for better color adjustments in complex scenes. It also incorporates a global-local feature propagation module with Transformer blocks to ensure frame consistency.
strength_weakness: strength_weakness: 1. This work firstly introduces a seamless colour fusion approach for videos, MovingColor, which tackles the problem of natural fusion of fine-grained video color adjustments.
2. This work builds the D5 dataset containing diverse 4K video clips, accurate semantic masks, ambient occlusion, reflection, sky mask, and depth, synthesised using 3D software. It can be used to comprehensively evaluate colour fusion performance and other video tasks.
3. The experiments, including various evaluation metrics, datasets, comparing methods, and  ablation study, are well designed and convincing. 
4. The paper is well written with abundant supplementary materials.
5. The dataset and code will be released in the future.
limitations: limitations: 1. Suboptimal Temporal Consistency: Although the method aims to achieve spatio-temporally consistent color fusion, it does not achieve state-of-the-art performance for several key temporal consistency metrics across three datasets. Meanwhile, it can be observed some noticeable flickering artifacts in provided demo videos. This suggests room for improvement in maintaining temporal coherence in the video sequences.
2. Clarification of Purpose and Application: The paper could benefit from a clearer explanation of the purpose and application scenarios for this task. It is not entirely clear whether this technique can be used for color fusion between two different videos or if it is intended solely for edge-aware color editing within local regions of a single video, functioning as a local region filter. Providing more discussion and examples would help to explain the potential use cases and benefits of this approach.
paper_summary: paper_summary: The paper introduces "MovingColor," a post-processing method for fine-grained color enhancement in videos. It redefines color fusion as a process that utilizes original full-frame textures and color editing information from non-edge areas to generate high-quality results.
strength_weakness: strength_weakness: 1. The idea of MovingColor is very interesting and effective.
2. The structure of the article is clear and progresses logically.
3. The experiments conducted are relatively comprehensive.
limitations: limitations: 1. This method appears to require a certain level of computational training cost. Some methods, such as "Ddcolor: Towards photorealistic image colorization via dual decoders [ICCV23]" and "Colorful image colorization [ECCV16]," retain the L channel values of the original video frames in the LAB space and replace the AB channels with the result's AB channel values. These methods can simply and effectively preserve structural edges, prevent color spillage, and require very short processing times. It is hoped that the authors will compare this post-processing approach and highlight the advantages of "MovingColor."

2. The applicability of this work is somewhat narrow. Although the concept is interesting, it heavily relies on the preliminary coloring results and the presence of some "obvious" boundaries (the cases chosen by the authors mostly have such boundaries).

3. The main text and supplementary materials of the paper and supplementary materials could include more cases, selecting a wider variety of scenes and environments. (Some do not clearly show the advantages, such as the green dog selected in the paper, which looks good in the picture but also has some artifacts in the video).

4. Add punctuation after formulas, line 274 in the paper.