Diverse Semantic Image Editing With Style Codes

Semantic image editing involves inpainting pixels guided by a semantic map. This is a challenging task, as the inpainted regions must both align harmoniously with the surrounding context and strictly adhere to the semantic constraints. Most prior methods approach this by attempting to encode all necessary information from the erased regions alone. However, when adding new objects-such as a car-to a scene, their style often cannot be inferred solely from the surrounding context. On the other hand, the models that can output diverse generations struggle to output images that have seamless boundaries between the generated and unerased parts. In this work, we propose a framework that can encode visible and partially visible objects with a novel mechanism to achieve consistency in the style encoding and final generations. We extensively compare with previous conditional image generation and semantic image editing algorithms. Our extensive experiments show that our method significantly improves over the state-of-the-art. Our method not only achieves better quantitative results but also provides diverse results. Demo and code will be released.