Image Edit - PowerPaint

Image Edit - PowerPaint

🛠

One Mask, Four Jobs — Remove, Fill From a Prompt, Rebuild a Border, or Fill a Shape

PowerPaint is a versatile inpainting model from the open-mmlab / Shanghai AI Laboratory team (ECCV 2024). Draw one mask and pick what should happen inside it: erase the object and rebuild the background, generate something new from a text prompt, extend the image past its original edge, or generate an object that conforms to the mask's outline. All four run on the same weights — the mode is selected by a learned task prompt, not by swapping models.

CNAPS Studio runs the PowerPaint v2-1 checkpoint. Weights are Apache 2.0; the reference code is MIT.

What it does

Most inpainting models are tuned for one intent. A model trained to fill a region with plausible background is optimised differently from one trained to synthesise a described object — which is why removal tools and generative-fill tools are usually separate products.

PowerPaint's contribution is doing both from one set of weights. The authors trained learnable task prompts — special tokens the model learned alongside the image data, one per inpainting intent — and select behaviour by prepending the right token. Two of these do most of the work: one steers the model toward synthesising an object, the other toward continuing the surrounding context. Object removal is then an elegant trick rather than a separate model: the object token is used as a negative prompt under classifier-free guidance, actively pushing the model away from generating anything in the masked region. Shape-guided fill comes from interpolating between the shape token and the context token, which is what makes the fitting degree a continuous dial rather than an on/off switch.

In CNAPS Studio this surfaces as a single dropdown — What to do with the masked region — and everything else stays the same between modes.

Problem it solves

  • Four editing jobs, one node — Removal, generative fill, outpainting and shape-guided insertion share a mask, a node and a set of parameters. No re-wiring the flow to switch intent
  • Removal that actually removes — Using the object prompt as a negative is materially different from filling a mask with a generic background model; it suppresses the model's tendency to hallucinate a replacement object into the hole
  • Outpainting without a separate tool — Extend a frame past its border for a different aspect ratio or a wider composition, using the same mask mechanic
  • Shape control when you want it, and not when you don't — A high fitting degree makes the generated object conform to the mask outline; a low one lets it take its natural shape inside the masked area
  • Everything outside the mask stays untouched — An explicit option keeps unmasked pixels bit-identical to the input, so the node is safe on assets where only one region may legally or editorially change
  • Light enough for batch — A Stable Diffusion 1.5-class model, not a 20B-parameter editor. Reasonable to run across hundreds of images

Input/Output

  • Input:three ports. In CNAPS Studio a filled connection point means required, a hollow one means optional.
    • Image — the source image (required)
    • Mask — the region to operate on (required)
    • Text prompt — what to generate inside the mask (optional)

      • The prompt is optional because two of the four modes don't use one: remove and rebuild a border work from context alone. For fill from a prompt and fill a shape, a prompt is what drives the result — leaving it empty in those modes will not fail, but it will give you context-fill behaviour instead of what you asked for. Port identity (which of the two image-type ports is the image and which is the mask) is inferred from port order and should be confirmed against the node before it's relied on in a written tutorial.

  • Output: a single edited image
    • Masked region regenerated according to the selected mode
    • With the pixel-identical option on, everything outside the mask matches the input exactly
    • No mask, confidence map or intermediate is returned
  • Parameters
    • What to do with the masked region (dropdown) — selects the task prompt. Four modes: The dropdown's exact label strings weren't captured beyond text-guided; the four behaviours are from the node description and the upstream model.
      • Mode
      What it does
      Prompt
      text-guided (shown selected)
      Generates the described object inside the mask
      Required in practice
      remove
      Erases what's there and rebuilds background from context
      Not needed
      rebuild a border (outpainting)
      Extends the image past its original edge
      Not needed
      fill a shape (shape-guided)
      Generates an object that conforms to the mask outline
      Required in practice
    • Diffusion steps (slider, 10 – 50, default 30)
    • Random seed (1 = random) — fix it to reproduce a result

      • Under Advanced

    • How strongly the prompt steers the fill (1 – 20, default 7.5) — guidance scale. For removal the upstream authors recommend 10 or higher, and raising it further if unwanted objects keep appearing in the masked area. The default of 7.5 is tuned for generative fill, not for removal
    • How literally shape-guided follows the mask outline (0 – 1, default 1) — the fitting degree, and it only affects shape-guided mode. The default sits above the range the authors recommend: roughly 0.8 – 0.95 when you want the object to fit the mask shape, and 0.5 – 0.6 when you don't. At 1 the object is forced to the outline, which is right for a bread in a square mask and wrong for a rabbit in one
    • How tightly the fill follows the surrounding pixels (0 – 2, default 1) — how strongly the surrounding image conditions the generated region. Lower gives the model more freedom; higher binds it to its surroundings
    • Negative prompt (text, empty by default) — what to keep out of the filled area
    • Keep everything outside the mask pixel-identical to the input (checkbox, on by default) — composites the generated region back over the original rather than returning a fully re-encoded image. Leave it on unless you have a reason not to; a VAE round-trip subtly shifts every pixel in the frame
    • Expand the mask outward by this many pixels (default 8) — dilates the mask to avoid a faint outline where the original object's edge used to be. Raise it if you can still see a ghost of what you removed

Performance

Family
Stable Diffusion 1.5-class latent diffusion. v2 and v2-1 add a BrushNet-style conditioning branch
Checkpoint
PowerPaint v2-1 — released 22 May 2024, fixing training issues present in v2
Parameter count
Not published on the model card. SD-1.5 lineage puts it in the sub-1B UNet class rather than the multi-billion editor class
Native resolution
Not documented by the provider — SD 1.5-family models are trained at 512 px and degrade at large deviations. Verify on your own asset sizes
Diffusion steps
10 – 50 (node), default 30
ControlNet
The upstream model is ControlNet-compatible (canny, depth, HED, pose). Not exposed on this node
End-to-end node timing
Not yet measured — verify before committing to a batch schedule
‣
Technical Details (for engineers)

How PowerPaint compares

CNAPS Studio has several ways to edit inside a mask. They differ mainly in scope and weight class.

LatentDiffusion (Object Removal)
QWEN-Inpaint
PowerPaint
Remove an object
✅
✅
✅
Generate from a prompt
❌
✅
✅
Outpaint past the border
❌
❌
✅
Shape-guided fill
❌
❌
✅ with a fitting dial
Model class
Latent diffusion
Qwen-Image-Edit-2511
SD 1.5-class + BrushNet branch

Rough guidance: PowerPaint when you want one node to cover several editing intents, or when you specifically need outpainting or shape control. A larger editor when single-image fidelity matters more than throughput. Scope claims here come from each node's own documentation; the models have not been benchmarked against each other inside CNAPS Studio.

All of these run inside CNAPS Studio — compare them side-by-side.

Model Source

  • Hugging Face:

Compliance & Provenance

Provider
PowerPaint team — Junhao Zhuang et al., published via open-mmlab (Shanghai AI Laboratory / Tsinghua)
Provider type
Specialized
License
Model weights Apache 2.0 (Hugging Face); reference implementation MIT (GitHub). Both permissive, no territorial or revenue restrictions
EU AI Act risk class
Limited Risk
Art. 50 transparency
Required — output is AI-modified image content and is marked with a visible AI label and machine-readable provenance. See AI Policy §2.
Region availability
Available globally
Training data summary
Described in the paper; the provider has not published a structured summary per Art. 53(d)

For more on how we classify models and mark outputs, see our AI Policy.

Run it now