Quantcast

Proposal for erasing background from an image

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Proposal for erasing background from an image

John Tapsell
Hi all,


  I'm guessing this is frowned on but I've written this up on my blog here:

https://johnflux.com/2017/03/02/erasing-background-from-an-image/

  It's mostly images and latex equations, so it's much better to read it
there.  I've pasted it below anyway.

Is anyone interested in this?


I have two opaque images -  one with an object and a background, and
another with just the background.

I want to subtract the background from the image so that the alpha blended
result is visually identical, but the foreground is as transparent as
possible.

I'm sure that this must have been, but I couldn't find a single correct way
of doing this!

I asked a developer from the image editor gimp team, and they replied that
the standard way is to create an alpha mask on the front image from the
difference between the two images.  i.e. for each pixel in both layers,
subtract the rgb values, average that difference between the three
channels, and then use that as an alpha.

But this is clearly not correct.  Imagine the foreground has a green piece
of semi-transparent glass against a red background.  Just using an alpha
mask is clearly not going to subtract the background because you need to
actually modify the rgb values in the top layer image to remove all the red.

So what is the correct solution?  Let's do the calculations.

If we have a solution, the for a solid background with a semi-transparent
foreground layer that is alpha blended on top, the final visual color is:

$latex out_{rgb} = src_{rgb} * src_{alpha} + dst_{rgb} \cdot
(1-src_{alpha})$

We want the visual result to be the same, so we know the value of $latex
out_{rgb}$ - that's our original foreground+background image.  And we know
$latex dst_{rgb}$ - that's our background image.  We want to now create a
new foreground image, $latex src_{rgb}$, with the maximum value of $latex
src_{alpha}$.

So to restate this again - I want to know how to change the top layer
$latex src$ so that I can have the maximum possible alpha without changing
the final visual image at all.  I.e. remove as much of the background as
possible from our foreground+background image.

Note that we also have the constraint that for each color channel, that
$latex src_{rgb} \le 1$ since each rgb pixel value is between 0 and 1.  So:

$latex src_{alpha} \le (out_{rgb} - dst_{rgb})/(1-dst_{rgb})$

So:

$latex
src_{alpha} = Min((out_r - dst_r)/(1-dst_r), out_g - dst_g)/(1-dst_g),
out_b - dst_b)/(1-dst_b))\\
src_{rgb} = (dst_{rgb} \cdot (1-src_{alpha}) - out_{rgb})/src_{alpha}
$

Proposal

Add an option for the gimp eraser tool to 'remove layers underneath', which
grabs the rgb value of the layer underneath and applies the formula using
the alpha in the brush as a normal erasure would, but bounding the alpha to
be no more than the equation above, and modifying the rgb values
accordingly.
_______________________________________________
gimp-developer-list mailing list
List address:    [hidden email]
List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Proposal for erasing background from an image

Simon Budig
John Tapsell ([hidden email]) wrote:
> So to restate this again - I want to know how to change the top layer
> $latex src$ so that I can have the maximum possible alpha without changing
> the final visual image at all.  I.e. remove as much of the background as
> possible from our foreground+background image.

This is what color-to-alpha does. Except that it uses a constant color
for the whole image. It might be interesting to convert this to a gegl
op that takes two input drawable and picks the "background color" from
each corresponding pixel in the background image.

Should be an easy fun hack. Who wants to take it?

Bye,
        Simon

--
              [hidden email]              http://simon.budig.de/
_______________________________________________
gimp-developer-list mailing list
List address:    [hidden email]
List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Proposal for erasing background from an image

Ofnuts-2
On 02/03/17 10:05, Simon Budig wrote:

> John Tapsell ([hidden email]) wrote:
>> So to restate this again - I want to know how to change the top layer
>> $latex src$ so that I can have the maximum possible alpha without changing
>> the final visual image at all.  I.e. remove as much of the background as
>> possible from our foreground+background image.
> This is what color-to-alpha does. Except that it uses a constant color
> for the whole image. It might be interesting to convert this to a gegl
> op that takes two input drawable and picks the "background color" from
> each corresponding pixel in the background image.
>
> Should be an easy fun hack. Who wants to take it?
>
> Bye,
>          Simon
>
Fun hack, but would it be useful in practice? Do we want the lion to be
partially transparent because its hair is more or less the color of the
dry grass (with some help from Darwin). Try the two pictures in
difference mode, and check all the places that are near black.
_______________________________________________
gimp-developer-list mailing list
List address:    [hidden email]
List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Proposal for erasing background from an image

Jay Smith-6
On 03/02/2017 05:22 PM, Ofnuts wrote:

> On 02/03/17 10:05, Simon Budig wrote:
>> John Tapsell ([hidden email]) wrote:
>>> So to restate this again - I want to know how to change the top layer
>>> $latex src$ so that I can have the maximum possible alpha without
>>> changing
>>> the final visual image at all.  I.e. remove as much of the background as
>>> possible from our foreground+background image.
>> This is what color-to-alpha does. Except that it uses a constant color
>> for the whole image. It might be interesting to convert this to a gegl
>> op that takes two input drawable and picks the "background color" from
>> each corresponding pixel in the background image.
>>
>> Should be an easy fun hack. Who wants to take it?
>>
>> Bye,
>>          Simon
>>
> Fun hack, but would it be useful in practice? Do we want the lion to be
> partially transparent because its hair is more or less the color of the
> dry grass (with some help from Darwin). Try the two pictures in
> difference mode, and check all the places that are near black.

If I understand this correctly, an example useful in practice, but the
opposite direction (fill the background instead of clearing it is in a
picture like this (postage stamp surrounded by a black background):

http://jsa.viewimage.net/jsa/web/Lists/Sweden/SpecStamps/sc0012a-f12g2_used-fvfplus_135668_r_l.jpg

(However, I selected this example image to show a problem -- that the
background is "connected" to the dark areas of the postmark where the
postmark reaches the edge of the stamp.)

For 15+ years I have been trying, without success, to solve this overall
situation and automate (or have a useful manually operated tool) the
process of filling the background with 100% black.  The scans are done
on a black background (but it is impossible to scan it 100% black
without distorting other aspects of the image); I have attempted using
other contrasting/unique colors for the background during scanning with
the idea of converting them to black, but there is always some sort of
halo or bleeding at the edge of the stamp's perforations, even when
there is not the "postmark problem".

Jay Smith

_______________________________________________
gimp-developer-list mailing list
List address:    [hidden email]
List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Proposal for erasing background from an image

John Tapsell
In reply to this post by Ofnuts-2
On 3 March 2017 at 07:22, Ofnuts <[hidden email]> wrote:

> On 02/03/17 10:05, Simon Budig wrote:
>
>> John Tapsell ([hidden email]) wrote:
>>
>>> So to restate this again - I want to know how to change the top layer
>>> $latex src$ so that I can have the maximum possible alpha without
>>> changing
>>> the final visual image at all.  I.e. remove as much of the background as
>>> possible from our foreground+background image.
>>>
>> This is what color-to-alpha does. Except that it uses a constant color
>> for the whole image. It might be interesting to convert this to a gegl
>> op that takes two input drawable and picks the "background color" from
>> each corresponding pixel in the background image.
>>
>> Should be an easy fun hack. Who wants to take it?
>>
>> Bye,
>>          Simon
>>
>> Fun hack, but would it be useful in practice? Do we want the lion to be
> partially transparent because its hair is more or less the color of the dry
> grass (with some help from Darwin). Try the two pictures in difference
> mode, and check all the places that are near black.


This is why I was suggesting that we do it with an option for the eraser.
So that the user can have that manual control over it.   How hard would it
be to do this with the erasure tool?

Thanks,

John



> _______________________________________________
> gimp-developer-list mailing list
> List address:    [hidden email]
> List membership: https://mail.gnome.org/mailman
> /listinfo/gimp-developer-list
> List archives:   https://mail.gnome.org/archives/gimp-developer-list
>
_______________________________________________
gimp-developer-list mailing list
List address:    [hidden email]
List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Proposal for erasing background from an image

Developers mailing list
In reply to this post by John Tapsell


On 03/02/2017 02:24 AM, John Tapsell wrote:
> Hi all,
>
>
>   I'm guessing this is frowned on but I've written this up on my blog here:

Absolutely not frowned upon :)

> ...

Hate to be a buzzkill, but it's already doable: clone tool + color erase
mode + registered alignment.

Note that this technique might not work as well as you imagine.  For
each pair of background/output colors, what you essentially do is pass a
ray from the background color, towards the output color, and look for
the ray's intersection with the hull of the RGB cube.  This point is
your foreground color, and the alpha is the relative position of the
output color along the background-foreground segment.  This means, that
all resulting foreground colors are on the hull of the color cube (i.e.,
fully saturated, including black, or full-value), and, conversely, all
output colors that aren't on the hull result in semi-transparent
foreground pixels.

Something that might be fun to try is this: suppose you have two pairs
of background/background+foreground images, where the backgrounds are
different, while the foreground object aligns (as much as possible)
across the two images  Then, at each pixel, you have two
background->output rays, and, ideally, the true foreground color is
their point of intersection.  Most chances are that the rays won't
intersect exactly, because of imperfect alignment/differences in color,
but you can still look for the pair of points, one along each ray, that
minimize the distance between each other; each one would be the
foreground color for the corresponding image pair.

That being said, I imagine you'd need such a controlled environment to
get good results with this, while there are usually simpler
alternatives, that this strikes me as something that will only be useful
is very specialized cases.

--
Ell
_______________________________________________
gimp-developer-list mailing list
List address:    [hidden email]
List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Proposal for erasing background from an image

John Tapsell
On 5 March 2017 at 08:43, Ell via gimp-developer-list <
[hidden email]> wrote:

>
>
> On 03/02/2017 02:24 AM, John Tapsell wrote:
>
>> Hi all,
>>
>>
>>   I'm guessing this is frowned on but I've written this up on my blog
>> here:
>>
>
> Absolutely not frowned upon :)
>
> ...
>>
>
> Hate to be a buzzkill, but it's already doable: clone tool + color erase
> mode + registered alignment.
>

Awesome!

I tested it out, and it actually worked really well for my specific use
case.  I wanted to erase the color from a shadow, leaving just the shadow,
and it did that beautifully.

In the example with the lion, it works really well when you have a yellow
mane against a blue sky.  But completely wrong for a yellow body against
yellow wheat.

I updated the blog post with pictures if you're interested:

https://johnflux.com/2017/03/02/erasing-background-from-an-image/
_______________________________________________
gimp-developer-list mailing list
List address:    [hidden email]
List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list
Loading...