deriving transform by comparing image before/after edit

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

deriving transform by comparing image before/after edit

Developers mailing list
Given a pair of before/after jpeg photos edited with global-effect
commands (vs. operations on selected areas), is it possible to derive
transformation matrix/matrices that reproduce the result from the
original? Presumably by iterating over all the pixels in a 1:1 mapping.

My hope is to train neural nets to predict the matrix operation(s)
required. Example:

http://phobrain.com/pr/home/gallery/pair_vert_manual_9_2845x2.jpg

Thanks,

Bill


--
Phobrain.com
_______________________________________________
gimp-developer-list mailing list
List address:    [hidden email]
List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list
Reply | Threaded
Open this post in threaded view
|

Re: deriving transform by comparing image before/after edit

Developers mailing list
Hi Bill,

I don't know of plugins that would calculate a matrix in GIMP. There's
a plugin called GetCurves by Elsamuko [1] that calculates a rough
approximation of a color transform between two layers and creates an
RGB curves preset.

Due to its nature, the plugin has obvious issues treating different
hue ranges differently [2] but maybe it's a motivation for someone to
come up with a better tool. Although I'd be damned if I knew how you
would then apply a generated 3x3 matrix to a different image :)

I recently looked at free/libre options to create 3D LUT files, and
the existing options like Harlequin [3] are not very encouraging yet.

[1] https://github.com/elsamuko/gimp-elsamuko/tree/master/plugins/elsamuko-get-curves
[2] https://twitter.com/lgworld/status/1338597821881192448
[3] https://github.com/dunkyp/harlequin/

Alex

On Sun, Dec 27, 2020 at 2:47 PM Bill Ross via gimp-developer-list
<[hidden email]> wrote:

>
> Given a pair of before/after jpeg photos edited with global-effect
> commands (vs. operations on selected areas), is it possible to derive
> transformation matrix/matrices that reproduce the result from the
> original? Presumably by iterating over all the pixels in a 1:1 mapping.
>
> My hope is to train neural nets to predict the matrix operation(s)
> required. Example:
>
> http://phobrain.com/pr/home/gallery/pair_vert_manual_9_2845x2.jpg
>
> Thanks,
>
> Bill
>
>
> --
> Phobrain.com
> _______________________________________________
> gimp-developer-list mailing list
> List address:    [hidden email]
> List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
> List archives:   https://mail.gnome.org/archives/gimp-developer-list
_______________________________________________
gimp-developer-list mailing list
List address:    [hidden email]
List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list
Reply | Threaded
Open this post in threaded view
|

Re: deriving transform by comparing image before/after edit

Developers mailing list
Thanks Alexandre!

I figured I'd most likely be writing the code if the math is possible -
it would start as a standalone batch prog that could be appified. But
I'd need an obvious path to even start, since I'm more of a
jack-of-all-trades programmer, e.g. not sure how close LUTs could come.

I realized it would be capable of reverse engineering any global
filters, so could irk anyone who makes money from them.

Here's where I tried the question earlier:

https://www.reddit.com/r/computervision/comments/kktdq4/derive_transformation_matrix_from_two_photos/

My current editing backlog is 150K photos.

Bill


On 12/27/20 4:06 AM, Alexandre Prokoudine via gimp-developer-list wrote:

> Hi Bill,
>
> I don't know of plugins that would calculate a matrix in GIMP. There's
> a plugin called GetCurves by Elsamuko [1] that calculates a rough
> approximation of a color transform between two layers and creates an
> RGB curves preset.
>
> Due to its nature, the plugin has obvious issues treating different
> hue ranges differently [2] but maybe it's a motivation for someone to
> come up with a better tool. Although I'd be damned if I knew how you
> would then apply a generated 3x3 matrix to a different image :)
>
> I recently looked at free/libre options to create 3D LUT files, and
> the existing options like Harlequin [3] are not very encouraging yet.
>
> [1] https://github.com/elsamuko/gimp-elsamuko/tree/master/plugins/elsamuko-get-curves
> [2] https://twitter.com/lgworld/status/1338597821881192448
> [3] https://github.com/dunkyp/harlequin/
>
> Alex
>
> On Sun, Dec 27, 2020 at 2:47 PM Bill Ross via gimp-developer-list
> <[hidden email]> wrote:
>> Given a pair of before/after jpeg photos edited with global-effect
>> commands (vs. operations on selected areas), is it possible to derive
>> transformation matrix/matrices that reproduce the result from the
>> original? Presumably by iterating over all the pixels in a 1:1 mapping.
>>
>> My hope is to train neural nets to predict the matrix operation(s)
>> required. Example:
>>
>> http://phobrain.com/pr/home/gallery/pair_vert_manual_9_2845x2.jpg
>>
>> Thanks,
>>
>> Bill
>>
>>
>> --
>> Phobrain.com
>> _______________________________________________
>> gimp-developer-list mailing list
>> List address:    [hidden email]
>> List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
>> List archives:   https://mail.gnome.org/archives/gimp-developer-list
> _______________________________________________
> gimp-developer-list mailing list
> List address:    [hidden email]
> List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
> List archives:   https://mail.gnome.org/archives/gimp-developer-list
--
Phobrain.com
_______________________________________________
gimp-developer-list mailing list
List address:    [hidden email]
List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list
Reply | Threaded
Open this post in threaded view
|

Re: deriving transform by comparing image before/after edit

Developers mailing list
In reply to this post by Developers mailing list
Bill,

unfortunately the result of a convolution operation can not always be
inverted. The most simple example is a convolution matrix (=folding
matrix) with all zeroes. It definitely wipes the whole image and that
can not be undone!

Unfortunately, the situation is such that the result of a matrix
operation almost never can be inverted. Here an explanation with a
little bit more reference to mathematics, leaving out the details of not
square sized pictures and convolution matrices like those in Gimp, which
only have up to 5*5 entries (think of them as being larger, expanded
with all zeroes around it to all sides to pad them to the image size):

Let M denote the Matrix and FT(M) the fourier transform of it (expanded
to the image size). The Fourier transform is a terribly complicated
thing and in fact, the result of a real valued picture becomes a complex
valued picture, i.e. two pictures: one for the real part and another one
for the imaginary one. Fortunately in mathematics (but not in Gimp)
there also is an inverse Fourier transform which is as complicated as
the FT itself, but once one has a program for FT, one also has one for
the inverse FT. Of course, the FT and its inverse operate on complex
valued "pictures", but you normally only look onto the real part of them
in the non-transformed image. (The FT image is often referred to as in
the frequency range: small and "high frequency" details in the picture
become large in the FT and vice versa, it is all really "convolved", as
one may say in the double meanig of this word).

Let P denote the picture and FT(P) the fourier transform of the picture.

Let M#P denote the convolution of M and P

Then FT(M#P) = FT(M)*FT(P) (point-wise multipliaction)

If FT(M) has entries with zeroes in it, these zeroes will cancel their
respective pixels in FT(P). This of course means that FT(M#P) can no
longer be FT(P). Unfortunatly almost all FT(M) would have zeroes
somewhere, if the convolution matrix would have image size.

The matrix convolution can be computed very effectively and it is quite
powerful, especially for small matrices. Fortunately it now works in
Gimp 2.10.22 (it did not work in earlier versions of 2.10) and
fortunately now we can load and store the convolution matrices.

I frequently convolve scans of documents with something like this

0    0    0    0    0

0   -1   -1   -1   0

0   -1    c   -1   0

0   -1   -1   -1   0

0    0    0    0    0

wherer c ins the centre-weight, a number which is larger than the
negative value of the sum of all other (negative) weights, e.g. c
somewhere in the range 9...12.

This matrix convolution subtracts from each pixel a weighted mean of the
surrounding ones. The purpose of such a filter is to remove background
like shadows or stains from the document before I apply OCR to it
(that's why I do this operation). By choosing c larger than the negative
sum of the surrounding ones, you can give the original picture more
emphasis than to the reduction of the background.

My convolution filter has some "differentiating property" which lets it
emphasize noise in the picture. In order to mitigate that, one can first
apply some gaussian filter. Another way is to integrate the the blurring
(local average) into the convolution matrix, i.e. simultaneously do some
kind of smoothing for the centre, for example

-1  -1  -1  -1  -1

-1    c   c   c  -1

-1    c   c   c  -1

-1    c   c   c  -1

-1  -1  -1  -1  -1

with some appropriate centre weight such that the sum of the c's ie a
bit larger that the negative sum of the -1's around. The centre part is
a box convolution which makes the image unsharp. (It would be nice if
one could change all the centre- weights in one place, but once you
think about that, you trigger the desire to shape the centre weights
like a bell shaped function, which comes closer to gaussian unsharpness.
Indeed, Gaussian unsharpness can almost be approximted by several times
folding with a box function with only positive values).

Of course it would be great, if the convolution matrix could be made
larger than 5*5 for such folding matrices to remove background.

For removing stains and shadows, it would also be beneficial if one
could apply some gray level transformation to each pixel of the image
(e.g. given by a polynominal) before the pixel value is mutiplied with
the matrix entry in the convolution operation and to apply some other
gray  level transformation (one which mostly anihilates the first one)
on the way back before the convolution image is shown.

In addition I would whish me something where I could apply two
convolution matrices to an image: the first one acting like now, but the
second one just to compute some "comparison value" which is used to set
the pixel to "black" if the outcome of the first mentioned convolution
is greater or equal to the outcome of the second convolution. Think of
the second convolution going to some "alpha channel".

But it would become necessary to be able to store and reload all those
matrices and the coefficients of the gray level transformations and the
coefficients of the comparison operation with that second convolution 9n
norder to be usefull.  Great things could be done with Gimp if there
would be some easy to perate playground for such image manipulations!

Try my background removal operation with the appended example file,
which is just a small part from an image taken from a newspaper. Play
with it with different values of the centre weight. Of course, it would
be best to first reduce the immage to a grayvalue-only image, i.e.
without colours.

Adalbert


Am 27.12.20 um 07:52 schrieb Bill Ross via gimp-developer-list:

> Given a pair of before/after jpeg photos edited with global-effect
> commands (vs. operations on selected areas), is it possible to derive
> transformation matrix/matrices that reproduce the result from the
> original? Presumably by iterating over all the pixels in a 1:1 mapping.
>
> My hope is to train neural nets to predict the matrix operation(s)
> required. Example:
>
> http://phobrain.com/pr/home/gallery/pair_vert_manual_9_2845x2.jpg
>
> Thanks,
>
> Bill
_______________________________________________
gimp-developer-list mailing list
List address:    [hidden email]
List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list
Reply | Threaded
Open this post in threaded view
|

Re: deriving transform by comparing image before/after edit

Developers mailing list
Thanks Adalbert,

I don't want to reverse the transformation, just characterize the
forward transformation enough so that I can train neural nets to suggest
one I like when shown any photo. You might say I want to use my
before/afters to derive a filter for each photo, then train nets with
large numbers of

    input_pic_i -> filter_i

so the net can learn to improvise a new filter for each new pic to my
taste.

As someone just put it on reddit:

    Params = f(jpeg, histogram) [f() is a neural net]

    and then for each pixel in the jpeg: rgb_transformed = engine(rgb,
    Params)

Alternatively, I could try having nets predict the output pic's pixels
directly, but it seems maybe too big for consumer GPU's.

Interface-wise, a plugin would 'watch' you edit and give you suggestions
if you asked.

Here's an example of how neural nets can pick up my taste in pairing photos:

    http://phobrain.com/pr/home/siagal.html

Bill


On 12/28/20 1:21 AM, Adalbert Hanßen wrote:

> Bill,
>
> unfortunately the result of a convolution operation can not always be
> inverted. The most simple example is a convolution matrix (=folding
> matrix) with all zeroes. It definitely wipes the whole image and that
> can not be undone!
>
> Unfortunately, the situation is such that the result of a matrix
> operation almost never can be inverted. Here an explanation with a
> little bit more reference to mathematics, leaving out the details of not
> square sized pictures and convolution matrices like those in Gimp, which
> only have up to 5*5 entries (think of them as being larger, expanded
> with all zeroes around it to all sides to pad them to the image size):
>
> Let M denote the Matrix and FT(M) the fourier transform of it (expanded
> to the image size). The Fourier transform is a terribly complicated
> thing and in fact, the result of a real valued picture becomes a complex
> valued picture, i.e. two pictures: one for the real part and another one
> for the imaginary one. Fortunately in mathematics (but not in Gimp)
> there also is an inverse Fourier transform which is as complicated as
> the FT itself, but once one has a program for FT, one also has one for
> the inverse FT. Of course, the FT and its inverse operate on complex
> valued "pictures", but you normally only look onto the real part of them
> in the non-transformed image. (The FT image is often referred to as in
> the frequency range: small and "high frequency" details in the picture
> become large in the FT and vice versa, it is all really "convolved", as
> one may say in the double meanig of this word).
>
> Let P denote the picture and FT(P) the fourier transform of the picture.
>
> Let M#P denote the convolution of M and P
>
> Then FT(M#P) = FT(M)*FT(P) (point-wise multipliaction)
>
> If FT(M) has entries with zeroes in it, these zeroes will cancel their
> respective pixels in FT(P). This of course means that FT(M#P) can no
> longer be FT(P). Unfortunatly almost all FT(M) would have zeroes
> somewhere, if the convolution matrix would have image size.
>
> The matrix convolution can be computed very effectively and it is quite
> powerful, especially for small matrices. Fortunately it now works in
> Gimp 2.10.22 (it did not work in earlier versions of 2.10) and
> fortunately now we can load and store the convolution matrices.
>
> I frequently convolve scans of documents with something like this
>
> 0    0    0    0    0
>
> 0   -1   -1   -1   0
>
> 0   -1    c   -1   0
>
> 0   -1   -1   -1   0
>
> 0    0    0    0    0
>
> wherer c ins the centre-weight, a number which is larger than the
> negative value of the sum of all other (negative) weights, e.g. c
> somewhere in the range 9...12.
>
> This matrix convolution subtracts from each pixel a weighted mean of the
> surrounding ones. The purpose of such a filter is to remove background
> like shadows or stains from the document before I apply OCR to it
> (that's why I do this operation). By choosing c larger than the negative
> sum of the surrounding ones, you can give the original picture more
> emphasis than to the reduction of the background.
>
> My convolution filter has some "differentiating property" which lets it
> emphasize noise in the picture. In order to mitigate that, one can first
> apply some gaussian filter. Another way is to integrate the the blurring
> (local average) into the convolution matrix, i.e. simultaneously do some
> kind of smoothing for the centre, for example
>
> -1  -1  -1  -1  -1
>
> -1    c   c   c  -1
>
> -1    c   c   c  -1
>
> -1    c   c   c  -1
>
> -1  -1  -1  -1  -1
>
> with some appropriate centre weight such that the sum of the c's ie a
> bit larger that the negative sum of the -1's around. The centre part is
> a box convolution which makes the image unsharp. (It would be nice if
> one could change all the centre- weights in one place, but once you
> think about that, you trigger the desire to shape the centre weights
> like a bell shaped function, which comes closer to gaussian unsharpness.
> Indeed, Gaussian unsharpness can almost be approximted by several times
> folding with a box function with only positive values).
>
> Of course it would be great, if the convolution matrix could be made
> larger than 5*5 for such folding matrices to remove background.
>
> For removing stains and shadows, it would also be beneficial if one
> could apply some gray level transformation to each pixel of the image
> (e.g. given by a polynominal) before the pixel value is mutiplied with
> the matrix entry in the convolution operation and to apply some other
> gray  level transformation (one which mostly anihilates the first one)
> on the way back before the convolution image is shown.
>
> In addition I would whish me something where I could apply two
> convolution matrices to an image: the first one acting like now, but the
> second one just to compute some "comparison value" which is used to set
> the pixel to "black" if the outcome of the first mentioned convolution
> is greater or equal to the outcome of the second convolution. Think of
> the second convolution going to some "alpha channel".
>
> But it would become necessary to be able to store and reload all those
> matrices and the coefficients of the gray level transformations and the
> coefficients of the comparison operation with that second convolution 9n
> norder to be usefull.  Great things could be done with Gimp if there
> would be some easy to perate playground for such image manipulations!
>
> Try my background removal operation with the appended example file,
> which is just a small part from an image taken from a newspaper. Play
> with it with different values of the centre weight. Of course, it would
> be best to first reduce the immage to a grayvalue-only image, i.e.
> without colours.
>
> Adalbert
>
>
> Am 27.12.20 um 07:52 schrieb Bill Ross via gimp-developer-list:
>> Given a pair of before/after jpeg photos edited with global-effect
>> commands (vs. operations on selected areas), is it possible to derive
>> transformation matrix/matrices that reproduce the result from the
>> original? Presumably by iterating over all the pixels in a 1:1 mapping.
>>
>> My hope is to train neural nets to predict the matrix operation(s)
>> required. Example:
>>
>> http://phobrain.com/pr/home/gallery/pair_vert_manual_9_2845x2.jpg
>>
>> Thanks,
>>
>> Bill
--
Phobrain.com
_______________________________________________
gimp-developer-list mailing list
List address:    [hidden email]
List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list
Reply | Threaded
Open this post in threaded view
|

Re: deriving transform by comparing image before/after edit

Liam R E Quin
In reply to this post by Developers mailing list
On Mon, 2020-12-28 at 10:21 +0100, Adalbert Hanßen via gimp-developer-
list wrote:
> Bill,
>
> unfortunately the result of a convolution operation can not always be
> inverted. 

It turns out that inversion is not needed. In cases like the example
shown, colour grading was probably all that was done (i didn't check
carefully). The term "matrix" in the post was i think perhaps not to be
taken literally.

For colour mapping/grading, there's a plugin included in gimp i think
that will take the colour from  one layer and apply it to another,
which might go a long way. And i think Produkine mentioned a script
someone wrote that tries to deduce a set of curves for colour/levels.

Many other transformations are much harder or even impossible, as
Adalbert states.

>
> Try my background removal operation with the appended example file,

Attachments do not get passed through the mailing list.


liam


--
Liam Quin - web slave for https://www.fromoldbooks.org/

full-time slave in voluntary servitude

_______________________________________________
gimp-developer-list mailing list
List address:    [hidden email]
List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list
Reply | Threaded
Open this post in threaded view
|

Re: deriving transform by comparing image before/after edit

Developers mailing list
In reply to this post by Developers mailing list
Hello Bill,

your problem could be solved much easier and doing that would open great
new things to Gimp:

Let us for short think of all of Gimp's functions (I just cal them
functi, but their names of coursee would be unique within Gimp), which
are accessible from any menu function, being split into a parameterizing
function parametrize_functi and an actual operating one operate_functi, i.e.

* parametrize_functi just asks for all parameters (if there are any) but
does not process them

* imagine thatparametrize_functi createsa log entry "functi, pi1, ....,
pin"with all necessary parameters which it has obtained when the ok
button is pressed (if there is one),

* those log entries would be recorded.

* when the ok button is pressed,operate_functi functions would be called
through one central placewith those recorded parameters. Thefuncti
areidentified by the firstentry in each recorded parameter set.

Then it would become very easy to automatically re-run such recorded
steps . Your question “How exactly did I achieve this result?” would
become superfluous.

Would that not be great?

In Gimp, all theses log entries should be recorded strictly inthe
sequence in which they are done. Of course, the number of the tab to
which they go (and the plane in it and son on) also have to be recorded.
When the clipboard is used, that has to be recorded too and if the
clipboard-content doesnot originate from this particular Gimp session, a
copy of it has to be stored to make everything reconstructible. (IfGimp
does not get notified when the system clipboard is changed outside of
Gimp, Gimp has to maintain ashadow copy of a clipboard with an image and
it has to compare the current system clipboard containing an image to
its own shadow copy, in order to storea reconstruction copy of the
system clipboard image, if any operation happens, where the system
clipboard from outside of Gimpis actually used in Gimp).

With such log-keeping, even all “destructive” functions could be undone
in Gimp without much storage requirements: In the worst case, the
sequence of processing steps would have to be redone from the very
beginning. One could speed that up by storingintermediate result toa
file, until the session ends.

When a picture is stored or exported, some cleanup has to be done to
eliminate all logged operations and possibly also all stored system
clipboards which entered the flow of operations but do not not influence
any non-stored result any more. Idemat the end of a session or closing a
tab,

Of course, loading an image has also a parametrize function: The one
which lets you select the image to be loaded and the tab to which it
goes. Of course, switching to anothertab also has to be recorded. When
you copy something from one tab and paste it e.g. to a plane in another
tab, the associated processing logs become “mangled” because the
pasted-to history now also depends on the history of the pasted-from tab
and its associated image.

The number of parameters of the functi differ from one function to
anotherone. There could be none, like in transform to grayscale. There
could be only a few, like in setting a threshold transforming to B/W or
when doing some brightness or colour manipulations. There would also
onlybe a few like when pasting something from the clipboard to a
specified place (i.e. the coordinates of the placement and possibly an
orientation angle). There could be functions which have many parameters,
like cutting out some fancy shape.

The whole log filecould be a plain ASCII file! It would besomething like

tab1, openFile, full_path_to_it

tab1, rotateImage, centerx, centery, degrees

tab1, cropImage, x1, y1, x2, y2

tab2, openFile, full_path_to_that

....

All selection operations, all cloning operations, all matrices used in
convolutions, all colour adaptation operations, really everything you
can do with Gimp could be recorded like this. Theoretically you could do
that by hand by just exactly write down each processing step together
with its parameters. But some functions have quite a lot of parameters,
like e.g. cutting parts with an irregular shape, or using
paintbrush-operations and the like. But a computer does not get tired
when it is told to do tedious things over and over.



Debug and improve mode

In such an Interpreter approach, one could also provide a mode, where
each parameterizingfunction is called again, but then all parameterizing
steps are preset as they were done when the Gimp session was recorded.
In the debug and improve mode the user can vary what wasdone before and
continue, when the varied parameters have been input.

If proper construction schemes are used, the parameterizing functions in
this special mode are exactly the same ones as they are used in the
normal mode, the only difference being that instead of only an ok
button, you also haveadditional buttons "interactive", "skip" and "exit".

* When you click on "interactive", you continue with arbitrary Gimp
processing steps, returning to the place after each step where you
clicked "interactive" in the old recorded procedure after each Gimp
processing step. Of course, during such an operation, all additional
functions and parameters are recorded as well, such that the whole debug
and improvesession yieldsa new plain ASCII description of all Gimp steps
used to achieve the new final result.

* When you click in "skip", the actual processing step is left out and
Gimp proceeds to the next step from the processing list.

* When you click on “exit”, the whole batch processing in the special
debug and improve mode is left and Gimp continues with its normal mode
(i.e. without continuing to process not yet processed recorded logged
operations).



Feasibility

Most probably, the main conditions for such an approach are already
there, the only missing thing would be to create and manage the log entries.

I have never looked at a single line of source code of Gimp, neither
have Iunderstood the programming paradigmsof Skript-Fu or of the other
programming tools. Perhaps almost all what I spoke abut is already
there. Possiblythe basics are also already in Gimp’s native image
storage format. Are they?



Further Benefits

1. It becomes possible to also include the logged processing stepsinto
the metadata of an exportedresult, even into aJPEG-file! Of course all
processing steps in other tabs which had no influence aparticular stored
or exported oneshould be cancelled in that.

2. Later edit such a sequence in order to improve a result achieved
earlier, but with a slight modification!

Recorded log fileswould be a great teaching resource: One could publish
“best practice” sequences: The original picture + all processing steps
leading to a particular result. The steps could be numbered as above and
some explanations why the authordid the particular processing steps
could be published in addition.

For that purpose I proposeto let the Gimp user record oral explanation
blocks to each processing step. If the authorgoes back some steps, I
would automatically wind up therespective oral explanations and replace
them with new ones, if he decides to proceed differently.



In the firs half of the 1980’s I lead a project, I which we designed an
electronics unit controlling microscope equipment. It was designed in
Modula 2 and it used exactly such an architecture. This system was using
an 8086 microprocessor and firmware completely in ROM. Butit could be
adapted by plugging additional function boardsinto the rack. The central
point in the software were lists, containing function names, boundary
conditions for the parameters and the like plus the entry points of the
operational functions andthe stack lengths of the parameter-depot for
them. The real sequence of operational functions could be called almost
as quickly in sequence as if they had been written in sequence as
function calls. All parameter checking, all conversions and so on were
done before.


Adalbert


2018-07-08_Standard

Am 28.12.20 um 11:02 schrieb Bill Ross:

>
> Thanks Adalbert,
>
> I don't want to reverse the transformation, just characterize the
> forward transformation enough so that I can train neural nets to
> suggest one I like when shown any photo. You might say I want to use
> my before/afters to derive a filter for each photo, then train nets
> with large numbers of
>
>     input_pic_i -> filter_i
>
> so the net can learn to improvise a new filter for each new pic to my
> taste.
>
> ...
>
> Here's an example of how neural nets can pick up my taste in pairing
> photos:
>
>     http://phobrain.com/pr/home/siagal.html
>
> Bill
>
_______________________________________________
gimp-developer-list mailing list
List address:    [hidden email]
List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list
Reply | Threaded
Open this post in threaded view
|

Re: deriving transform by comparing image before/after edit

Developers mailing list
Hi Adlbert,

That might add all kinds of great functionality to gimp, enabling it
could be optional, although wouldn't handle the old edits I want to
train nets on so I can just stop editing 'now'. :-)

In planning it, I'd think of what core functions are required for basic
touchup, so a network could be trained to make optimal use of them once
they could be worked out mathematically from all one's old edits
(worst-case, autoadjust 20? numerical sliders til you get the least net
'distance' from the edited pic, likely faster with some optimization
library, then accept or reject each best-effort by visual inspection
before it is used for training). It'd be like having your own custom
version of equalization but I expect it'd be good and final about 80-90%
of the time for people who spend less than a few minutes editing each
photo. I wonder if a joint effort with ImageMagick or other groups might
be worthwhile, given the possibility for shell-scripting as well as
within workbenches.

I'm being gradually educated / figuring out how NNs might be applied in
further discussion here:

https://www.reddit.com/r/computervision/comments/kktdq4/derive_transformation_matrix_from_two_photos/

Bill

https://twitter.com/photoriot/status/1344064933248421889?s=20

On 12/29/20 4:46 PM, Adalbert Hanßen via gimp-developer-list wrote:

> Hello Bill,
>
> your problem could be solved much easier and doing that would open great
> new things to Gimp:
>
> Let us for short think of all of Gimp's functions (I just cal them
> functi, but their names of coursee would be unique within Gimp), which
> are accessible from any menu function, being split into a parameterizing
> function parametrize_functi and an actual operating one operate_functi, i.e.
>
> * parametrize_functi just asks for all parameters (if there are any) but
> does not process them
>
> * imagine thatparametrize_functi createsa log entry "functi, pi1, ....,
> pin"with all necessary parameters which it has obtained when the ok
> button is pressed (if there is one),
>
> * those log entries would be recorded.
>
> * when the ok button is pressed,operate_functi functions would be called
> through one central placewith those recorded parameters. Thefuncti
> areidentified by the firstentry in each recorded parameter set.
>
> Then it would become very easy to automatically re-run such recorded
> steps . Your question “How exactly did I achieve this result?” would
> become superfluous.
>
> Would that not be great?
>
> In Gimp, all theses log entries should be recorded strictly inthe
> sequence in which they are done. Of course, the number of the tab to
> which they go (and the plane in it and son on) also have to be recorded.
> When the clipboard is used, that has to be recorded too and if the
> clipboard-content doesnot originate from this particular Gimp session, a
> copy of it has to be stored to make everything reconstructible. (IfGimp
> does not get notified when the system clipboard is changed outside of
> Gimp, Gimp has to maintain ashadow copy of a clipboard with an image and
> it has to compare the current system clipboard containing an image to
> its own shadow copy, in order to storea reconstruction copy of the
> system clipboard image, if any operation happens, where the system
> clipboard from outside of Gimpis actually used in Gimp).
>
> With such log-keeping, even all “destructive” functions could be undone
> in Gimp without much storage requirements: In the worst case, the
> sequence of processing steps would have to be redone from the very
> beginning. One could speed that up by storingintermediate result toa
> file, until the session ends.
>
> When a picture is stored or exported, some cleanup has to be done to
> eliminate all logged operations and possibly also all stored system
> clipboards which entered the flow of operations but do not not influence
> any non-stored result any more. Idemat the end of a session or closing a
> tab,
>
> Of course, loading an image has also a parametrize function: The one
> which lets you select the image to be loaded and the tab to which it
> goes. Of course, switching to anothertab also has to be recorded. When
> you copy something from one tab and paste it e.g. to a plane in another
> tab, the associated processing logs become “mangled” because the
> pasted-to history now also depends on the history of the pasted-from tab
> and its associated image.
>
> The number of parameters of the functi differ from one function to
> anotherone. There could be none, like in transform to grayscale. There
> could be only a few, like in setting a threshold transforming to B/W or
> when doing some brightness or colour manipulations. There would also
> onlybe a few like when pasting something from the clipboard to a
> specified place (i.e. the coordinates of the placement and possibly an
> orientation angle). There could be functions which have many parameters,
> like cutting out some fancy shape.
>
> The whole log filecould be a plain ASCII file! It would besomething like
>
> tab1, openFile, full_path_to_it
>
> tab1, rotateImage, centerx, centery, degrees
>
> tab1, cropImage, x1, y1, x2, y2
>
> tab2, openFile, full_path_to_that
>
> ....
>
> All selection operations, all cloning operations, all matrices used in
> convolutions, all colour adaptation operations, really everything you
> can do with Gimp could be recorded like this. Theoretically you could do
> that by hand by just exactly write down each processing step together
> with its parameters. But some functions have quite a lot of parameters,
> like e.g. cutting parts with an irregular shape, or using
> paintbrush-operations and the like. But a computer does not get tired
> when it is told to do tedious things over and over.
>
>
>
> Debug and improve mode
>
> In such an Interpreter approach, one could also provide a mode, where
> each parameterizingfunction is called again, but then all parameterizing
> steps are preset as they were done when the Gimp session was recorded.
> In the debug and improve mode the user can vary what wasdone before and
> continue, when the varied parameters have been input.
>
> If proper construction schemes are used, the parameterizing functions in
> this special mode are exactly the same ones as they are used in the
> normal mode, the only difference being that instead of only an ok
> button, you also haveadditional buttons "interactive", "skip" and "exit".
>
> * When you click on "interactive", you continue with arbitrary Gimp
> processing steps, returning to the place after each step where you
> clicked "interactive" in the old recorded procedure after each Gimp
> processing step. Of course, during such an operation, all additional
> functions and parameters are recorded as well, such that the whole debug
> and improvesession yieldsa new plain ASCII description of all Gimp steps
> used to achieve the new final result.
>
> * When you click in "skip", the actual processing step is left out and
> Gimp proceeds to the next step from the processing list.
>
> * When you click on “exit”, the whole batch processing in the special
> debug and improve mode is left and Gimp continues with its normal mode
> (i.e. without continuing to process not yet processed recorded logged
> operations).
>
>
>
> Feasibility
>
> Most probably, the main conditions for such an approach are already
> there, the only missing thing would be to create and manage the log entries.
>
> I have never looked at a single line of source code of Gimp, neither
> have Iunderstood the programming paradigmsof Skript-Fu or of the other
> programming tools. Perhaps almost all what I spoke abut is already
> there. Possiblythe basics are also already in Gimp’s native image
> storage format. Are they?
>
>
>
> Further Benefits
>
> 1. It becomes possible to also include the logged processing stepsinto
> the metadata of an exportedresult, even into aJPEG-file! Of course all
> processing steps in other tabs which had no influence aparticular stored
> or exported oneshould be cancelled in that.
>
> 2. Later edit such a sequence in order to improve a result achieved
> earlier, but with a slight modification!
>
> Recorded log fileswould be a great teaching resource: One could publish
> “best practice” sequences: The original picture + all processing steps
> leading to a particular result. The steps could be numbered as above and
> some explanations why the authordid the particular processing steps
> could be published in addition.
>
> For that purpose I proposeto let the Gimp user record oral explanation
> blocks to each processing step. If the authorgoes back some steps, I
> would automatically wind up therespective oral explanations and replace
> them with new ones, if he decides to proceed differently.
>
>
>
> In the firs half of the 1980’s I lead a project, I which we designed an
> electronics unit controlling microscope equipment. It was designed in
> Modula 2 and it used exactly such an architecture. This system was using
> an 8086 microprocessor and firmware completely in ROM. Butit could be
> adapted by plugging additional function boardsinto the rack. The central
> point in the software were lists, containing function names, boundary
> conditions for the parameters and the like plus the entry points of the
> operational functions andthe stack lengths of the parameter-depot for
> them. The real sequence of operational functions could be called almost
> as quickly in sequence as if they had been written in sequence as
> function calls. All parameter checking, all conversions and so on were
> done before.
>
>
> Adalbert
>
>
> 2018-07-08_Standard
>
> Am 28.12.20 um 11:02 schrieb Bill Ross:
>> Thanks Adalbert,
>>
>> I don't want to reverse the transformation, just characterize the
>> forward transformation enough so that I can train neural nets to
>> suggest one I like when shown any photo. You might say I want to use
>> my before/afters to derive a filter for each photo, then train nets
>> with large numbers of
>>
>>      input_pic_i -> filter_i
>>
>> so the net can learn to improvise a new filter for each new pic to my
>> taste.
>>
>> ...
>>
>> Here's an example of how neural nets can pick up my taste in pairing
>> photos:
>>
>>      http://phobrain.com/pr/home/siagal.html
>>
>> Bill
>>
> _______________________________________________
> gimp-developer-list mailing list
> List address:    [hidden email]
> List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
> List archives:   https://mail.gnome.org/archives/gimp-developer-list
>
--
Phobrain.com
_______________________________________________
gimp-developer-list mailing list
List address:    [hidden email]
List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list