Recent advancement in image resolution enhancement

[Patrick Gauthier]

I read this article a little while ago and thought some may be interested in advancements in digital image enhancement technology that (fingers crossed) may become available/applicable for photographers down the road. The context of their objectives is somewhat dissimilar to the typical photographers, yet it seems there is much overlap in the overall goal: better post-processing methods for enhancing image resolution.

https://arxiv.org/abs/1612.07919

[Leigh]

It all depends on what you mean by "enhancing image resolution".
You can increase the pixel density so larger prints look "normal" rather than pixelated.

But there's no valid way to add fine detail that's not in the original digital image.

- Leigh

[Patrick Gauthier]

It all depends on what you mean by "enhancing image resolution".
You can increase the pixel density so larger prints look "normal" rather than pixelated.

But there's no valid way to add fine detail that's not in the original digital image.

- Leigh

I would say it all depends on what you mean by "valid" The rest is rather objectively defined in the article.

[Leigh]

I would say it all depends on what you mean by "valid" The rest is rather objectively defined in the article.

Valid detail is detail that existed in the original subject.

One example might be the lug nuts on the wheels of a car.

Take an image of that car from a mile away using a wide angle lens.

If you can enhance that image to the point of seeing the lug nuts and determining their angle of rotation, then you have recovered "valid detail". If not, you're just fantasizing.

- Leigh

[mmerig]

It all depends on what you mean by "enhancing image resolution".
You can increase the pixel density so larger prints look "normal" rather than pixelated.

But there's no valid way to add fine detail that's not in the original digital image.

- Leigh

+1

The article is interesting, but the comparison of the grasshopper images says it all. Basically, their test is to down-res an image and see if their algorithm can up-res it to match the original (the ground-truth image). As they explain, their method tends to add things that were not there in the original, and loses things that were (as do other methods). To me, that is not a valid (true) outcome, even though their method does work better than the others.

[Patrick Gauthier]

+1

The article is interesting, but the comparison of the grasshopper images says it all. Basically, their test is to down-res an image and see if their algorithm can up-res it to match the original (the ground-truth image). As they explain, their method tends to add things that were not there in the original, and loses things that were (as do other methods). To me, that is not a valid (true) outcome, even though their method does work better than the others.

The results are comical, but when you see the file they had to work with its pretty impressive. Figure 7 shoes some issues with the texturing smooth features adjacent to patterned features that are problematic.

Still, it's pretty amazing how they can generate novel information that renders an otherwise blur startlingly similar to the ground-truth image. I'm very curious what it will do to an already somewhat acceptable image (for example a botched focus 4x5 negative that could still be acceptable for an 8x10 print) in terms of increasing potential/acceptable print size. My thinking is that while the image examples they provide in the paper as a whole leave much to be desired, perhaps for very large images (i.e. LF scans) the egregious artifacts in their final product would be less noticeable.

I'm going to try with a scan of an out of focus negative to see what happens. Will post results here.

[mmerig]

Your test would be more practical for most photographers compared to theirs. Who would down-sample an image and then try to reconstruct it using a complex algorithm, instead of just using the original, better image? The bicubic smoothing for down-sampling may give a different set of errors than a low-resolution image obtained in practice (low pixel count, blurry from motion, out of focus, etc.) But bicubic smoothing provided a consistent, well-understood starting point for their tests of course.

Although their method can give better-looking images, it fails from a forensic standpoint without ancillary information (e.g., Figure 7). It's basically pearls on a pig. A lot of the utility of their method depends on whether the objective is a more truthful image or something that just looks better somehow.

Regarding your test with an out-of-focus image, it would be better to also have a well-focused image, that is the same in every other way, to compare the improved out-of focus one to. Also, a scan of an analog enlargement of a section of the in-focus photo rather than an an initial zoomed-in scan would minimize the degradation during digital conversion of the true image. The same could be done for the out-of focus image. The file sizes and computation time would be smaller too.

Despite my negativity about the algorithm, I look forward to your comparisons, and thanks for letting us know about their work and what you plan on doing.

[Jac@stafford.net]

I read this article a little while ago and thought some may be interested in advancements in digital image enhancement technology that (fingers crossed) may become available/applicable for photographers down the road. The context of their objectives is somewhat dissimilar to the typical photographers, yet it seems there is much overlap in the overall goal: better post-processing methods for enhancing image resolution.

https://arxiv.org/abs/1612.07919

Cruise more academic literature and look for common proposals. The abstract posted is rather empty, but most abstracts are click-bait for scholars and hackers. In the end they are selling something, you know.

[Patrick Gauthier]

You can try it out for yourself! However, their code downsizes your image to make low-res examples (i.e., to see if their method can produce something similar to the original (ground truth). If you're savvy in python maybe you can try on full-res images.

https://github.com/shakeh3r/Enhancenet

[consummate_fritterer]

No offense intended to anyone. I know some software developers hype their wares far too much.

I once worked with a guy who believed a PS plug-in developer claiming that a six pixel image of a small bird could be made to look reasonably like that bird via use of their plug-in. Apparently, many people bought into it.

After taking a closer look, I have to admit their results are better than I expected. Still, what is not there cannot be put back with any automated process.