A different approach to scanned image histograms

[barnacle]

Most scanning software and most picture editing software - all that I have seen, in fact - uses a simple histogram to give some hint as to relatively how many samples are at any particular grey-scale level. I don't find this particularly useful, as often a picture may have a larger range than is immediately visible, but with a relatively few samples at either extreme. This can lead to adjustments to the picture which remove information at the extremes, since the operator is relying on the accuracy of the display system to show him the image.

Outside a professional image editing suite, with calibrated displays and room lighting, I suspect that accuracy is rarely to be had. Angle of view on many LCD screens is a particular matter for concern, with a movement of only a few degrees vertically changing the apparent contrast and brightness of the screen (IPS screens are much better in this respect).

I spent over thirty years in professional broadcast engineering (the BBC) and there is a tool used regularly there to accurately judge exposure (even in a regularly calibrated environment) - a waveform monitor. In such a display, the signal is scanned across its width and the vertical position of the trace shows the luminance of the signal; if large areas are at the same brightness then trace increases in brightness.

I've been playing with a proof-of-concept for such a display using Python: This is very much an unfinished work in progress. Here are some examples (each picture being worth a thousand words, this saves eight thousand words of explanation):

147294

147295

147296

147297

Any interest in this kind of display? It's currently living on a Linux Mint system, running Python 3.4 with the GTK+ toolkit and with the UI built in Glade. I don't think there is any difficulty porting to other OSes but that's someone else's problem; I don't have access. If anyone else wants to play, you're welcome; I'll post the code.

Neil

[Peter De Smidt]

Good stuff, Neil. I take it the advantage of this approach is that there are many more samples than with a traditional scanner histogram? Is that correct?

[Nodda Duma]

Hi Neil,

That's very interesting, thank you for posting. It looks like you scan every line, correct?

I agree that histograms *as presented in scan software* can be misleading. Properly used, histograms can provide valuable information. They form the input data for almost all automatic gain control algorithms used in digital imaging systems today.

For work I pull images into Matlab and do a histogram there so I can look at actual numbers -- including min & max values, peaks etc.

[fishbulb]

Most scanning software and most picture editing software - all that I have seen, in fact - uses a simple histogram to give some hint as to relatively how many samples are at any particular grey-scale level. I don't find this particularly useful, as often a picture may have a larger range than is immediately visible, but with a relatively few samples at either extreme. This can lead to adjustments to the picture which remove information at the extremes, since the operator is relying on the accuracy of the display system to show him the image.

Yep, this is a problem with the histograms in DPL. When I scan, I leave the histogram a bit lower contrast, moving the black point and white point a bit outside the histogram that DPL gives me. In general, this works quite well to preserve detail in the shadows and highlights.


Any interest in this kind of display? It's currently living on a Linux Mint system, running Python 3.4 with the GTK+ toolkit and with the UI built in Glade. I don't think there is any difficulty porting to other OSes but that's someone else's problem; I don't have access. If anyone else wants to play, you're welcome; I'll post the code.

If you want to make some money, build it as a Photoshop and Photoshop Lightroom plugin, then sell it by publicizing it on the big photography blogs (petapixel, photographylife, etc).

There are tons of photography nerds who would go for this level of detail, not just for scanning but for digital imaging. But with Linux-only, you are really limiting your addressable market to a microcosm of the larger photo community.

[barnacle]

Thanks all.

This is still, as I said, in need of further development, but I doubt I'll release it as commercial software - apart from anything else, it would probably not comply with any future employer's conditions. It might find its way into the Gimp, perhaps - I have been an open source user for many years and only rarely get to give anything back. Also - I'm really unhappy with the way that Windows has gone in recent years and also many high end windows programs, as the transition to a software-for-rent model. I don't see why I should support such systems. </rant>

I don't know if there are more samples but there is more precision, even though it's a scaled image that I use to measure. In practice, every vertical line of the image is used to build a histogram, and the same vertical line on the display shows that histogram, but with the brightness of the point showing the height of the histogram. Another way to look at it is that a single faint green line is drawn across the display for each line in the image with the vertical height of the line indicating the brightness. As more and more lines are drawn, if they happen to meet an existing line, they make themselves brighter.

What I wanted it for was to adjust the black and peak levels when scanning and in particular to see what the scanner was doing when it expanded the range - because its internal histogram indicated that there were huge gaps in the signal (i.e. quantisation).

Looking at this image again

147313

it's clear that it's a pretty much full range scan; the whites are very close to peak white but don't crush (which would be indicated by a line at peak white) and while there is detail well into the blacks, it's apparent that some of the blacks may be crushed in the scan on the left hand side where they are all at the same level a couple of bit values above zero.

Neil

[Struan Gray]

I have written my own similar tools to analyse spatially-resolved spectroscopy data. I loved the technique. Other people - mostly nerdy research types, so not afraid of interpreting data - loved the analysis when used by me for a case study as part of a talk, but did not want to learn how to interpret the displays themselves. You may run into a similar resistance :-)

If you can refine the code to the state where it could work reliably as a GIMP plug-in, that would be an excellent way to get it to people who are likely to use it.

One finesse you might want to consider is a toggle that allows different sorts of normalisation.
First setting plots a fully-white point for every data value (no grey scale values). The allows rapid identification of extremal points and data ranges.
The second normalises each histogram separately. I think this is what your code is doing here. It gives the best overall view of the data values and their spread.
The third does a global normalisation. This allows a comparison of the intensity of the histogram peaks for different scan lines.

I also overlaid (well, underlaid) lines or coloured bands representing max/min, average, or average+-a given number of SDs.

Finally, as a photographer, I would like the option here to switch from producing histograms of vertical lines to histograms of horizontal ones. That would make it easier to avoid clipping/compression in things like seascape skies.

[barnacle]

Yeah, it's one of those things... any broadcast engineer will recognise it instantly. Your point about different gain normalisations is well made; picking a suitable gain is easy with a twist of a knob on the hardware tool.

Here's a few done with 100% - any active point gets full brightness green.

147496 147497 147498

Neil

[Struan Gray]

Here's me thinking I'd invented something :-)

You need a snappy name. 'Waveform monitor' is too analogue. 'Waterfall plot' is taken. My wife christened my diagrams 'dreich plots' because they looked like curtains of rain sweeping through a typical Sunday afternoon in Cumbernauld. I adopted 'drizzle plots' to avoid confusing the Swedes with yet another cultural shock.

I like the way the cathedral photos look like their waveform plots.

[Taija71A]

Hello Neil!

Without going into 'too much' detail... What are your thoughts on existing Software Programs --
Such as FastRawViewer and RawDigger? Thank-you!

Best regards, -Tim.

[barnacle]

Struan, how about a 'WFM plot'?

Tim, I can't comment on those programs; I've never used either and I rarely do much with raw formats. A quick look at their websites though indicates that while they have *lots* of histograms, they don't tell you much more than the simple histograms in Gimp.

As an aside, an excellent how-to on the use of waveform monitors in filming: https://www.youtube.com/watch?v=x-8o_TvyNjc

Neil

[Taija71A]

No problem Neil. Understood in full.
Thank-you, for your update to the Thread.

-Tim.

[Struan Gray]

'Take a look at my excellent WFM plot' doesn't exactly trip of the tongue. 'Waveform plot' is easier to say. If they can stick with antiquated tube nomenclature for imaging chip sizes, the industry can probably cope with that.

I've been trying to find ways of efficiently preserving subtle colour in scans and direct digital capture. Scanner software and most raw developers (and built-in jpeg engines) push saturation and contrast too hard for my liking. A tool like this would make it easier to assess the best parameters for scanning or capture. I know it's available in hardware for some add-on monitors for DSLR filming, but not for most scanning packages.

Are you using plain Python, or also one of the add-on image processing or numerical packages?

[barnacle]

Plain python 3.4, GTK+ toolkit. Might actually be easier in 2.7, as there is more functionality in the GTK port at the moment.

http://www.dnbprojects.co.uk/python/image.py
http://www.dnbprojects.co.uk/python/image2.glade

Please bear in mind that this is very much unfinished, just a proof of concept, and very much remains to do to tidy things up. Like, e.g., not having everything happen in the response from the file select menu...

Neil

[Struan Gray]

Thanks Neil. I do various tinkering things with an old license for an image processing package (IDL), but it's not worth the substantial cost of an upgrade. Looking for options, ideally open source, as all the commercial vendors seems to be trying to lock their users into rent-rack agreements.

*All* my programs have dangling live wires. All the users (me) just know not to go licking them :-)

[BetterSense]

Thank you for the interesting post. I did not know such a thing existed.

Also, props for running Linux. I agree 100% that we all need to consider alternatives to Windows as their recent policies, not to mention quality of the product itself, has me very turned off. With Steam offering many videogames on Linux, I hope more vendors consider supporting Linux workstations.

[barnacle]

"wuffum"? :p

Thing is, is it really a waveform? It is on a linear TV signal, and it's generated as if it were the same, but... how about 'luma plot'?

Neil

[Struan Gray]

Luma won't really work: the video whippersnappers have no respect for tradition, and work with separate red, green and blue wuffums for each channel - on a colour display the individual wuffums can be overlaid without too much confusion, and it is possible to show only one of them.

I called them 'line scans' or, more accurately, 'line scan histograms'. In microscopy (my field was STM) you often rotate to an arbitrary angle to align with surface features, so it was always explicit that you were extracting histograms perpendicular to a given direction. 'LSH' rolls off the tongue a bit easer than 'WFM', but 'wuffum' is pure genius, and is growing on me :-)

[barnacle]

An alternative (better?) display brightness function - where the brightness is summed at line 64 and following, replace with


# we increment the brightness by some part of the difference between its current level and peak
b = 256 - brightness[brightptr]
brightness[brightptr] += int(b/4)

This restricts the brightness range but emphasises the less bright parts; divisors of 2, 3, and 4 work well. This image has a divisor of 4.

147956

Neil

[Struan Gray]

Log works pretty well for bringing up the floor, if you can live with the calculation time/memory.

[stawastawa]

a glance at this wuffum and i might be concerned I had clipped the window.

The examples in the video tutorial seem to be easy to read and correlate to the images - what brightness scaling are they doing?

the 100% brightness examples posted earlier were to harsh. a divisor of 4 seems not strong enough. IMO

very interesting data visualizer!




An alternative (better?) display brightness function - where the brightness is summed at line 64 and following, replace with


# we increment the brightness by some part of the difference between its current level and peak
b = 256 - brightness[brightptr]
brightness[brightptr] += int(b/4)

This restricts the brightness range but emphasises the less bright parts; divisors of 2, 3, and 4 work well. This image has a divisor of 4.

147956

Neil

[barnacle]

147999

Here is is with a scale of 3. Struan, I'm interested in your log function if you'd care to detail it. I think what I have here is an inverse exponential: what I was after was making the dimmest lines bright enough to see but successive lines provide less input. With a scale of 2, the outputs available are 128, 196, 228, 244, 252, 254, and 255 (I think) so anywhere that gets more than seven or eight hits is full brightness. That's extended for larger divisors, and the dimmest lines are proportionally dimmer.

Nicholas, the brightest part of this image, apart from the white surround, is on the pillars to the right of the door.

Neil

[Struan Gray]

Just take log(1+pixel_value) and rescale so the maximum is at 255.

This compresses the highlights and expands the shadows, but leaves the spatial structure comprehensible. It doesn't do much that is useful for normal pictorial images, but is used a lot for data (particularly Fourier transforms, where one prominent peak can hide everything else). I found it helpful for quick assessment of the kinds of distributed histograms we're discussing here. It's a toggle control on most image processing packages, so you can quickly switch back and forth.

I used brute force, changing values to double precision, and then rescaling back to integer/byte. Since you know your wuffums are 8-bit and pre-scaled, you could probably do the log function as a look up table.

[barnacle]

Thanks Struan.

This version just increments the histogram linearly in the collection phase, then applies the log function in the display phase.

Using a straight log(10) still leaves the single trace a bit gloomy:

148035

Adding an offset and scaling the log a little less lifts the black too much:

148036

So gating out the zero samples seems an improvement:

148037


if brightness[brightptr + x] > 0:
green = 64 + int(75 * math.log(brightness[brightptr + x] + 1, 10))


Neil

[Struan Gray]

FWIW, I used to use the log scaling a lot more often when applying a universal normalisation. When normalising each histogram separately, there's less need to bring up the shadows, and as you've shown, when overdone it ends up making the overall shape of the data harder to interpret.

You have a nice tool there. I look forward to downloading a copy :-)

[barnacle]

Thanks. I'm investigating how to write plugins for Gimp; I'll post again once there's something to see.

Neil