How Does One Test Their Lenses ala Perez/Thalman??

[Scott Rosenberg]

good day...

i thought i remembered reading on this site a thread or article detailing the procedure one should follow to test the resolution of their lenses. however, search as i might, i am coming up empty. does anyone have a link to this information, or if i'm not remembering correctly, how would one go about testing their lenses? i'd like to run tests similar to the ones kerry thalmann and chris perez did for my lenses.

thanks,
scott

[Ernest Purdum]

The usual method is to obtain a lens "chart". There are several designs, but they all have series of patterns iin sizes which can be related to lines per inch or millimeter. To use it, you fasten several (at least five) copies of the chart onto a flat board, place it at a specified distance from the camera. You then take photographs and carefully examine the result.



There are many potential problems with this approach. Your board has to be very flat and placed accurately perpendicular to the camera. Your film must be perfectly aligned with the groundglass and the groundglass must be perfectly perpendicular to the lens. Over or under exposure will prevent accuracy. The contrast of the film and development has an affect on the results. Partly as a result of the above, and partly because of recognition that resolution is not the only factor affecting perceived "sharpness", other methods of testing have been developed.

One of the popular charrts is the United States Air Force (USAF) pattern. Thbe Air Force once did some uncommon testing by painting a version of their test chart onto the fuselage of a U-2 airplane, usually used for taking photographs of ground targets from extremely high altitiude. In this instance the process was inverted, the patterns on the aircraft being photographed from the ground.

[Scott Rosenberg]

ernest...

thanks for the information. it sounds like the first method of testing would bias the results in favor of lenses intended for close-up and 2-D copy work, as the test procedure closely resembles what they are optimized for. not to disparage those optics, but i wonder if this gives lenses like the fuji a series and the schneider g-claron series an advantage on these types of tests. please don't flame me for that observation... i have and love two fuji A lenses (the 300 and 240).

is there standard way to test lenses using 3d objects at infinity? it sounds like simply taking the same picture with a variety of lenses and then comparing the fine details, such as letters on a distant sign, shingles on a roof, or some other small detail, might give results more in-line with how i actually use my lenses.

[Dan Fromm]

Shooting distant subjects, which you suggested as an alternative to shooting test charts closer than lightyears away, is one of the things I do to help me decide which of my lenses not to use. But it doesn't give resolution numbers that can be compared with other peoples' tests and it is vulnerable to inaccurate shutters, focusing errors, use of different emulsions for different shots, ...

The standard approach with test charts, probably USAF 1951, at 50 focal lengths away is not biased towards performance closeup. 50 focal lengths away isn't that far from infinity. And it has two great advantages. It gives a better sense of just what each lens tested can and can't resolve than shooting at subjects that don't have many, many scales of relatively fine detail. And it allows relatively easy comparisons with other peoples' results.

I started testing macro lenses formally by photographing a microscope stage micrometer that had 100 marks/mm. Your idea. This was somewhat informative, but I got a better sense of what my lenses couldn't do by using a USAF 1951 on glass target. Similarly, if you look at Klaus Schmidt's pictures of microchips -- see www.macrolenses.de -- you'll see that they really aren't as informative as one could want.

As is often the case, quick and dirty lens tests are easy but not always the best. If all you want to know is which of your lenses not to use, quick and dirty will serve. But if you want to know more than that, you should ask the question more formally.

Cheers,

Dan

[Emmanuel BIGLER]

Scott. May I foresee a (small) problem using real objects at infinity in a landscape, i.e. potential blur due to haze or other atmospheric effects. I like to test film & lenses with my favourite urban landscape, I just have to open my windows at home and take a picture of the old roofs of my old city, plus the cathedral's clock ;-) this is more fun than even the French CSP target (the French equivalent of the 1951 USAF target).
BTW when the roofs have been renewed recently it is an excellent test for moiré effects in digital inkjet prints ;-);-) older roofs being less regular are less prone to moiré ;-) More seriously, I have noticed that old rows of tiles when they look super sharp on a print hardly ever exceed the value of 40-50 lp/mm on film, as far as the periodicity of the tile pattern is concerned. Going back home, the advantage of doing tests indoors is that you get rid of potential atmospheric blurring defects ; however indoors you have to test your lenses at a finite distance.
50 times the focal length is considered reasonably far away for general-purpose lenses optimized for infinity.
However I remember reading some posts on photo.net by Kornelius Fleischer who insisted on taking at least 100 times the focal length for proper testing of high performance long tele lenses. But he insists on finding 200 line pairs per millimetre, so ;-);-)
May be a sports hall would be the best place.. except that you have to properly lit the targets ! so testing outdoors with a generous sun solves the question of an homogeneous arrangement of artificial lights, something tricky to properly adjust if you want all your targets to be evenly illuminated.

[Bob Salomon]

Not to mention the variables that lead to less then perfect comparisons:
Lighting, are the emulsions the same, developing identical, atmospheric conditions identical (dust or smoke in the room), loupes for evaluating results the same, eyestrain or lack of it when examining, shot at optimal aperture on each lens, developing time identical, chemistry dilution identical, etc.

There are so many variables that it is almost impossible to repeat the test with the same lens and chart a few months later and obtain the same result.

[Ralph Barker]

As has been suggested by the other responses so far, there are both a variety of methods and a number of variables to consider. I would suggest, however, that perhaps the first thing you need to determine is what you want the tests to accomplish, Scott. If you want to compare your results with those of Kerry Thalmann or Chris Perez, so you can see if your lenses are in line with those they tested, then I think you need to duplicate their testing methods in order for direct comparisons to be valid. In contrast, if you simply want to make comparisons between your lenses under typical shooting conditions, a different methodology would be appropriate.

For the latter style of testing, you might, like Emmanuel, select a convenient and consistent target, and do test images where certain factors (e.g. magnification and film/developer) are kept consistent. The "best" approach depends on what you want the tests to accomplish.

[sanking]

Another consideration in testing of this type is that there may be slight differences in the T-dimension of your sheet film holders , or in how close to the film plane your roll film camera is able to hold the film. Or your groundglass may be off with respect to the film plane. These differences may seem inconseqeuntial but in practice they can easily skew the result of your tests and suggest incorrect differences in the performance of your lenses.

[Scott Rosenberg]

thanks for all the information...

the piece about testing at 50 focal lengths is the one is was missing. i guess i should have been more clear in my original post - i don't want to compare my results to those of kerry or chris, as whatever results i conclude from that comparison would not be statistically valid. there are too many sources of variation that can not be accounted for. furthermore, there is no information as to the gauge reliability / repeatability of the study chris and kerry conducted. comparing my findings to their's would not reveal any useful information... it would be like two people using non-matched scales to measure the same distance - it's a fools errand to be sure.

i was looking for a way to test my lenses, and possibly a friend's lenses as well, controlling as many of the variables as possible ... using the same film holder, the same camera, the same lighting, the same film, equivalent magnifications (now i know it's 50 focal lengths), etc. i would only vary the lenses in question and do my best to keep all other variables constant. this would yield data for my particular set of lenses only, drawing comparisons to anyone else's tests would be invalid, as many of you have noted, these results are driven as much by outside variables as the lens on the camera, variables that chris and kerry controlled using methods of which i have no knowledge.

if i wanted to venture a little experiment, where does on obtain the test charts?

thanks,
scott

[Glenn Kroeger]

Scott:

Edmund Scientific used to (I haven't checked lately) sell a 24x36" chart with AF test targets arranged in all possible orientations and also in each primary color. This is the one I use, dry mounted to a gator board.

[Scott Rosenberg]

if anyone should stumble across this thread in the future, there's a good bit of information here (http://www.jimdoty.com/Tips/Equipment/USAF_Test/usaf_test.html) and here (http://www.normankoren.com/Tutorials/MTF5.html).

[Scott Rosenberg]

thanks, glen! looks like we were typing at the same time.

[Donald Hutton]

Scott

Testing lenses in lpm is a very subjective exercise - what is very useful about Kerry and Chris' tests is that they essentially performed the same tests on many lenses under the same conditions (a very substantial undertaking). Comparing just two lenses of the same focal lengh is a much easier exercise and I frequently have done it - couple of tests targets placed at appropriate focal distance from a secure camera position, polaroid type 55 and a microscope and you will know within 20 minutes which lens resolves more on a high contrast target. But, there is a lot more to lens performance - normal shooting typically does not involve high contrast targets (least not black on white type high contrast); field curvature and a host of other factors make a difference too.

[Bob Salomon]

"equivalent magnifications (now i know it's 50 focal lengths"

Now comes the next problem. A macro lens like the 120 or 180 mm Apo Macro Sironar or the 120mm Apo Macro Sironar Digital is corrected for an image ratio of 1:5 to 2:1. An Apo Sironar N is corrected for 1:10 to infinity. An Apo Sironar S is corrected for 1:10 to infinity. An Apo Ronar is corrected for 1:1. A Macro Sironar is corrected for 1:3 to 3:1.

50 focal lengths will not be at the design criteria for each of these lenses. Now how do you compare them?

Additionally the optimal aperture is different for many of these lenses.

[Emmanuel BIGLER]

If you want the magnification to be 1/50 very precisely you should place the target 50 focal lengths ahead of the front focal point of the lens i.e. approx 51 focal lengths from the middle of the lens. A typical 'off-by-one' possible error ;-) but not so important for the assessment of lp/mm ;-)
And Don is right to point out that you'll need a microscope to examine your negatives.
The naked eye resolves approx 7 lp/mm seen at 250mm, the reference distance for defining magnification of loupes. So with a 10x loupe you'll painfully reach 70pl/mm and probably much less if the loupe is not a very good one. To be comfortable and avoid any loss of quality due to the inspection system you should use a least a 20X microscope i.e. a 2x objective plus a 10X ocular or any combination yielding 20x or more..

[Dan Fromm]

Scott, if all you want is to decide which lens not to use, just shooting around at the apertures you expect to use most of the time and on the emulsions you'll use most of the time, as you initially suggested, will find the bad 'uns. For fine discrimination among the good 'uns, well, more formal testing is necessary.

But there's not much point in making those fine discriminations, is there? Good enough is plenty good enough.

[Scott Rosenberg]

bob... as long as you are comparing lenses within a design, it is fine to test at whatever x focal length you chose. you will not be able to make any statements about the absolute performance of the lens, but you could make a very valid conclusion about the relative performance of lens a vs. lens b... again, assuming they were of the same design. the opposition to testing all lenses at the same distance that i expressed in my second response to this thread addressed that, when i noted that some lenses are optimized for the testing parameters, while others may be a bit 'out-of-their-element' and therefore are at a disadvantage. however, it would be perfectly valid to test two of the same lenses, as the bias would be applied to each sample.

dan... i don't make a habit of printing shots of test targets, so as you suggested, the closer the test is to the real world application of the lens, the more valid it's results will be. resolving power is only one of the many characteristics of a lens that go into making a pleasing image. i agree completely that taking pictures would be a good enough indicator for me!

still though, shooting test charts might be a fun exercise, if only for purely academic reasons.

thanks again to everyone who offered suggestion and inputs!
scott

[Bob Salomon]

"s long as you are comparing lenses within a design,"

The lenses I mentioned are all different designs. They are not variations of the same lens modified for different ratios.

[Ole Tjugen]

The one thing I would like to know about the Thalman/Perez results is: What is "edge"? How far out at the edge of the image circle is that? How can I know hwich format's coverage they've tested?

[Scott Rosenberg]

bob...

i understand and am in complete agreement with you. did you fully read my response?
my statement,

"as long as you are comparing lenses within a design"

was an answer to your question,

"50 focal lengths will not be at the design criteria for each of these lenses. Now how do you compare them?"

i never suggested comparing a lens optimized for 5:1 against one optimized for 1:infinity... in fact, i made an objection to this early on in the thread, suggesting that only samples optimized for similar subject matter could be compared within their group. testing two dissimilar lens designs using similar parameters would be no more valid than testing the sonic quality of speakers designed to go in the yard against a pair or wilson watt puppies... they're not the same animal.

as to your question, the only valid way to compare the performance of lenses optimized for different things would be to design a test that mimics what the lens was designed for and then compare the absolute performance of each sample when tested to gauge the performance for which it was designed.

[Armin Seeholzer]

Hi Scott

I tell you my testing for my lenses, the first part I did with all of my lenses I had at thad time meanwhile I have 3 more and they need also the first part. My first part is a house with very small ceramic stones at about 500 meters distance and the next hill as background for "bokeh" test also. All shoot with Sinar P very heavy tripod 10kg with the Sinar 6x9 rollfilmback the back is fully shifted down so to get not only the sharp middle also the outer part of the lens gets tested disway. I did this test only for the longer then 120mm ones!
For the shorter and wide ones I will go to a distance of 50 meters from it.
My second test will be in my studio where a tape 5 doublesides of a newspaper on the wall especially the fine written ones 1 in the middle the others in the corners and then I shoot it from 2-5 meters depends on the lens lenghs and I'm happy tho find my weakest lens in every part of the game.
This gives me no lines per mm but I learn the strongest lens in every part to know and I find the lemons! Know your equipment I say to it!

[Scott Rosenberg]

armin...

your real-world testing is the only kind that matters to me. thanks for posting your procedure.

scott

[Michael S. Briggs]

Scott, since you aren't really after a quantitative measurement of your lenses, I suggest a simpler way of comparing several lenses that you have and learning about their performance at various apertures.

Find a several story brick wall (building). A brick wall works well because it has detail that is the same across a wide area.

Step back some distance from the wall and point your camera at the wall. Don't worry too much about the exact distance -- just so that you are not making a closeup with a lens that you use for landscapes. Try to get the optical axis as perpendicular to the wall as you can -- levels on the camera and grid lines on the ground glass are very useful for this. You might want to use rise and/or shift to probe farther out in the image circle projected by the lens -- just be sure to write down how much rise & shift you used so that you can locate the optical center on the film. Now take photos with your lenses at several apertures. Place a piece of paper with the exposure number in the scence and be sure to change the number, otherwise you risk getting the films confused.

Develop the films and make prints, or save money by examining most with a loupe. Use a loupe that approximately corresponds to the maximum print size that you plan. I suggest using the maximum print size that you make, or an equivalent loupe, because if you use a stronger loupe you may find unsharpness and worry about when it will never show on your prints.

With films from two lenses side by side on the light table, you can qualitatively compare the results. If two lenses appear equally good on a print, who cares what the lpmm are? Comparing films at various apertures, you will probably find that the center has excellent sharpness at a fairly wide aperture, but that you need to stop down more to get good sharpness in the corners. From this you will learn what apertures are suitable if you use a roll film back, and which work well if you need to use large movements. Also, at which small aperture does sharpness begin to suffer from diffraction.

[Scott Rosenberg]

michael, thanks for the reply. that's a very good way to gauge my lenses qualitatively, which is, as you observed, all i'm looking to do. there's a large brick building in downtown dallas with a mural containing lots of small elements which will work wonderfully with your process.

thanks again,
scott

[Emmanuel BIGLER]

The legend says that Rollei used one of their brick walls at the factory in Braunschweig for testing their optics ;-) so you can go for it ;-)

[Donald Hutton]

Emmanuel

I think they used that wall for styling the ergonomics of their products too...

[Glenn Kroeger]

Scott:

Here is the one I use:


http://www.edmundoptics.com/onlinecatalog/displayproduct.cfm?productID=1665 (http://www.edmundoptics.com/onlinecatalog/displayproduct.cfm?productID=1665)