In 2008, I wrote an article This Lens is Soft, and Other Myths that pointed out that lenses and cameras all have tolerance ranges and sometimes a given lens can be sharp on one camera body, and soft on another. At the time, autofocus micro-adjustment was a fairly new feature on camera bodies and my conclusion was that maybe 80% (yes, I just made up that number, but it seems reasonable in my experience) of the “my copy is soft” complaints could be corrected by careful micro-focus adjustment.
But now that everyone (just about) is using autofocus micro-adjustment to get the best performance out of their lenses, what about that other 20 percent? The times that the lens, adjusted for the best focus possible, is still giving a poor image. Well, some rather good information has come to light in 2009. I find it interesting and I have the distinct impression some of the camera manufacturers would prefer you didn’t know about it (or at least didn’t think about it), so it’s probably worthwhile.
Can we just have the summary first, in case we don’t want to read all this?
Sure. I’m going to conclude that some lenses have intrinsic problems that make them bad even when properly focused (yeah, I know you already know that). I’m also going to conclude that some camera bodies have intrinsic problems that make many lenses bad on that body despite accurate autofocus (yeah, a few of you knew that, too). Then I’ll talk about what kind of things can cause those problems and speculate at length about “bad batches”, “bad cameras” and what it all means for the photographers, gearheads, and pixel-peepers among us (yes, those are three different, if somewhat overlapping, groups).
So what is different since 2008?
Camera sensors, as they always do, have increased in resolution, magnifying any flaws in the lens. What might not be noticeable on an 8 Mpix crop sensor can be frighteningly obvious on a 21 Mpix full frame camera.
Additionally, since 2008 a number of world-class wide and ultra-wide angle lenses have become available: The Zeiss 21, Canon 14 f/2.8, Canon 17 TS-E, and Nikon 14-24 f/2.8 among others. A telephoto lens has a narrow angle of view—maybe 5 or 10 degrees—which means the light rays don’t have to be bent a lot before they get to the sensor. The angle of view of a wide angle lens can be 80 degrees or more so the light rays, especially from the sides and edges have to be bent a great deal before they get to the sensor. A slight difference in the way refracts light from one side to the other is much more noticeable in a wide angle compared to a telephoto lens. With a full-frame sensor any problems are even more apparent.
I’m not trying to say that wide angle lenses are more prone to problems than other lenses. It’s simply that some recent observations were first made with wide angle lenses because they were more obvious with them. Until recently, wide angle lenses tended to be soft in the edges and corners and people just accepted it. Recent lenses are so much better that differences from one side to the other became noticeable (the difference between sharp and soft is more obvious than the difference between soft and softer, apparently).
Finally, people have started to accept that copy-to-copy variation occurs in both bodies and lenses, to understand that manufacturing tolerances are just that: a range of acceptable values, not an exact point. In other words, what is specified as a 1/4 inch diameter screw may be anything between 0.247 inches and 0.253 inches in diameter (1, 2 ). The machines that make them can’t be more accurate than that. Additionally, the screw diameter will vary slightly with temperature, etc. Nothing can be made exact at reasonable cost.
Given that people accepted that there must be some variation, a few lens testers and reviewers started looking for it. We all expected to find some random variation: a group of copies would all be a little different, in different ways. Some early results, though, were surprising in regards to how and where variation might occur.
Some outside information
We have the privilege of working fairly closely with some excellent reviewers and testers, and get to discuss their results with them. But everything I’ll be mentioning here is available for you to read from the original source and I’ll link to their articles. But full disclosure first: Lensrentals furnishes some lenses for testing to both SLRGear and to Lloyd Chambers and have advertised on Diglloyd, SLRGear and TheDigitalPicture We’re rather picky in who we tend to associate with and have, in all three cases, chosen to affiliate with these guys because we respect the work they do. But I think the internet these days is way too full of business relationships that aren’t mentioned by bloggers and writers, so I want to make sure I mention these. Now on to their observations:
Lens variation isn’t always random
I was totally shocked by this. I expected that variation between copies of lenses would be totally random. If someone tested a hundred copies of a lens, I would expect equal numbers would front or back focus. Equal numbers would be softer on the left side and the right side. And most would focus properly and not be soft on either side. But some early evidence is suggesting this isn’t so. SLR gear recently tested 5 copies of the Nikon 50 f/1.4 and found some minor, random variation. When they tested 5 copies of the Canon 50mm f/1.4, all fairly new from our stock however, they found all 5, to varying degrees, were softer on the right side compared to the left side. Five out of five softer on the right isn’t random variation.
While not studied, there has long been a lot of talk that certain lenses had “bad batches”, or that manufacturer’s made “silent upgrades” to certain lenses, improving quality. Certain Canon date codes on a few lenses are unpopular because people have reported lots of trouble with just that date code (no slam on Canon, at least they put a date code on their lenses). Other lenses are widely believed better if you get a later copy, the early ones had problems.
Our own experience supports this. With some hesitation we recently tried a few copies of the Sigma 50-150 f/2.8, tested them, rented them for 6 months, and decided we would stock them in quantity, since they seemed great. We ordered 6 more copies and all 6 were horrible, incredibly soft, and went straight back. The Digital Picture found exactly the same thing. We assume (and assumptions are dangerous) that the second set of six came from a different assembly line run than the first few we bought. We’ve had similar experiences with other lenses we bought in batches. For example, of the initial dozen Sigma 150-500s we bought, almost all failed very early and we stopped carrying them. Lots of people at that time reported similar things, but there’s a lot of folks saying that more recent copies don’t have those problems. The Canon 300 f/4 IS was one of our most trouble-free lenses, but we purchased 6 copies last December, of which 4 developed electrical problems. None of about 30 other copies bought earlier or later had that problem, we only saw it in that one batch of lenses.
There’s no science here, but the phenomenon has been reported so often that I do think where there’s smoke, there’s fire. I think getting 4 of 4 good copies in one shipment and 6 of 6 bad copies of the same lens in another isn’t coincidence. I don’t believe that 4 of 6 in one batch developing electrical problems, while none of nearly 30 other copies did, is a coincidence, either. Call me paranoid if you want to.
Camera body variation also occurs
Take a second and look at the heavy, industrial-strength lens mount on your camera, and the one on your lens. The lens, mounted to the camera, is supposed to be at exact right angles to the sensor. When Lloyd Chambers was testing multiple copies of a very sharp, wide-angle lens on a very high resolution camera he found that the right side was consistently blurred on all but one copy. On a different camera, the left side was consistently blurred (all but two copies). Further testing with different cameras and lenses showed similar findings. The end result is the discovery that on high resolution cameras, with high-quality wide-angle lens (because side to side variation is much more apparent with a wide angle lens) a very small difference in alignment between the lens mount and the sensor will cause this kind of problem.
How small of a variation? One of his sources said as little as 20 microns (0.02mm or 0.0008 inches) is sufficient to cause side to side variation. You can only detect that amount of variation with laboratory grade laser equipment, I’m told. Medical-grade machine parts (used in arthroscopes, etc.) are expected to have tolerances of about 50 microns, (3) and it would seem unlikely that a camera lens mount would be made more than twice as accurately as a medical arthroscope. In other words, with a top quality wide-angle lens on a high resolution sensor, we can perceive a 20 micron difference, but the manufacturer probably can’t make the part more accurately than +/- 50 microns at a reasonable cost.
It’s not just major manufacturer’s lenses
Before the fanboys among you gear up (pun intended again) to use one sentence out of this to claim your brand is better than their brand (I actually read a psychological study on why people feel the need to do that, perhaps another article some day, but the summary is: it says a lot more about the fanboy than it does the product in question) there’s plenty of evidence that there is variation among every brand. Joseph Holmes wrote a nice article about variation in expensive Medium Format lenses. They seem to have at least as significant a degree of sample-to-sample variation as do SLR lenses, despite costing a lot more. Even Zeiss (who’s quality has been clearly excellent, although perhaps somewhat because manually focused lenses are simpler to assemble) apparently will use hand select optics for their new Compact Prime II Cine lenses compared to the standard ZE lenses (source Vincent Laforte ) which I assume means tighter tolerances in these more expensive ($6k) lenses.
Let’s take a logical look at the manufacturing process
Have you ever watched Canon’s very nice video of how a lens is made ? Or read Zeiss’s article on making lenses? Did you notice how about 90% of it is about the glass elements? That’s interesting, of course, and after looking at them I believe the various glass elements in the lens have superb quality control and are as perfect as they could be. I also believe, having read a lot about it, that lens design today is far better than it’s ever been. But do you notice they don’t show you anything about how the lens barrel, or helicoid, or circuit boards are made. Or where they are made? Since, as best I’ve been able to find out, none of the lens manufacturers have metal foundries or machine lathes, I’m left to assume all those metal parts, and probably the circuit boards, are made by subcontracting manufacturers.
We know every manufacturer outsources some of their parts, and the outsource suppliers are rarely known. I would assume that also means suppliers for all those metal parts that hold the lens elements in supposedly perfect positions put in bids and supply the parts to the manufacturer for later assembly. The subcontractor may change, the machine tools may wear out, and one batch of helicoids may be slightly different than another batch, or one set of circuit boards more prone to short out. Perhaps this explains why one batch of a given lens seems to have problems while other batches seem to do fine. It may also explain the much speculated “silent upgrade” event (when people claim that lenses that used to have a high problem rate, no longer do because the manufacturer supposedly did some kind of ‘silent upgrade’). The “silent upgrade” may simply be that a subcontractor of a part was simply replaced by a different subcontractor that made a better part. Recently, for another example, Nikon’s 70-200 f/2.8 VR II lens has been noted to have small metal sparkles (shavings? flakes?) which Nikon states are “air holes remaining in the metal portion of the barrel in the process of component production”. We all know they won’t be there in the next batch through the assembly line. We’ll never know if that’s because the subcontractor is making better barrels or maybe Nikon got a better subcontractor.
But, even if everything is as good as it can be, I believe we now have camera sensors and optical designs that are simply too good for the current standards of mass-production. Let’s assume the source that stated a 0.0008 inch difference in alignment between the lens mount and sensor can make a visible difference in side-to-side sharpness of a 21mm lens, assuming the lens and sensor are capable of very high resolution. (by the way – I know the source, I trust the source, but I’m not at liberty to name the source.) Look at your camera mount and the lens mount, at the little screws that hold it in place. Think about dangling your 70-200 f/2.8 lens on that mount for a few days. Do you really think the mount is placed that accurately on the camera? Or even that it’s machined that accurately? I don’t believe it’s possible.
And inside each lens are up to 16 elements, some of which slide up and down a spiral grove to focus, others to zoom. How far tilted from one side to the other would one element have to be to make a noticeable difference? Or how slightly off true must the internal barrel be to cause some similar effect? I have no clue, but I’d bet it’s not much. So let’s think back on why would one batch of 50 f/1.4 lenses, bought at nearly the same time, all be softer on the right side? I’d guess because a shipment of lens barrels or helicoids or some other internal part was out of spec and all the lenses assembled using that part have an element that’s slightly forward on the right side compared to the left. Why would one group of lenses be likely to suffer electronic failure, when other copies of the same lens rarely do? Probably a batch of bad circuit boards, perhaps a batch of bad solder affecting one run of the assembly line.
In the days of film the plane of film in the back of the camera wasn’t nearly as flat, and was a much more variable distance from the back of the lens than is a digital sensor—so such minor differences probably didn’t matter. Even with low resolutions sensors, the camera couldn’t detect these small variations in lens and lens mount assembly. With full-frame, high resolution sensors and cutting edge optical design, particularly for wide-angle lenses, they have now become apparent.
To summarize, these points seem pretty obvious to me. I’m not certain they’re all completely correct, but I bet they’re all pretty close to reality.
- The mechanical parts that are assembled to form a lens, lens mount, and sensor are going to vary a bit with every lens and every camera.
- This variation will cause every copy of a lens, and every copy of a camera body, to have slightly different characteristics.
- A lens may be fine on one camera and not another. A camera may do fine with one lens and not another.
- Some lenses (and cameras) will be far enough out of spec to just suck, no matter what they are mounted to.
- It seems logical that ‘bad batches’ can occur because a shipment of one or more parts is defective and not caught during routine testing (or the manufacturer decides it’s cheaper to ‘ship and repair’ than to hold a shipment).
- When the manufacturer knows about a “bad batch”, they probably identify the problem and correct it for future lenses, but they aren’t going to announce it unless they absolutely have to – when something is so bad it’s affecting overall sales of that item. Roger’s Rule of Problem Announcements: Once its announced that 5% of lens X has a certain problem, 50% of the members of any online forum will announce their lens has the problem. Whether they own lens X or not.
- Of course future batches aren’t necessarily better, just different. Problem A may have been fixed, but the new supplier of part 32543 may make a bad batch, or the machine tools used to lathe the last set of part 2433 may have become more worn and less accurate.
First, the practical conclusion. Test every new lens you buy, right out of the box, and make sure it’s reasonable. Speaking as someone who has lots of test charts and moderately fancy equipment to do that quickly and efficiently, I’ll just say you don’t need it. Just find a couple of brick walls, fences, etc. that are in a flat plane and take some pictures. A bad problem will be evident very quickly. If the lens is bad, send it back to the store for exchange, not in to the manufacturer for repair (Because certain manufacturers are very likely to say “impact damage” and deny warranty even if you just unpacked it. No, I won’t name names, I don’t have strong enough evidence to say one does this more than another.). Which brings up another point: don’t buy anything from a place that won’t take it back. But don’t confuse the need for autofocus micro-adjustment with a bad lens, it’s not.
It is a bad time to be a pixel-peeper. If you look closely enough on a high resolution full-frame camera, chances are you’ll find some minor flaw with every lens you own. If you go through 13 copies and get one that’s just perfect, remember to never, ever upgrade your camera body, because it probably won’t be perfect on the next one. That’s just how it is: manufacturing processes are not up to the lens design and sensor resolution we have at the moment. Put on your Batman undies and cope with it. Or shoot with a Holga for a while and get over it. And stop looking at 100% images on your monitor: not only will it make you go blind, 50% screen resolution is more resolution than your printer can reproduce anyway. Unless you print billboards designed to be viewed from 5 feet away.
I suspect that the manufacturers are much more aware of this stuff than I am. Of course, their first response is going to be “let’s hope nobody notices”. But I’m sure they’re also looking at cost-effective ways to improve manufacturing tolerances and reduce the issue. By the way – for those of you who are screaming “for $2,000 my zoom lens should be perfect” let me introduce a bit of perspective. A nice, cinema quality (read the best it can possibly be) prime lens runs anywhere from $8,000 to $25,000, while a top quality cinema zoom runs from $25,000 to $70,000. And even at that price a technician will need to spend an hour or so adjusting the lens to the camera that’s using it to get the absolute best results. The best SLR lenses don’t even approach the quality (or quality control) of an entry level cinema lens. Not to mention you probably don’t have a full-time camera tech.
As an aside, I think this is probably going to put the last nail in the Megapixel war’s coffin. The 4/3 companies have already said 12 Mpix is as far as they intend to go. I suspect the full-frame manufacturers are going to call a halt at 30 Mpix or so, just because there’s no sense in it: they’re already out-resolving the quality control of their best lenses.
P.S. – someone wrote and said: “Roger, you’re always trying to predict the future, so what’s your future prediction about this?” okay, here it is: I don’t believe better manufacturing tolerances are likely at reasonable cost. What I think will be done is we’ll take advantage of the computer software programs that are already being used to correct for known lens aberrations. Panasonic does this with the 20mm f/1.7 (4) and Hasselblad does something similar with their new H4D. I could picture the last step on the assembly line being shooting a laser target and programming correction into the lens or bodies EPROM that will compensate for some of the hardware abnormalities that are inevitable. I haven’t been able to verify this, but I’ve been told Leica is doing something like that (although simpler) with S2 system lenses now. Or perhaps lenses will be designed with the ability to adjust mount variation or side-to-side element variation just like we now do front focus and back focus with autofocus microadjustment.