Definitive A900/D3X noise/ISO samples

[David Kilpatrick]

David,

Nice work, very well done comparison versus what I have seen done elsewhere to date.

I think there are two final steps to do before posting your final comparison.

1) Test using other lenses and see if the Nikons are always about 2/3 stop more sensitive at the same EV, or if it might be an issue with the transmissivity of the glass you tested here or perhaps out of spec aperture values.

2) Do the test at a long shutter speed so you can easily tell if the shutter speeds might be out of spec.

If 1 and 2 come out the same and yet the Nikon is still 2/3 stop more sensitive, then that's that. But we can't make a final judgment on sensitivity until we are certain there is no variation on EV caused by the lens configurations or by inaccuracies in shutter speed.

I've taken enough pictures with the various lenses I have (14-24, 50mm f1.4 new AF-S, 45mm PC-E, 24-105mm VR, and 100mm Micro VR) to know that the Nikon genuinely is more sensitive, see my last post. There is more chance of the apertures being out of spec than the shutter speeds (at the sort of times used here, modern electronic shutters are as accurate as modern electronic watches!), and this would be more likely to apply to the old, well used Minolta lenses. As you know, 1/3rd of a stop is a tolerance considered acceptable. But, I've made tests using both a range of apertures and a range of shutter speeds before doing this and I am satisfied that the equipment was all working better than permitted tolerances.

The major difference was in metering - the Nikon matrix and Sony matrix give radically different results under similar conditions, whether with multizone AF or spot AF (the matrix being linked to both). Not only is the Sony sensor likely to give darker results (midtones/highlights) at the same exposure, its metering system is far more likely to underexpose - the Nikon metering is more likely to overexpose. It can lead to as much as two entire stops difference in real exposure.

This is where Sonolta's advice on knowing your camera is vital. I have been shooting with the Nikon today, repeating my visit to Edinburgh's Christmas market and fairgrounds (no soft Santas around) but using the 14-24mm and 1600, 3200, even 6400 on the D3X. Most shots were either chimping as pretty bright and well exposed at normal metered exposure, or too light and needing -1 or -0.7. I have not used the A900 there, but based on my night/dusk shots in Cologne when I first got it, the results would be much darker - maybe better in many cases.

David

[bfitzgerald]

Don't know about the A900 exposure wise. But the A200 is def cautious exposure wise, it's completely different to the 5d.
That's good and bad. Good for low ISO when the 5d could tend to blow out, but bad for high ISO, where it darkens the shadows. The 5d would in most cases give very good shadow details. Hard to know how much is metering, A200 gives faster shutter speeds at high ISO, but I don't know how close it is to the real ISO value, and then of course the sensor, and whatever tonal adjustments go on there.

Ideal spot would be about half way between both cameras. In general I would be tempted to add + exposure for the A200, and in some cases (wide angle contrasty) - for the minolta. Don't know if the A900 responds in a similar fashion...

[mdcromer]

I am suspicious of the Nikon files. The shadows are very good. The files are very good. But I simply don't see any substantial chroma noise at all.

Methinks there is some kind of Chroma NR before the "RAW" file gets written out. . .

[David Kilpatrick]

RAWDeveloper provides tools to linearize the raw conversion and reset the black and white points and gain curve to neutral. There may be some issues with 16 bit, 14 bit, 12 bit on this which I can not comment on but the difference between a Sony A900 raw and a Nikon D3X raw is huge. With all curves and adjustments set to flat/maximum, the Sony provides a pale flat purple-coloured image which then needs massive adjustments including a really strong gamma lift and a black point of 78 (!). The Nikon file, incredibly, has a black point of 0 - what it writes as zero output is a zero value in the Bayer file. It also has about half the gamma lift required of the Sony.

You could see this difference as just the difference between 14-bit and 12-bit.

Here is the Sony, straight line no curve, black level zero:


And here is the Nikon using exactly the same removal of raw process corrections:


I think we can get some clue as to where Sony's files are suffering in raw conversion from this comparison. But, Canon's raw files have an ever softer 'straight' extraction with a more magenta look and black levels at 138, and they stand up well to the steep corrections and curve applied. Running through a lot of raws, Nikon's approach seems to be unique - they are doing most of the tone value correction before the Bayer file is written.

David

[mdcromer]

Don,

It is true that the Sony can apply some digital NR to the RAW files (if NR is on), but there is an option to turn it off (after the debacle with the Alpha 700).

I don't see any Chroma noise in the Nikon files. That is almost certainly due to some kind of NR that (presumably) does not get turned off.

Of course, the Sony on-chip RAW NR was crappy, while the Nikon appears to be good.

[bfitzgerald]

Have to say Sony jpegs don't do a lot for me. Maybe Sonolta is right, they ported their consumer digicam look over. I even think my compact gives the sony a run for it's money for high ISO jpegs! Sure hope we get a firmware update...
I was convinced sony would have done a new firmware for the A900 by now too

[j_p_p]

Well i have seen mention of the "new firmware for the a900" in a couple of foruns, but until now, nothing...

What puzzles me, like DK said, is that nikon can deliver better images using the same (or similar) Sony sensor than sony itself...i mainly shoot RAW, so JPEGs are no big deal /important to me . Specially if you shoot RAW the differences shouldn't be too great (i know there are many other factors, but...)

Before deciding on (re)buying a a900, i am waiting to see exactly what Sony can release FirmwareWise specially in the Metering /Noise aspects.

Hope it doesn't take a year or so

[jcoffin]

Well i have seen mention of the "new firmware for the a900" in a couple of foruns, but until now, nothing...

What puzzles me, like DK said, is that nikon can deliver better images using the same (or similar) Sony sensor than sony itself.
[ ... ]

I have a guess as to why this is so. Sony is using the A->D converters on the sensor chip. Nikon is clearly using separate A->D converters (that's how they get more bits from the same sensor). The amplifiers that modify the effective ISO have to come before the A->D converters, so for Sony they have to be on chip as well. I'd guess Nikon is using off-chip amplifiers -- quite possibly built into the same chip(s) as their A->D converters.

My guess is that the source of the noise isn't the sensor itself, but the amplifiers that are feeding the D->A converters. At low gain, you're primarily seeing the relatively low noise of the sensor -- but as the gain goes up, so does the visibility of the amplifier noise. Nikon's amplifiers have lower noise, so as their gain goes up, the noise doesn't rise nearly as much.

There are two obvious ways to cure this problem: one is to use separate amplifiers like Nikon did -- and probably at a cost similar to Nikon's. The other is to re-design the amplifiers on the sensor chip to improve the SNR. The latter will almost inevitably happen eventually -- but it'll most likely be in a new version of the camera, not something that can be done with a firmware change.

Firmware may still be able to improve the pictures though. We're in a little the same situation as the early history of the Hubble telescope -- we can't change the hardware, but there's still a lot that can be done to maximize the information extracted from what the hardware produces. OTOH, the relatively "random" look of the noise in the A900 pictures indicates that Sony is already doing pretty good processing to remove the noise that they can. Just for example, I'd expect Canon to have much better luck in eliminating the vertical bars in the output from the 5DII -- it's relatively easy to get the computer to recognize and eliminate the pattern.

[douglasf13]

Nikon has long been known to clip blacks, and I think that's what David may be seeing in his conversions.
http://forums.dpreview.com/forums/readf ... pping&qf=m" onclick="window.open(this.href);return false;

As far as the 12 vs. 14 bit issue, I've read some pretty convincing evidence regarding the 14bit mode of the D300 and D3x, and it points to Nikon simply re-sampling the information coming from the 12bit on-sensor ADC's, and that is why the performance slows down in 14bit mode, as opposed to the completely different, true 14bit design of the D3/D700. It is very possible that the lower noise of the D3x's 14bit mode is due to the fact that it goes through a slower processing chain. Nikon themselves claim that the ADC's are on the chip, just like the A900, and I'd put my money on the above being correct. bobn2 over on the Nikon D3x dpreview forum goes into all of this in detail.

[jcoffin]

"The D3X’s image sensor incorporates 14-bit A/D converters on the chip, delivering fast conversion, high signal-to-noise ratio and low power consumption.
-Sonolta

That certainly seems to indicate that my guess was wrong -- I probably should have read through their marketing material more thoroughly before even forming a guess.

[mdcromer]

David,

I just read your article on the main page.

I did not realize that you shot at 14 bit with the Nikon! This changes everything.

Two things that I have read elsewhere:

1) the 14 bit mode shoots at 1.8 fps instead of 5 (which is probably just fine in studio, but not necessarily for event photography which is where you are more likely to be using high ISO!)

and

2) The 14 bit mode uses an off-chip A/D converter. It does not use the Sony on-chip A/D convertor or (presumably) the Sony on-chip analog amp.

So,

The bottom line is probably that the Sony analog amplifier is not nearly as good as the Nikon off-chip analog amplifier. Not really any surprise there.

Can you do the same test with 12 bit mode and see if the results are any different? If my points above are correct, it seems very likely that the results will be much closer to the Sony.

[douglasf13]

mdcromer, the A/D converter is on the chip, just like the D300. Nikon says so in their literature. That's why the re-sample, "virtual" 14 bit method I mentioned above seems rather credible. There doesn't appear to be an analog out off the chip for the use of off board ADCs.

[mdcromer]

Well perhaps the "resample 4x" times could account for the greatly improved shadow detail on the high ISO. Neat trick if that's what Nikon is doing! It would also explain the drastic slowness of the 14 bit mode.

David, let's see the same test with 12 bits instead of 14. That should shed a lot of light on things. . .

[David Kilpatrick]

I can't repeat the test itself at 12-bit, but I can do a test at 14-bit, 12-bit, 12-bit lossless compressed and 12-bit lossy compessed; and Sony 12-bit uncompressed and cRAW. Maybe just at 6400, to see whether the bit depth and compression has any real effect.

I was not going to mention that the Nikon setting black level to zero amounts to clipping the blacks; I've been shot down too often for mentioning this in the past. Nikon both clips and crushes the blacks and deepest shadow tones. Canon do not clip, but crush heavily. Sony does neither. They seem content to let everything through... along with the noise.

David

[Vidgamer]

....
I was not going to mention that the Nikon setting black level to zero amounts to clipping the blacks; I've been shot down too often for mentioning this in the past. Nikon both clips and crushes the blacks and deepest shadow tones. Canon do not clip, but crush heavily. Sony does neither. They seem content to let everything through... along with the noise.

David

You can easily enough crush your own blacks in your RAW converter or paint program.

What I gained out of this thread is looking at the DxoMark site, the A100 still holds its own compared to the newer cameras.

I remember people talking about the A100 noise and... well, noise is still there. Really, though, there was noise when we used film, right? I mean, as long as I stay at ISO 800 or better, it looks good to me. I have more capability than I ever had, and using different RAW converters, results can vary greatly. I try to use 100 or 200 most of the time, but I don't obsess over 400 or 800 when needed. When I have a problem is with 1600... if you really need that kind of speed, ok, then the A100 looks like doggie-doo. The newer cameras are definitely better there, but I'm surprised at the Dxo results, even so.

I'm not getting an A900 or Nikon-whatever either way.

BTW, and maybe this was on another thread, but someone mentioned Ken Rockwell ... he says not to take his site too seriously. Read his "about" page.