Definitive A900/D3X noise/ISO samples

[pakodominguez]

Pako, you had an a900 for the weekend and shot 6 MP JPGs?

Enough quality for this: http://newyork.joonbug.com/photos/oFrMa6zowqM" onclick="window.open(this.href);return false;

[InTheSky]

So many different things are going on under-the-hood with these cameras that it's almost impossible to do a perfect test of this type. The best thing to do if you are a raw shooter is take non-underexposed files from each camera and use several processors to see what you come up with as an optimum result. Not a default result but an optimum result. Make sure you captures scenes in varying amounts and types of light, as well as with varying amounts of shadow and background detail from each camera or you will be short changing yourself on the final result. If you are testing high ISO capabilities then you may want to expose-to-the-right when shooting the tests.

If you are a JPG shooter then do the same thing after tweaking all of the settings in-camera for the specific scene and then fire away to see what you get straight-out-of-the-box.

The bottom line is the longer you work with a camera, the more you get to know how it meters, where your settings need to be, and how your camera makes top images in general. All of the testing in the world does not mean much unless you have at least figured out the cameras' and the converters' fine nuances to make sure you have given the cameras the best chance to give you top results.

In-other-words...firing away at the same aperture and shutter speed at the same target these days sometimes does not mean a lot.

-Sonolta

Hum ... this is not a bad idea ... why not start a kind of game ... , where David you are putting Raw file at ISO 200, 1600 and 6400 like ... After that ... we have to take a Pro Nikon PP user, and a Pro Sony PP user. The rules, cannot add element from another files ... but for the rest, make you best to bring us the best picture. But ... this is dangerous, because for me I'm a lover of Vignetting, this means that if I'm working on the picture i will add some .

Probably some person has there tricks to make the picture looks better. But I'm agree with the fact that in this word ... the JPEG compare is the only solution from camera , asking which camera can bring the best inside PP.

On my side, i have accepted after first few days that my A900 is plenty of noise over ISO 1600, but... I'm dealing with it. And in low light concert ... I'm reducing my size to 1500x1500 maximum and it look good for myspace website . But this is my own utility.

But ... a Raw challenge will be interesting to see , and will save DK time too.

Regards,

Frank

[David Kilpatrick]

But ... a Raw challenge will be interesting to see , and will save DK time too.
Regards,
Frank

Not with this challenge yet, because there are too few converters out which can handle the files. ACR or Lightroom won't tackle D3X yet, nor C1Pro, not Lightzone, not Aperture, not DxO - and so on. Some don't even do the A900 yet and it has been out since September.

When all the converters out there can handle the files from both cameras, a mass 'find the best solution' using a 6400 raw file could be possible. It is easy to set up a single shot with different parts overexposed and underexposed (the shot I set up is already in that category) so a range of exposure is not really needed, just a range of light levels across a group of subjects. Rather more taxing is the size of the files, the Nikon raw files at 49.9MB for example are not all that friendly for hosting and download - though I've got solutions for that.

David

[pakodominguez]

Pako, you had an a900 for the weekend and shot 6 MP JPGs?

Enough quality for this: http://newyork.joonbug.com/photos/oFrMa6zowqM" onclick="window.open(this.href);return false;

Good post, Pako. Just what I figured... Looks like you had a lot of fun with the a900!

-Sonolta

Just a job to do...

[mdcromer]

David,

Nice work, very well done comparison versus what I have seen done elsewhere to date.

I think there are two final steps to do before posting your final comparison.

1) Test using other lenses and see if the Nikons are always about 2/3 stop more sensitive at the same EV, or if it might be an issue with the transmissivity of the glass you tested here or perhaps out of spec aperture values.

2) Do the test at a long shutter speed so you can easily tell if the shutter speeds might be out of spec.

If 1 and 2 come out the same and yet the Nikon is still 2/3 stop more sensitive, then that's that. But we can't make a final judgment on sensitivity until we are certain there is no variation on EV caused by the lens configurations or by inaccuracies in shutter speed.

[David Kilpatrick]

I have now updated the samples to use the Nikon images which are matched in density to the A900, rather than those which were matched in EV setting (aperture/exposure time at ISO 200). I have also three histograms which are of some interest technically, because they show how the A900 can cope with a large dynamic range while the D3X can not (or at least, appears not to based on whatever raw extraction is carried out on both filetypes by RAWDeveloper).

Here is the A900 histogram at ISO 200 (0.8sec, f16, tungsten balance set manually on the camera):

The red and blue channels are clipped at the shadow end, but the green (main luminance channel information) is at level 5. The highlight is held just below 255 on all channels, indicating a perfect raw file for normal post-processing with no highlight clipping. The exposure was set for these tests to avoid highlight clipping.

Here is the D3X histogram at ISO 200 for exactly the same highlight clipping avoidance - 0.6sec was required instead of 0.8. The shadow ends of the channels are all clipped (0,0,0):


You can see clearly how the Sony tungsten balance is almost exact for studio flash modelling lights at full power; the Nikon balance is warm (the separation between the red, green and blue in the main peaks).

Here is a Nikon image taken at the same actual exposure as the Sony, 0.8s at f16:


It shows clipping at both ends, though in the actual image the midtones are definitely the best defined. The tungsten balance is still warm. This was the original example file posted at ISO 200, now replaced by the version which matches the Sony highlight positioning. If the Sony exposure was increased to the point of highlight clipping, it is likely that the shadows would be falling short (all levels above 0). This type of tone curve tends to emphasise noise, while Nikon's tone curve (sacrificing the shadows, pulling the midtones up, and accepting some clipping at the top end) tends to reduce it.

One of the common criticisms of Sony images is that they are 'soft', which certainly doesn't mean lacking sharpness; it seems to be a judgment of their midtone contrast function or slope, combined with retention of shadow detail when it's perhaps not really needed, and retention of highlight detail. A more contrasty range is being squeezed into the 256 steps per channel, like printing a contrasty negative on Grade 1 paper instead of using Grade 2 and getting some nice dense shadows and paper-white highlights. Technically, the Sony approach is 'better' but doesn't always look as good. It results in best of class results for dynamic range but can give some pictures, especially those in darker or dull conditions, a bit of a flat muddy appearance. The D3X strategy is closer to consumer camera tone curve. This is all marginal; where the Sony positions the average tone (value 128) around level 113 for this scene, the Nikon positions it around 135 (for the same nominal exposure). But when the Nikon's highlights are reined in to avoid clipping, its average is darker than the Sony at 110.

If you were to measure ISO by the old film standard of 'first measurable density above fog level' where fog level is 1,1,1 (insignificant bit level) Sony would actually have higher ISOs for the given setting than the Nikon. But if you measure using EI (Exposure Index) based on a digital midtone (128,128,128) Sony is 1/3rd stop lower. If you were to measure the sensitivity at 255,255,255 Nikon would be at least 2/3rds of a stop 'higher'. These histograms and others (RAWDeveloper is a useful program for this purpose) confirm practical experience so far with the cameras.

The solution, as ever, is Sonolta's favourite - 'expose to the right' with the Alpha 900.

David

[David Kilpatrick]

David,

Nice work, very well done comparison versus what I have seen done elsewhere to date.

I think there are two final steps to do before posting your final comparison.

1) Test using other lenses and see if the Nikons are always about 2/3 stop more sensitive at the same EV, or if it might be an issue with the transmissivity of the glass you tested here or perhaps out of spec aperture values.

2) Do the test at a long shutter speed so you can easily tell if the shutter speeds might be out of spec.

If 1 and 2 come out the same and yet the Nikon is still 2/3 stop more sensitive, then that's that. But we can't make a final judgment on sensitivity until we are certain there is no variation on EV caused by the lens configurations or by inaccuracies in shutter speed.

I've taken enough pictures with the various lenses I have (14-24, 50mm f1.4 new AF-S, 45mm PC-E, 24-105mm VR, and 100mm Micro VR) to know that the Nikon genuinely is more sensitive, see my last post. There is more chance of the apertures being out of spec than the shutter speeds (at the sort of times used here, modern electronic shutters are as accurate as modern electronic watches!), and this would be more likely to apply to the old, well used Minolta lenses. As you know, 1/3rd of a stop is a tolerance considered acceptable. But, I've made tests using both a range of apertures and a range of shutter speeds before doing this and I am satisfied that the equipment was all working better than permitted tolerances.

The major difference was in metering - the Nikon matrix and Sony matrix give radically different results under similar conditions, whether with multizone AF or spot AF (the matrix being linked to both). Not only is the Sony sensor likely to give darker results (midtones/highlights) at the same exposure, its metering system is far more likely to underexpose - the Nikon metering is more likely to overexpose. It can lead to as much as two entire stops difference in real exposure.

This is where Sonolta's advice on knowing your camera is vital. I have been shooting with the Nikon today, repeating my visit to Edinburgh's Christmas market and fairgrounds (no soft Santas around) but using the 14-24mm and 1600, 3200, even 6400 on the D3X. Most shots were either chimping as pretty bright and well exposed at normal metered exposure, or too light and needing -1 or -0.7. I have not used the A900 there, but based on my night/dusk shots in Cologne when I first got it, the results would be much darker - maybe better in many cases.

David

[bfitzgerald]

Don't know about the A900 exposure wise. But the A200 is def cautious exposure wise, it's completely different to the 5d.
That's good and bad. Good for low ISO when the 5d could tend to blow out, but bad for high ISO, where it darkens the shadows. The 5d would in most cases give very good shadow details. Hard to know how much is metering, A200 gives faster shutter speeds at high ISO, but I don't know how close it is to the real ISO value, and then of course the sensor, and whatever tonal adjustments go on there.

Ideal spot would be about half way between both cameras. In general I would be tempted to add + exposure for the A200, and in some cases (wide angle contrasty) - for the minolta. Don't know if the A900 responds in a similar fashion...

[mdcromer]

I am suspicious of the Nikon files. The shadows are very good. The files are very good. But I simply don't see any substantial chroma noise at all.

Methinks there is some kind of Chroma NR before the "RAW" file gets written out. . .

[David Kilpatrick]

RAWDeveloper provides tools to linearize the raw conversion and reset the black and white points and gain curve to neutral. There may be some issues with 16 bit, 14 bit, 12 bit on this which I can not comment on but the difference between a Sony A900 raw and a Nikon D3X raw is huge. With all curves and adjustments set to flat/maximum, the Sony provides a pale flat purple-coloured image which then needs massive adjustments including a really strong gamma lift and a black point of 78 (!). The Nikon file, incredibly, has a black point of 0 - what it writes as zero output is a zero value in the Bayer file. It also has about half the gamma lift required of the Sony.

You could see this difference as just the difference between 14-bit and 12-bit.

Here is the Sony, straight line no curve, black level zero:


And here is the Nikon using exactly the same removal of raw process corrections:


I think we can get some clue as to where Sony's files are suffering in raw conversion from this comparison. But, Canon's raw files have an ever softer 'straight' extraction with a more magenta look and black levels at 138, and they stand up well to the steep corrections and curve applied. Running through a lot of raws, Nikon's approach seems to be unique - they are doing most of the tone value correction before the Bayer file is written.

David

[mdcromer]

Don,

It is true that the Sony can apply some digital NR to the RAW files (if NR is on), but there is an option to turn it off (after the debacle with the Alpha 700).

I don't see any Chroma noise in the Nikon files. That is almost certainly due to some kind of NR that (presumably) does not get turned off.

Of course, the Sony on-chip RAW NR was crappy, while the Nikon appears to be good.

[bfitzgerald]

Have to say Sony jpegs don't do a lot for me. Maybe Sonolta is right, they ported their consumer digicam look over. I even think my compact gives the sony a run for it's money for high ISO jpegs! Sure hope we get a firmware update...
I was convinced sony would have done a new firmware for the A900 by now too

[j_p_p]

Well i have seen mention of the "new firmware for the a900" in a couple of foruns, but until now, nothing...

What puzzles me, like DK said, is that nikon can deliver better images using the same (or similar) Sony sensor than sony itself...i mainly shoot RAW, so JPEGs are no big deal /important to me . Specially if you shoot RAW the differences shouldn't be too great (i know there are many other factors, but...)

Before deciding on (re)buying a a900, i am waiting to see exactly what Sony can release FirmwareWise specially in the Metering /Noise aspects.

Hope it doesn't take a year or so

[jcoffin]

Well i have seen mention of the "new firmware for the a900" in a couple of foruns, but until now, nothing...

What puzzles me, like DK said, is that nikon can deliver better images using the same (or similar) Sony sensor than sony itself.
[ ... ]

I have a guess as to why this is so. Sony is using the A->D converters on the sensor chip. Nikon is clearly using separate A->D converters (that's how they get more bits from the same sensor). The amplifiers that modify the effective ISO have to come before the A->D converters, so for Sony they have to be on chip as well. I'd guess Nikon is using off-chip amplifiers -- quite possibly built into the same chip(s) as their A->D converters.

My guess is that the source of the noise isn't the sensor itself, but the amplifiers that are feeding the D->A converters. At low gain, you're primarily seeing the relatively low noise of the sensor -- but as the gain goes up, so does the visibility of the amplifier noise. Nikon's amplifiers have lower noise, so as their gain goes up, the noise doesn't rise nearly as much.

There are two obvious ways to cure this problem: one is to use separate amplifiers like Nikon did -- and probably at a cost similar to Nikon's. The other is to re-design the amplifiers on the sensor chip to improve the SNR. The latter will almost inevitably happen eventually -- but it'll most likely be in a new version of the camera, not something that can be done with a firmware change.

Firmware may still be able to improve the pictures though. We're in a little the same situation as the early history of the Hubble telescope -- we can't change the hardware, but there's still a lot that can be done to maximize the information extracted from what the hardware produces. OTOH, the relatively "random" look of the noise in the A900 pictures indicates that Sony is already doing pretty good processing to remove the noise that they can. Just for example, I'd expect Canon to have much better luck in eliminating the vertical bars in the output from the 5DII -- it's relatively easy to get the computer to recognize and eliminate the pattern.

[douglasf13]

Nikon has long been known to clip blacks, and I think that's what David may be seeing in his conversions.
http://forums.dpreview.com/forums/readf ... pping&qf=m" onclick="window.open(this.href);return false;

As far as the 12 vs. 14 bit issue, I've read some pretty convincing evidence regarding the 14bit mode of the D300 and D3x, and it points to Nikon simply re-sampling the information coming from the 12bit on-sensor ADC's, and that is why the performance slows down in 14bit mode, as opposed to the completely different, true 14bit design of the D3/D700. It is very possible that the lower noise of the D3x's 14bit mode is due to the fact that it goes through a slower processing chain. Nikon themselves claim that the ADC's are on the chip, just like the A900, and I'd put my money on the above being correct. bobn2 over on the Nikon D3x dpreview forum goes into all of this in detail.