Definitive A900/D3X noise/ISO samples

[David Kilpatrick]

By using the same utility, with all sharpening and NR disabled, I have produced a set of A900 and D3X noise tests which have no variables. The exposures are identical for each ISO, no adjustment has been made, and the point of focus (the pink spotty jelly bean with visible lettering) carefully aligned with 100mm Minolta macro and 105mm Nikon macro. f16 probably introduces a little diffraction softening but also irons out any likely lens difference factors.

Go to:
http://www.pbase.com/davidkilpatrick/image/107193731
view at original size (the crops are about a 5 megapixel area of the image)

and click NEXT to continue through the series. The filenames are clear and EXIF data will confirm.

Please note that the tile the items are resting on is a printed ceramic, and the appearance of what may seem to be sensor lines on the A900 6400 image is just the printing pattern, as checking with the D3X 6400 will show. Remember that NO noise reduction has been applied to any of these samples. RAW Developer produces a very fine crisp noise, not a soft blobby type; it also tends to reveal hot pixels (etc) where other programs mask them. It is a very 'pure' processor. While the A900 comes out OK, the D3X comes out far better, making us once again ask why Sony can not produce exactly the same results from what is basically the same sensor silicon.

David

[InTheSky]

hello David,

The Nikon again look to have a gain of light compare to the Sony.

The white balance is a little bit warmer ... making the blue pattern less apparent.

But you are right the ISO 3200 and 6400 looks better. In fact the ISO3200 of the Sony looks like the ISO6400 of Nikon. But, this is strange the the shadow is different like that between two camera. Like if a DRO is active all the time on the Nikon ... but this will generate more noise ...

I know it is completely impossible, but is there a way with a digital sensor to expose not the same time by region on a sensor ?

Regards,

Frank

[mdcromer]

It is curious.

This is a good test. The shadows are definitely significant better on the Nikon in the ISO 6400 and 3200 shots.

I also notice the Sony has better highlight DR control over the D3x. Comparing the ISO 200 Sony to Nikon, the highlights are much better controlled with the Sony. Of course, there may be more headroom with the files that you didn't use in this particular conversion. Or it could be related to the difference in image brightness as I discuss in the next paragraph.

The Nikon images are somewhere between 1/2 and 2/3 stop brighter. Perhaps it has brighter microlenses or a more transmissive bayer filtration, or both? Or perhaps the Nikon lens is brighter at any given aperture (better antireflective coatings)? The brightness difference could account for the better highlight control of the Sony and the better shadows of the Nikon.

I would like to see some additional studio shots stopped down a lot less, perhaps to f/8. I'm very curious to see which camera resolves better, and using the lens at a sweet spot would help us see that.

[David Kilpatrick]

It would have been possible to adjust the WB on the raw conversions to match. Instead, I let the WB closely follow the result of in-camera JPEGs. In this respect, the Sony was far better - under tungsten light the WB was more neutral. You may also notice that the Nikon does not give a linear exposure. The exposures were made using Aperture priority, carefully adjusted using the +/- over-ride to be identical at ISO 6400 (since this is the setting where a matched exposure is most critical to results). The Sony maintained perfect linearity in doubling the exposure with each ISO step. The Nikon gradually increases the exposure with each slower ISO step, until at 50 the image is quite over exposed. I also made some tests at reduced exposure for the Nikon, but decided to use the more 'correct' Sony result as the benchmark.

DRO etc does not apply to raw files, on Nikon or Sony, so the difference in shadow densities and contrast is entirely down to A to D processed curves. It is possible that RAW Developer applies different curves within the camera profiles used. I can turn off the profiles entirely and get a very flat, almost colourless result from both cameras but I think this would be unacceptable to most people viewing the files. RD's processing is not identical to IDC, CaptureNX2 or ACR and its colour handling can use ICC profiles (including custom ones generated for the camera). It was chosen because a) it actually can process D3X images b) it preserves the 'native' grain noise and does not create any sort of mush.

Of course, the moment you start using sharpening, NR and applying individual exposure, brightness and tone curves to files from the cameras far better results are possible. Since that goes for the Nikon as well as the Sony, you can probably guess what the results are like. Using CS4, I loaded all the images into tabs. Using the tabs, you can switch from one to another instantly, so a comparison is perfectly made. This is quite revealing, like animating the effect. The D3X is almost perfect from 100 to 800.

Sharpness: wider than f8 might be needed. Actually, I think that the rendering on the point of focus in this shot is about as sharp as you can get - so sharp it almost looks soft with a subject like a jelly bean!

David

[mdcromer]

Thanks for the info David.

Looking at the ISO 6400 shots, I am still seeing a significant increase of exposure with the Nikon over the Sony. At least 2/3 stop. This includes highlight areas, where the Sony is much less blown out than the Nikon.

Again, I think this can be accounted by one or more of the following:

1) Lens difference: Perhaps the coatings are better with the Nikon lens, or there are fewer elements, or the apertures are out of calibration for one or the other lens. This would mean that one or the other lens is letting in more or less light during the exposure.

2) Microlens difference: Perhaps the Nikon microlenses are more efficient at gathering light than the Sony.

3) RGB Filter difference:

4) AA filter difference: Perhaps the Sony AA filter loses significantly more light than the Nikon.

5) Shutter speed difference: Perhaps the Sony and/or Nikon is out of spec with shutter speed.

6) RAW convertor tone curve: Perhaps the RAW convertor is applying differing tone curves to each image. The best scenario would be to apply the SAME tone curve to each image, if possible.

7) Some unexplainable difference in the sensors.

Given that both cameras use the same sensor, and should be showing the same QE, I bet the answer is not 6 or 7 and is instead 1-5.

You should be able to test for 1, 5 and 6 without too much difficulty. 2,3 and 4 probably can't be puzzled out without a tremendous effort.

[pakodominguez]

Noise on the A900 is painfull, even at 400 ASA: Sony, we need a nu firmaware soon!
In other hand, The Micro Nikkor result has CA wile the Minolta Macro is CA free.

Regards

[mdcromer]

Pako,

The sony files are underexposed, it is not surprising that the noise is "painful".

Also, a touch of color NR will do wonders. I prefer to leave the luminance noise in place though.

[pakodominguez]

Pako,

The sony files are underexposed, it is not surprising that the noise is "painful".

Also, a touch of color NR will do wonders. I prefer to leave the luminance noise in place though.

Got the A900 last weekend (I borrowed from the Adorama Rentals) and used it for a couple of small gigs and I went to the Brooklyn Botanical Garden (the greenhouse is my favorite place during winter time...)
JPGs @ 400 ASA are really bad, reminds me to the A100 JPGs -since I only needed 6MP for those gigs, noise wasn't that notorious.

Painful, yes...

[mdcromer]

Pako,

1) Why are you shooting jpegs?

2) If you are going to shoot jpegs, have you tried David's modification to the jpeg parameters that drastically reduce the noise?

[KevinBarrett]

Pako, you had an a900 for the weekend and shot 6 MP JPGs?

[pakodominguez]

Pako,

1) Why are you shooting jpegs??

What's wrong about shooting JPGs?
1. I was spoilewd by 7D's JPGS
2. there were small gigs, internet only -no prints and, if printed, in small sizes. No need for more (and they don't pay for more)
3. I'm not getting on an useless debate RAW vs JPG. Period.

2) If you are going to shoot jpegs, have you tried David's modification to the jpeg parameters that drastically reduce the noise?

David didn't "discover" those parameters, but a guy from Rusia (or ex-soviets) that post it on DPR forum. Then: noise is there. You know that. I know that. Sony people knows that and they know they have to improve the JPG engine and other software issues, firmware and IDC.

I know we will get to a better noise files (not to a low-available-light camera) but changing the Zone Matching + sharpness it's just a tric, not a solution.

Regards

[pakodominguez]

Erhmmm...it ain't DK's modification and the settings do not drastically reduce noise...what actually happens is the effective ISO is reduced and on some frames the shadows are being crushed using the aborgasta ZM settings. The smart thing to do at high ISO is expose to the right.

-Sonolta

Don, all this discution happened while your camera was on service. Did you ever tried those settings? You right about exposing to the right, but I'm just curious...

[mdcromer]

Don,

You are correct that the Alpha 900 overrates ISO (although less than the Canon 5D2). We don't know yet whether or not the Nikon overrates ISO.

What I am trying to present is some reasons why the Alpha might need more light than the Nikon.

I think it is important that David figure out which of these is the case if he wants to present the true "Definitive" noise comparison.

For example, if it is simply a difference in the transmissivity of the lenses, inaccuracies in shutter speeds, or inaccuracies in aperture, that's very important.

[David Kilpatrick]

Thanks for the info David.

Looking at the ISO 6400 shots, I am still seeing a significant increase of exposure with the Nikon over the Sony. At least 2/3 stop. This includes highlight areas, where the Sony is much less blown out than the Nikon.

I have noticed this in real life (not test) general shooting and remarked on it already. The Nikon is around 1/3 to 2/3 stops more sensitive at the high ISO end, do not judge on the low ISO exposures because the metering response is not linear. When doing flash tests at a fixed aperture, it is obvious that the Nikon is nearly one full stop more sensitive (=bright) than the Canon 5D MkII at the same nomal ISO rating. Yet technical tests (DxO, dPreview etc) often don't flag up these extreme differences. It is also clear that the Sony ISO is non-linear.

I could do a similar test using a dozen different methods - variable exposure bracketing in 1/3rds then picking the result to exact match density by eye/histo, constant fixed exposure (manual), constant metered exposure with correction initially to match manual (the method used). I used this method because it reflects the actual performance of the camera, when used with its own built in metering system. The one thing I have learned from doing test shoots like this is that no matter what method is used, people will complain that the field is not level etc etc etc.

The Nikon definitely outperforms the Sony, which is what we expected to see. Pako commented on CA on the Micro Nikkor VR 105mm; this image is such a small section from the frame (only 5 megapixels of the 24, and centered) that the outer field is excluded. I believe the Minolta 100mm f2.8 is a better lens, based on working with the test files and shooting several other sets in the process of working out how best to do these. This is subjective.

David

[David Kilpatrick]

Pako,

The sony files are underexposed, it is not surprising that the noise is "painful".

Also, a touch of color NR will do wonders. I prefer to leave the luminance noise in place though.

The Sony files are correctly exposed. The Nikon tends to overexpose, or is more sensitive at the higher marked ISOs. The image is intended to have some dark or murky areas, it was set up to show colours fully lit, a specular reflected highlight which is difficult to render, fully lit colours with a range of light contrast (beans not colour patches on a checker) and a dark area (shadow of the jar on the tile).

See the complete image and its histogram, click it to open (Mac 24 inch screen shot) - the histogram also has figures, it's just about ideal in containing the tonal range:

However, I do have a complete set of Nikon files adjusted in exposure bias to resemble the Sony more closely as shot; these are visible to the left of the thumbnail strip, the Sony (cooler) to the right. I can substitute those tonight, they will be a better match although the EXIF data will no doubt get Nikon folk claiming the test is invalid. I'll do that before creating a main website post showing the results.

David