An idea for sensor layout, partly 'foveonish'

Discussion of all digital SLR cameras under the Minolta and Konica Minolta brands
Forum rules
No more than three images or three external links allowed in any post or reply. Please trim quotations and do not include images in quotes unless essential.
agorabasta
Viceroy
Posts: 1198
Joined: Wed Oct 22, 2008 7:41 pm

An idea for sensor layout, partly 'foveonish'

Unread post by agorabasta »

Don't know if somebody already patented something like that, but here goes -
layout.jpg
(8.17 KiB) Downloaded 8285 times
The general geometry could be striped with RGBBGR columns. The red filters must be dense, the green channel filters must be yellow-green and block the blue, the blue channel filters must be only slightly bluish passing most of the green and red through. Obviously, the colour resolution may be made much better than a full Foveon has; and the quantum efficiency may be higher than that of a normal CFA since the filters in the green and blue channels may let more light through.

Another property that may be useful is that the left- and right-hand RGB/BGR groups have different directional properties for the green channel that may be used for PDAF. But such PDAF would have horizontal sensitivity only.

A much more complex alternative would be to arrange the RGB elements into 2x2 groups with the blue parts facing towards the group centre. Then cross-type PDAF readout may be possible. At higher densities of the sensor elements it would make sense to read the colour elements by 4B/4G/4R binned groups while reading the two G-G differential signals when needed for PDAF; or simply have one red, one blue, four green reads from a square group.
alphaomega
Viceroy
Posts: 1196
Joined: Mon Aug 18, 2008 11:20 pm

Re: An idea for sensor layout, partly 'foveonish'

Unread post by alphaomega »

I am not an expert on this so I can only offer a general comment from a part time "semi pro" i.e. submitting images to Alamy.
I remember when the Fujifilm S5 Pro came out I was impressed by the dynamic range when using the full 6+6Mp on the sensor. In testing the actual detail reproduction was rated more like 8-9Mp equivalent. In other words softer than an equivalent 12Mp Bayer sensor. Fast forward to the Sigma SD15
Sigma claims it reproduces colour more accurately than conventional sensors. Certainly it delivers images with a different look and feel than we are used to, with subtle tones and shading that makes shots appear less obviously digital and in fact a little more painterly. A polite way of saying they can look a little soft.
Photography Blog test. I believe this is not a unique view of this Foveon sensor's performance. So we have two "unconventional" sensors both performing below a Bayer sensor in terms of detail reproduction. With the latest Sony Bayer sensors producing in excess of 13 EV dynamic range it would appear that beating that and maintaining the resolution capability of the latest Sony APS-C and FF sensors for a given Mp rating is a tall order. Just a layman's opinion.
agorabasta
Viceroy
Posts: 1198
Joined: Wed Oct 22, 2008 7:41 pm

Re: An idea for sensor layout, partly 'foveonish'

Unread post by agorabasta »

Foveons aren't soft, far from that. Bayer array has 1.5 times lower res than Foveon at the same pixelcount. The Foveons are actually too sharp, as Bayer layout provides for some natural antialiasing at demosaicing.

But that's all of very low importance at currently possible pixel densities. If pixel density of 1" sensor used in an RX100 is translated into FF, that would make about 150Mp or about 60Mp for APS. So there is essentially no need for a Bayer layout.
Furthermore, virtually all means of better-quality image reproduction are now natively using the striped layouts, those are all the best screens and printers. So there's actually less res loss going from one striped layout to another than there is going from Bayer to stripes.
agorabasta
Viceroy
Posts: 1198
Joined: Wed Oct 22, 2008 7:41 pm

Re: An idea for sensor layout, partly 'foveonish'

Unread post by agorabasta »

I was hoping for some discussion here :(

I really think that a sensor layout having a native PDAF capability over entire sensor surface at no loss to light-gathering ability is no small thing...
User avatar
Greg Beetham
Tower of Babel
Posts: 6117
Joined: Sun May 27, 2007 3:25 pm
Location: Townsville, Qld. Australia
Contact:

Re: An idea for sensor layout, partly 'foveonish'

Unread post by Greg Beetham »

Well I for one can’t offer you much discussion agorabasta sensor design is something from alchemy as far as I’m concerned…eye of bat, ear of newt…it looks to my suspicious mind with that design of yours that you could be looking at moire issues with those alternate colour columns, a lot less green than a Bayer layout too it seems, you would have the two blue columns a single green column then apparently two red columns then a single green column then two blue columns and so on.
Didn’t I see somewhere that they were experimenting with backlit layers now, or more correctly layered wells with the shortest wavelengths collected at the top (blue) and the receptor for the longest (red) penetrating further more easily at the bottom, but I have no idea how they would separate frequencies with that idea. A more energetic red frequency might end up exciting the blue or green receptor on the way through perhaps. I don’t know how that idea will work anyway, light photons are very slippery critters, (uncertainty principal applies) as far as I know they have a tendency to become a wave when you want measure it as a particle and change into a particle when you are dealing with it as a wave, so the properties of a given wavelength could change after passing through other layers…gigo.
I still don’t know exactly what is going on with the different size pixels either, some appear to be twice the size of others, are those the half green and half exposure (white?) pixels? And what is the exact purpose of the two pixels per transistor? One transistor can only send one value per read…can’t it, or do they now do multiple reads per exposure? Is one pixel less sensitive than the other so when the more sensitive one overflows it is nul and the other one is read instead, maybe the full one overflows into the empty one? or are they both additive for extra dynamic range in low light? Is it for bit depth? Is one for stills and one for video-live view and if so how do they switch over?
Lots of questions and a shortage of answers.
Anyway how do you do phase ranging with cmos light detector pits, don’t you need a special type of detector for focus?
Greg
agorabasta
Viceroy
Posts: 1198
Joined: Wed Oct 22, 2008 7:41 pm

Re: An idea for sensor layout, partly 'foveonish'

Unread post by agorabasta »

Thanks, Greg!
You bring up simple questions, and I've got simple answers - it's no thicker than some basic rocket-science :)
I'll be back in a day or two - can't really reply with one-liners.
alphaomega
Viceroy
Posts: 1196
Joined: Mon Aug 18, 2008 11:20 pm

Re: An idea for sensor layout, partly 'foveonish'

Unread post by alphaomega »

As I have said before I am not an expert on sensor technology, but Sony would appear to think they are moving in that direction http://www.sonyalpharumors.com/sony-3-l ... in-detail/ Would seem as if they want to feature a Foveon sensor at some stage to beat any weaknesses in the Sigma version. Not sure how that differs from Agorabasta's proposal.
agorabasta
Viceroy
Posts: 1198
Joined: Wed Oct 22, 2008 7:41 pm

Re: An idea for sensor layout, partly 'foveonish'

Unread post by agorabasta »

Greg Beetham wrote:it looks to my suspicious mind with that design of yours that you could be looking at moire issues with those alternate colour columns, a lot less green than a Bayer layout too it seems, you would have the two blue columns a single green column then apparently two red columns then a single green column then two blue columns and so on.
That's not a problem since every two adjacent mirrored sensels make a totally symmetric pair and may be read as one given double horizontal density. Then only the columns needed for PDAF would need to have their halves read separately.
If horizontal density is kept single, then there's a very simple demosacing procedure that may be used, like averaging between adjacent pairs to produce an interpolated value for the positions between the sensels. And then all of the sensor surface provides data for PDAF.
Greg Beetham wrote:Didn’t I see somewhere that they were experimenting with backlit layers now, or more correctly layered wells with the shortest wavelengths collected at the top (blue) and the receptor for the longest (red) penetrating further more easily at the bottom, but I have no idea how they would separate frequencies with that idea.
Yes, that's exactly the way a Foveon sensor functions.
I propose adding some colour filters while also adding to the regular Foveon layout some areas that let the traditional colour-filtered capture for the red channel that normally suffers the most in the regular Foveon design.
Greg Beetham wrote:Anyway how do you do phase ranging with cmos light detector pits, don’t you need a special type of detector for focus?
All PDAF attempts working from the main sensor use the 'normal' sensels, but they get special masks making them direction-sensitive by shading some of directions and thus losing some light and then distorting the output.
My proposal adds native directionality to the sensels' sensitivity while increasing their light-capturing ability.

Now if you look at typical layout of a multilayer sensel as is shown in that Sony patent application, you may see that my proposal does not make any drastic changes to the sensel geometry - I simply propose to extend some areas to put dense filters over them and make them active, and then to mirror the layouts in adjacent columns. Below is a figure from that Sony patent.
(Mind also that Sony's application was published later, but it is also discussing some Foveon-filtered hybrid layouts to improve colour separation. When many people independently start thinking along the same lines, it's a clear indication of nearing practical implementation of the idea.)
Attachments
SonyFoveon.jpg
(140.01 KiB) Downloaded 6542 times
User avatar
Greg Beetham
Tower of Babel
Posts: 6117
Joined: Sun May 27, 2007 3:25 pm
Location: Townsville, Qld. Australia
Contact:

Re: An idea for sensor layout, partly 'foveonish'

Unread post by Greg Beetham »

I’m getting old agorabasta, my memory doesn’t work as good as it should so I don’t try to absorb whole bunches of stuff much anymore because I’m sure to forget some of it by the next day. That’s why I don’t really bother with researching and understanding all I can find out about sensors, although I’ve noticed if I get really interested in something then I tend to remember everything about it, but thing is I’m not having a lot of success becoming interested in sensors to that degree, and I’m not sure why that is either.
Probably it’s the sheer size of the task, to get on top of sensor knowledge you would have to know lots of stuff that is proprietary or secret information and not therefore available in any case.
Take the matrix #25 for example; what is it made out of, silicone? Is it opaque or not? Judging by that diagram it would need to be highly transmissive and I thought the wafer was opaque so that’s how much I know. And what does 24-1 do, is it an amplifier or something related to video and if so how does that work? Why are the feeds running over the top of the sensor I thought they moved the wires to the reverse side, and not only that the size of the sensels themselves are tiny in respect to the overall size of the unit. That layout doesn’t seem to have much prospect of moving the sensels closer together.
Why does there appear to be two channels for a colour, take blue 23B-1, it has 26B-1 and 27B-1? The same thing is repeated for the green and red as well.
Basically I don’t know enough about sensor design to offer much useful discussion on the subject; I’ve got lots more questions than answers.
Greg
agorabasta
Viceroy
Posts: 1198
Joined: Wed Oct 22, 2008 7:41 pm

Re: An idea for sensor layout, partly 'foveonish'

Unread post by agorabasta »

Greg Beetham wrote:Probably it’s the sheer size of the task, to get on top of sensor knowledge you would have to know lots of stuff that is proprietary or secret information and not therefore available in any case.
Most of that proprietary stuff is confined to engineering. And that engineering is there to allow the physics of the sensor to function unobstructed. That physics is very simple and straightforward and it's century-old.
Greg Beetham wrote:Take the matrix #25 for example; what is it made out of, silicone? Is it opaque or not? Judging by that diagram it would need to be highly transmissive and I thought the wafer was opaque so that’s how much I know.
The area 25 is bulk non-doped silicon. Silicon is not opaque, it's semitransparent, especially in thin layers. But silicon becomes highly reflective and opaque when there are high concentrations of conductivity carriers (electrons/holes); and those carriers also appear in result of exposure to light. So if you stare at a bulk piece of silicon in bright light, it appears reflective/opaque because of the effect of light on the medium. But the lower the illumination, the more transparent the silicon becomes; it also becomes more transparent if the conductivity carriers are fast removed by electric current, as it happens in photovoltaic elements.
Greg Beetham wrote:Why are the feeds running over the top of the sensor I thought they moved the wires to the reverse side, and not only that the size of the sensels themselves are tiny in respect to the overall size of the unit. That layout doesn’t seem to have much prospect of moving the sensels closer together.
The design shown is not back-illuminated. Larger sensors don't yet require BI due to lower sensel densities. The layout is depicted out of true scale, in reality the wiring and 'service' areas are much smaller than the active light-gathering areas. So they simply don't show the areas without much detail in those sketchy drawings.
Greg Beetham wrote:Why does there appear to be two channels for a colour, take blue 23B-1, it has 26B-1 and 27B-1?
The 23B-1 is the active area of the blue channel, the 26B-1 and 27B-1 just form the arrangement for the blue channel signal pick-up. So 23B-1 holds the physics, the 26B-1 and 27B-1 are just engineering.
User avatar
bfitzgerald
Subsuming Vortex of Brilliance
Posts: 3996
Joined: Fri Sep 26, 2008 10:48 pm

Re: An idea for sensor layout, partly 'foveonish'

Unread post by bfitzgerald »

I had an exchange a while back with Eric Fossum on the DPR forums a while back and he dismissed completely any weight or arguments for non bayer sensors.
I then proceeded to produce a list of "can't be done" from some of the greatest minds ever to have lived including Albert Einstein.

http://en.wikiquote.org/wiki/Incorrect_predictions

I'll be emailing Eric when bayer is rolled out into the technology graveyard. Change is on the way...very clearly
User avatar
bakubo
Tower of Babel
Posts: 5865
Joined: Tue Nov 06, 2007 7:55 am
Location: Japan
Contact:

Re: An idea for sensor layout, partly 'foveonish'

Unread post by bakubo »

I had this on my office wall for years back in the 1980s:

Old Chinese Proverb: Man who says it can't be done should not interrupt man who is doing it.
User avatar
Greg Beetham
Tower of Babel
Posts: 6117
Joined: Sun May 27, 2007 3:25 pm
Location: Townsville, Qld. Australia
Contact:

Re: An idea for sensor layout, partly 'foveonish'

Unread post by Greg Beetham »

That was interesting Barry, there are some whoppers in that list, for example, rockets will never fly above the atmosphere ha ha.
I have reservations about having rows of the same colour, not because I know all that much about sensor design but by comparison to the Bayer layout Agorabasta’s proposed design is a huge departure.
In the Bayer layout the only continuous row of any colour is green but even then it is in the diagonal direction, not vertical or horizontal.
I haven’t counted them but if you look at the dispersal of the pixel colours in the Bayer layout green is represented more than red or blue and I think there was a reason for that that I read somewhere but I can’t remember what it was now.
Actually the dispersal is 50% green 25%red and 25% blue according to this Wiki article http://en.wikipedia.org/wiki/Bayer_filter and it says that layout approximated the receptors in the human eye, whether that was a good design choice in a practical sense I don’t have any idea, I can only assume it was.
As I mentioned before Agorabasta’s layout will have red and blue far more represented than green with double rows or red and blue and only a single row of green in between and I don’t have any idea whether that will be better than a Bayer layout either. Whatever the layout there would have to be a specifically designed demosaicing algorithm compiled for it, like there is for the Bayer layout.
I don’t know enough about demosaicing to know what the result would be with Agorabasta’s design but it could turn out that too demosaic a strong linear layout with low representation of green pixels what began as high resolution could end up as low resolution by the time it was done and be costly in processing time as well, Agorabasta would have more knowledge of whether this was a possibility than I would.
The Bayer design in single layer layout has served us very well so it could even fall under the heading of ‘If it ain’t broke…
Greg
agorabasta
Viceroy
Posts: 1198
Joined: Wed Oct 22, 2008 7:41 pm

Re: An idea for sensor layout, partly 'foveonish'

Unread post by agorabasta »

The Bayer array was necessary to provide max res at min sensel count. Such array produces as many pixels as there are sensels, though the real res is about 1.5 times worse due to demosaicing and LPF smoothing. A Foveon has to have three time more sensels than there are of output pixels, and generallly speaking it also needs an LPF at currently available densities. So a low-density Bayer uses its sensels three times more effectively than a Foveon does.

But now the age of low-density sensors is over, we may easily have sensors far outresolving our lenses, so we don't need Bayer any more. As the sensor resolution is no problem, the more important problems now are quantum efficiency and spectral (colour) resolution.
Greg Beetham wrote:Actually the dispersal is 50% green 25%red and 25% blue according to this Wiki article http://en.wikipedia.org/wiki/Bayer_filter and it says that layout approximated the receptors in the human eye, whether that was a good design choice in a practical sense I don’t have any idea, I can only assume it was.
Sure Bayer has twice the density of green, as that green is actually providing the basic luma resolution, but that also means that too little of red and blue colour info is collected, so Bayer has too much colour noise compared against the luma noise. And it also means that fine coloured detail gets somewhat blurred which may be very noticeable in the reds. The best balance between chroma/luma noise properties would be achieved with the channel weights very different from Bayer.
Greg Beetham wrote:As I mentioned before Agorabasta’s layout will have red and blue far more represented than green with double rows or red and blue and only a single row of green in between and I don’t have any idea whether that will be better than a Bayer layout either. Whatever the layout there would have to be a specifically designed demosaicing algorithm compiled for it, like there is for the Bayer layout.
All not true.
My sketch above in the thread shows exactly same total surface areas for all R-G-B channels. The stripes of equal width go as R-G-B-B-G-R-R-G-B...
But that's not all, as I propose that part of the incident light is registered the same way it is with the Foveon. It means that the green and red mostly pass through the blue surface area into the deeper green and red layers; the blue filter is blueish just as much as it's necessary to enhance the poor colour separation of a native straightforward Foveon, so that blue filter density must be adjusted for best balance between colour separation and total quantum efficiency. Same goes for the green surface areas that let for some red to pass to the deepest red layer.
Furthermore, the widths of the colour channel surface areas may be attuned to any desirable balance of light-gathering ability between the channels - you may have the green channel stripes twice wider than other colours, if need be.
And absolutely no demosaicing is necessary as the group of six sensels as depicted delivers naturally co-sited values for R-G-B. So a sum of each colour pair in that group delivers the channel value while the difference of readings in those colour pairs delivers the phase difference signal for PDAF function.
Also, pairing each half of such group with the adjacent halves of neighbouring groups does also provide for co-sited values in all the channels. So very little of total resolution is lost. (Though some special 'demosaicing' may be used to counter LoCA fringing.)
User avatar
bfitzgerald
Subsuming Vortex of Brilliance
Posts: 3996
Joined: Fri Sep 26, 2008 10:48 pm

Re: An idea for sensor layout, partly 'foveonish'

Unread post by bfitzgerald »

Post Reply

Who is online

Users browsing this forum: No registered users and 13 guests