yashsingh1/xdaforums huggingface dataset:
[Google Pixel 8 Pro] CHH2:
Oh you very much can apply DSLR rules here because it's the all the same physics. The whole point of the quad bayer pattern in the Pixels is to fake larger photo sites. Four small photo sites are physically and electronically grouped together to act like one bigger photo site. The 50mp sensor is literally physically made into a 12.5mp bayer pattern sensor. You can keep trying to tell yourself otherwise but the specs and tech sheets are all right out there in the open.

You keep coming up with new excuses but none of them fly. I'm still waiting on actual proof of your points that support that is what Google is doing. (Side note here, every working photographer I've ran this by says the same thing. Once you put that quad bayer and processing pipeline in place, the sensor is reduced to quarter resolution.) Meanwhile, I've pointed to a reputable source that very clearly states what Google is doing which is taking several photos and interpolating them into a "full res" file.

And yes, in night sight mode, they take several photos and combine to increase the SNR but this is not night sight. Even in broad daylight where there's all the light and more than anyone could possibly need or want, it still is taking several frames and it certainly isn't to up the SNR.

The fact is that they are taking multiple frames to fake the full 50.1mp of resolution. Fun fact, the UW and T sensors which we know for a fact are 48mp sensors are also magically creating 50.1mp images with the same pixel count width and height as the W sensor. That alone should be proof that Google isn't giving us real full resolution files but is making them up in post through a bit of trickery.

Without providing any sort of source to explain your theory on how Google is making these magic full resolution files out of sensors that aren't even that resolution, I can only guess you have no actual idea and are just making random uneducated guesses. Come back to me with a source that can effectively counter the source we already have and the understanding of every mission critical working photographer I know or there's not really any point in continuing this conversation.
[Google Pixel 8 Pro] MrBelter:
Just for clarification a quad Bayer sensor is not the same as a Bayer filter, even the Canon 20D from 2004 uses a Bayer filter on its 8.2mp sensor, it can even be removed from the sensor (if you are brave enough).
[Google Pixel 8 Pro] Mrowkarz123:
All color sensors use Bayer filter, that's how we get color information in one image. With Quad Bayer, each color filter covers a group of 4 fotocells.
[Google Pixel 8 Pro] MrBelter:
Maybe i worded it wrong, quad Bayer is used when pixel binning is to be employed, you couldn't pixel bin with a standard Bayer filter so you cannot really compare the 2.

Also given the filter can be removed with a bit of patience it proves the pixel count of the sensor has nothing to do with the filter.
[Google Pixel 8 Pro] Mrowkarz123:
There are two types of pixel binning, hardware and software. Hardware binning combines charges on a sensor, before it goes through the ADC. This type of binning is only possible with CCD sensor technology because they allow for different readout patterns. It was never possible with CMOS sensors, and I don't think it's different with the ones we have in smartphones.

Second type is a software when the pixels are combined by a software, after the ADC. This type of binning is used with CMOS sensors. Software binning is less effective than hardware, but as CMOS sensors have much lover readout noise than CCD, the end result is still better. And software binning can be used no matter the bayer matrix.
[Google Pixel 8 Pro] MrBelter:
but we aren't talking about software binning here are we :)
[Google Pixel 8 Pro] Mrowkarz123:
We do, CMOS sensors cannot do hardware binning as CCD sensors can. Quad Bayer matrix is not a hardware binning, that is still done digitally after the signal goes through the ADC.

Explained: What pixel binning means for smartphone photography

Pixel binning may be the latest buzzword in consumer tech, but astroimagers know the technique has been around since the dawn of charge-coupled devices.

www.astronomy.com
[Google Pixel 8 Pro] Mrowkarz123:
About the details however. I printed some focus/sharpness pattern and took two photos from the same distance. One in 12Mpx mode and second in 50Mpx mode, both in RAW. I then copied the 50Mpx mode, and exported it in half resolution as tiff (25Mpx). After that I cropped those images to the pattern itself and exported the cropped parts. The results are, that 50Mpx mode clearly gives you a lot more resolution than 12Mpx mode. With 25Mpx vs 50Mpx the difference isn't that high, but it is till there and you can still resolve more details.

I upscaled them to the same reslutions, so you can see it better without the need to download them.
[Google Pixel 8 Pro] Utini:
If you want, then you can send me those three pictures and I will upload them on my webserver.

That way people can view it in full resolution since xda is compressing the attached pictures?
[Google Pixel 8 Pro] Mrowkarz123:
For that I will take better pictures tomorrow, on a tripod. That above was just handheld to show if the difference is visible.
[Google Pixel 8 Pro] Utini:
Nice, thanks for your effort! Let me know when you are ready :)
[Google Pixel 8 Pro] CHH2:
I still have yet to see any real world proof to counter my initial thoughts. Sitting around shooting resolution targets in your basement is great if you are the type that likes to sit around and masturbate to them later but does nothing to prove that the high res files aren't just post production trickery. They also mean zilch for real world use. No one sits around creating ideal conditions to shoot ideal targets. Going out shooting 100s of side by side photos does and in the process of doing so, I still stand by my initial assessment.
[Google Pixel 8 Pro] Mrowkarz123:
Actually I have not seen any proof from your side for your "initial thoughts". You're talking about some "post production trickery", yet you cannot explain what, and you have no proof for that. Untill you do, I don't see a point in further argument with you.

The fact that you do not agree with me changes nothing.
[Google Pixel 8 Pro] Mrowkarz123:
I've repeated it on the tripod. I did basically the same processing, but on the export of 50Mpx crop I upscaled it 2 times to make it better visible (as they are small crops of the image) and matched resolution of the rest to it. I also added 75% of the resolution (37.5Mpx) and there is still visible difference between it and 50Mpx. I've also added screenshot of the pixel grid in photoshop. All the cropped images and raw files are avaible here https://drive.google.com/drive/folders/1kerouXu0KsgxL6A66vUTwaAku02p0GQ2?usp=sharing

So yes, although bayer matrix and demosaicing may lower the detail level it cannot lower the detail limit which is set by the pixel size. In this sensor there is a difference in chrominance and luminance data resolutions, and higher luminance detail is utilized in image processing, that's why we can resolve more details even between 75% resolution and full 50Mpx (tho the difference isn't as high, but it's visible). On the screenshot with pixel grid we can see that we don't get pixel level detail and that's normal. Even with full size cameras only the best of the lenses can give you pixel level detail with higher res sensors (which have much larger pixels than we have here anyway) and even those might need stoping down to f5.6-f8 to get the most detail. So such small optics we have here simply cannot resolve pixel level detail pith 1.2um pixels. We don't have a perfect lens here, and even if we had, the detail could get limited by the diffration limit (Ayiry disc).

So in conclusion, do we get more detail than the 24/25Mpx chrominance data? - YES

Is it worth using? - It depends.

50Mpx mode will give you less signal to noise reatio and will require more light. It also takes a bit longer to take a photo as we know. But you can also see it on the examples below, in case of high density patterns it might allow you to avoid strong moiré (which is very strong on the 12Mpx photo).
[Google Pixel 8 Pro] CHH2:
In denial much? I told you exactly the place to find proof. Ignoring what it said is on you.
[Google Pixel 8 Pro] Mrowkarz123:
Oh no, I'm not going to try to find proofs for your claims, that's on you :D So far you provided no proofs.

I have provided proofs that your claims are not true. You claim that we get at most 24/25Mpx of resolution, yet the test shows that in reality it's even more than 37Mpx. No software trickery (that you can't even explain nor understand, yet you claim it to be true) could give you so much detail over those 25Mpx with just few frames taken in that 1-2s it takes it to take those 50Mpx shots.

The reality of this is different than you claim it to be. Unless you have true and detailed technical explanations for this, and this all "software trickery", your claims are just false.
[Google Pixel 8 Pro] CHH2:
Oh, no. I stated it. You went and read it and then ignored everything. Remosaicing and trying to combine data from 8 frames IS interpolation which is just a fancy word for computer trickery to create data that is not there and cannot be there due to the physical design. Case closed.
[Google Pixel 8 Pro] Mrowkarz123:
And by stating this you don't even understand the physical design of that sensor and how it works. If you believe that 8 frames will give you so much detail you're really naive :)

You still didn't show a real data of how it works, not a single real test showing that it is how you say it is. I did my own test and I do see what detail I get from those images. I do also know that you need a ton of frames to get more detail with, for example drizzle algorithm, 8 frames will never give you that.
[Google Pixel 8 Pro] CHH2:
Your test don't mean squat. 🤣 And to say I don't understand the physical design of the sensor and how it works?!? What drugs are you on because I want some of those. That's some seriously mind altering ish you're on. I literally gave you a source that states that the P8P has to take several frames in order to create the "50mp" files. All anyone has to do is put the phone into "50mp" mode and take a photo and see the dot in circle come up (doesn't matter how much light, the dot in circle ALWAYS comes up) because it HAS to take several frames in order to create the "50mp" file because it isn't doing a true straight read off of the sensor. That's cool that you have some base idea of how digital imaging works but that's all you have. You are base. Adios. Good bye. K.M.A.
[Google Pixel 8 Pro] CHH2:
Seriously. What kind of drugs does one have to be on to know that the sensors on a phone are physically handicapped down to a quarter of full resolution and still argue that they can natively put out full resolution?!? I want to know because I want to buy them so I can experience this mind altered state. 🤣

I've said it before and I'll say it again. I've pointed to sources that state the P8P is taking several photos in order to create the "50mp" photos from a sensor that is handicapped by the physical filter system that sits over/before any light hits the actual photo wells. By even Mrowkarz123's own words, the system has to put those frames together (remosaic/interpolate) a "50mp" image.... even from sensors that only have 48mp worth of photo sites.

So, please, don't tell me "I don't understand". Point me to a paper or technical site that says that it is physically possible to create a photo file that is beyond the physical limitations of how a sensor is set-up. I'm still waiting but all I get is "you just don't understand". I understand. I'm very much a man of science. Until I see peer reviewed research that alters my understanding.... you're just talking BS.
[Google Pixel 8 Pro] Mrowkarz123:
You pointed to one source that tells it takes few frames, but does not specify what exactly it does with them xD

You definitely have to be on drugs to believe you can get such detail gain over 24/25Mpx from just few frames, that is just impossible. You're also altering what I said (putting the frames together and remosaicing is not the same).

And it seems that chrominance and luminance data are the same for you - that's where you don't understand the differences.

Seriously with so much bs you said it's not worth continuing that conversation with you xD
[Google Pixel 8 Pro] CHH2:
Welp. I've given you more proof than you've given us. Seeing as I've pointed to sources and you can't (you literally can't) I think that about sums up who is talking BS.
[Google Pixel 8 Pro] sedazin:
You can check sample galleries for example on dpreview.com:

iPhone 15 Pro sample gallery: Do its high-res photos measure up?: Digital Photography Review

The iPhone 15 Pro allows users to capture 48MP photos in HEIF or JPEG format in addition to Raw files, while new lens coatings claim to cut down lens flare. How do the cameras in Apple's latest flagship look in everyday circumstances? Check out our gallery to find out.

www.dpreview.com

Google Pixel 8 and 8 Pro galleries: Digital Photography Review

We've been shooting Google's two newest smartphones on two continents: from the parks of Madrid to the mountains of Washington state and the streets of Seattle. See what the search giant's latest cameras are capable of.

www.dpreview.com

or gsmarena:

Apple iPhone 15 Pro Max review

The Apple iPhone 15 Pro Max has a total of four cameras - three on the back and one selfie camera. Each of these setups is further assisted by a LiDAR...

www.gsmarena.com

Google Pixel 8 Pro review

The Pixel 8 Pro continues in the footsteps of the previous generation with its camera system, offering a trio of 50-ish megapixel cameras on its back. In...

www.gsmarena.com

I own both phones. The resolution advantage on the iPhone using 48MP HEIC or ProRaw is clearly visible compared to the 12MP mode.

With the P8P I struggled because the advantage was not so visible and Google obviously cooks more than Apple when it comes to the high res pictures but hence I am not short on memory/space I chose to go with the higher resolution.

How useful this resolution advantage is, depends largely on what you are doing with the images. I assume for large prints and crops it is most beneficial.
[Google Pixel 8 Pro] sedazin:
I can really not see much evidence in your claims. Yes, there is for sure upscaling from 48MP to 50MP on other than the main camera and yes, google is cooking heavily with multiple exposures but that could also indicate the standard google magic on HDR which is also baked into the DNG files.

I can not see evidence that there is upscaling on-going on the main sensor and that multiple exposures are used for it. Looking at the sensor specs - and here your assumption of a quad-bayer pattern is wrong - you can see that there is no need for up-scaling:

It looks like the sensor is a Samsung GNK with Dual Tetrapixel RGB Bayer Pattern which is the same pattern the Samsung Isocell GN1 uses. The trick is that the sensor has 1) a full resolution standard bayer pattern (small pixel size, small circle of confusion, due to that much light needed to use it in this mode and short exposure times) and 2) a low resolution mode in which four pixels of the original 50MP bayer pattern are combined into one pixel (larger pixel size, larger circle of confusion, captures more light due to 4x area, less light needed and longer exposure time possible).

ISOCELL GN1 | Mobile Image Sensor | Samsung Semiconductor Global

Samsung ISOCELL GN1 brings every detail of your scene to life. It delivers 50MP(million pixels), 1.2um pixel size and 1/1.31 inch size.

semiconductor.samsung.com

-> klick on moon and sun icon to see the patterns

The high-res mode is more demanding when it comes to stabilization due to the smaller pixel size. It also will produce more noise which in combination with the usual post-processing for noise reduction and including Google's HDR magic will lead to vastly longer image processing time due to the 50MP. There is simply much more data to be processed when combining let's say five raw readouts with 50MP compared to five readouts with 12 MP.
[Google Pixel 8 Pro] CHH2:
Oh boy. No, the sensor does not have two different bayer patterns. I don't care what the animation on their site says, it does not magically switch from a full resolution bayer pattern and a quad bayer pattern. That would be cool if it did but I can guarantee you that it does not actually do that.
[Google Pixel 8 Pro] sedazin:
"I can guarantee you" is not providing evidence.

I did not claim that there is any quad bayer pattern. I made reference to the spec sheet of the manufacturer where you can clearly see that it is exactly _not_ a quad bayer pattern (which would mean adjacent 2x2 pixels are the same color).

It is a 50MP bayer pattern. The sensor has an effective resolution of 8,160 x 6,144 (50MP) according to its spec sheet. So there is tecnically no need for any upscaling magic but just de-bayering when it comes to 50MP images.
[Google Pixel 8 Pro] CHH2:
First, you linked to the marketing page for the GN1 sensor and not a technical page. Huge difference between the two. (And it should be pointed out that the GN1 does not match the capabilities of the P8P sensor. The GNK does point to the same capabilities though.) That page is where the marketing team gets to put their spin on things. Case in point their images on their showing two different bayer patterns is simplifying things to the point of being wrong.

Sure, they list 50mp because it contains that many photo sites. The bayer pattern bit is a bit of "better to ask for forgiveness than to ask for permission" or fudging the specs. It IS physically a quad bayer pattern sensor. That's why they call it a tetrapixel sensor. Tetra = quad = 2x2 = 4. So you have 50 million photo sites but that only means you have 50 millions spots to measure grey scale but physically you only have 12.5 million color collection points due to the nature of the physical. And to the human eye, that's a huge disconnect. Color detail matters. A lot.

Which is why there has to be computer trickery behind this sensor to get it to make an actual 50mp image. (And even more so to make it from the 48mp sensors.) Which we know is happening because A. It's Google and that's Google's solution to all things photography. B. I can take two photos in the same exact blue sky sunshine day. One in the 12.x resolution and one in 50 resolution. The 12.x will snap off near instantaneously as any camera would and should. The 50 reso will have the dot in circle UI because it is taking several photos to combine to make the photo. At no point in any light does this happen instantaneously like a real photo.

"Debayering" as you put it or, as it is better known, demosaicing is what happens in order to get the 12.x mp image. To get the 50mp image, the information is demosaiced, then remosaiced. This is where the extra computer trickery comes in to attempt to generate the missing color information that would have been there if this were a true 50mp bayer pattern sensor.

But hey, lucky for y'all I was finally able to track down a little better information. It's for the GN2 sensor but that's closer to the GNK that's supposed to be in the P8P than the GN1.

https://hc33.hotchips.org/assets/program/conference/day2/(Final)Hotchips2021_CIS_Samsung_ISOCELL_GN2.pdf
[Google Pixel 8 Pro] sedazin:
OK, let's assume the martketing pages are wrong and the pattern is a quad pixel pattern (actually I was not able to find spec sheets and I agree that most likely you are right). Looking at the sheet you linked it looks like the remosaic algorithm is already part of the sensor or the pipeline provided by Samsung. You think google takes a multi-capture approach to gather data to improve the color information? This would make sense.
[Google Pixel 8 Pro] Mrowkarz123:
Or just take few frames for a plain HDR composition. I guess it cannot use Staggered HDR in Full Res mode as it uses alternate rows of pixels for different expousures (which is not a problem in 12Mpx mode as those rows will be combined anyway).

It is possible to improve to color data by combining multiple frames, it is used in astrophotography with drizzle, but it requires a ton of frames (more like hundreds) with small, sub pixel shifts. There is no way you could get that from only few, randomly shifted by hand (or even without the shift on a tripod) frames. You wouldn't be able to bring that extra data from the noise of those freames.

Of course it have to use special remosaicing algorithms for colour, never said it does not. But again, it's only color, part of the story, where luminance is actually more important when it comes to the detail reproduction, and the picture below is extreme example why. You can have half resolution of chrominance, and the picture overall will be worse than with full res color data, but you will still have more detail, if your luminance is full res. And although quad bayer sensors are physically limited in terms of color data, they CAN utilise full luminance resolution and it does inorove the detail reproduction.
[Google Pixel 8 Pro] CHH2:
"...only color..."

And this is where the disconnect is. I'm guessing y'all have zero idea how human vision works. (My uni subwork was in human anatomy and physiology in terms of human performance. Photography has been something I've been working with previous and post to that for decades in both film and digital.) Color is king to human vision. There's a huge reason camouflage works so well. The use of color to break up silhouettes. Color detail matters A LOT to human comprehension of the environment around them.

Unlike a photo site, that has both a color and grey scale value coming out of it, human vision takes a step backwards... kinda. The main part of our vision is color sensing cells gathered around the main part of our vision. These are the cone cells. Mostly around the periphery are the rod cells which see a murky grey scale but they are also more sensitive to motion. There are a few rod cells scattered among the more central cone cells but the main focus of our sight is cones/color.

So as I alluded to earlier, camouflage works great because the colors break things up.... until whatever is "camouflaged" moves quickly. Then we get a "disconnect" and can pick out the incongruity. And that's picked up "out of the corner of our eyes" which is where the rod cells are more concentrated. (Fun tip: In the past when I coached athletes in various martial arts, I would always teach them to never directly look at their opponent as that would slow them down. Always put them slightly off center and soften the gaze. Reaction times always went up!)

Our eyes when taking a critical look at the world around us...... are like a traditional bayer pattern in which each individual site is seeing color AND scale of color. Anything else is a disconnect.

The sensors like in the P8P are seeing a 2x2 grid of one color in scale vs a pixel by pixel scale of color. Then each pixel in the 2x2 grid only has 2 out of 4 adjacent pixels to help it interpret what it is seeing vs a standard bayer patter which has 4 out or 4 adjacent pixels to help it interpret what it is seeing. Which means additional algo's need to be created and applied to guesstimate further beyond what would be available to a normal sensor to make the image make sense to us. The grey scale data is nothing to us without the appropriate color data unless what we are looking at is moving and then only so on an extremely base evolutionary level.