yashsingh1/xdaforums huggingface dataset:
[Xperia Z1 General] wezzel98765:
Can use camera fv-5 to save in PNG as it is, which ends up as 18mb files... so pretty much already have RAW imho
[Xperia Z1 General] RHBH:
They should give us the option to choose which format and the compression rate.

I would like JPEG images with 8~10 MB per file. 18MB PNG is too much!
[Xperia Z1 General] danw_oz:
Yep, that is as good as you will get right now, there is no denying that.

As a side comment, if you open a stock jpg from the phone in Photoshop through camera raw do nothing with it and save it at quality 12 jpg you will end up with jpg files as large as your png ;) no difference in quality just bigger file :eek:

Do you really understand what camera raw gives you? And how in reality it is nothing like saving a file as PNG, as per the comment directly above, that's whats happening with your fv-5 app.

You have a native image straight from the image sensor, no software processing of the image just what the image sensor saw. (most importantly no noise reduction or sharpening)

Your images currently are only 8bit, chances are raw images will be able to support more bit depth allowing far more shades of colours.

You get to choose you white balance to any temperature as if you were selecting it at the time of shooting in jpg, but after the fact with no strange hues as you would if you tried the same with a jpg.

Lossless processing of photos.

You have pretty much complete control of the output of the final file, as a hypothetical example, superior auto vs manual control.

jpg - 8bit file format is capable of displaying 16.7 Million colours (256 shades per primary colour)

10 bit raw file - 1 billion colours (1024 shade per primary colour)

12 bit raw file - 68 billion colours (4096 shades per primary colour)

My DSLR 14bit raw files are capable of capturing 4.4 Trillion colours (16384 shades per primary colour)

In the example above when working in camera raw with raw files you have far more shades accross the image to work with provided the image is not over or underexposed, this is also part of the reason why DLSR images look so much better especially with skin tones because the representation of these skin tones is much closer to what you can see with your eye.

Once you have tuned your shot exactly as you need then saving as a jpg will create the correct file indexed out of the 256 possible shades per primary colour based on the information provided from the colours within the final edited image.
[Xperia Z1 General] wezzel98765:
I'll walk away with,My tail between my legs now ;)
[Xperia Z1 General] danw_oz:
Yeah but the biggest pain in the behind for raw images, is having to process each shot you take :eek:
[Xperia Z1 General] supercoolman:
when image size gets above 20MP, it's going to die on write speed of slow internal/SD.
[Xperia Z1 General] danw_oz:
Yeah with you on that, also if they introduce the 4k video recording that the processor is capable of.

I am in a dilemma now for exactly this, tying to future proof as I am in the Market for a 64GB card and wondering if there is any benefit to get the Sandisk ultra @ 30Mb/s or Extreme @ 80MB/s, because I am only seeing about 12MB/s on average from my current 32GB Ultra.
[Xperia Z1 General] One Twelve:
This is why there are specialised chips to do RAW on DSLR's. The card write speeds are also much faster. There isn't a need for anything above class 6 in present android devices as they cannot write faster. But the only 64GB microsd card is a class 10. Nobody makes anything that size in less speed.

Some points i disagree with in the OP's arstechinica article.

Lol, its a defnite improvement over the nex4 in image quality.

Only if you're from the lowest common demominator which iphones target. Please no options at all, just one button to click.

The article goes on about how RAW was not ready for kit kat. So that means we ain't seeing anything until the next version of android come out. When will that happen ? next oct. So this is just a taste of things to come a year from now.

The main advantage of RAW is it allows more post processing with minimal loss to image quality. With jpegs, the most you can do is colour correct, contrast boost and then maybe a crop in desktop software before loosing too much detail.
[Xperia Z1 General] danw_oz:
That doesn't mean that because it was not available for launch of Kitkat that it couldn't make it's way to 4.4.1 or other iteration before lime pie or lolly cake or whatever they call it :)
[Xperia Z1 General] One Twelve:
True. I was not expecting a 4.3 or 4.4 thought it would jump to 5.0

I don't know whether there will be a 4.5. If so then there is a chance.

it will happen eventually though if Nokia is enabling RAW in the 1520. First i hear of that.

What this will mean is over processing is no longer an issue. A small power starved device like a cell phone isn't the best at doing image compression in the first place.
[Xperia Z1 General] TheCraig:
My Xperia Z1 kicks my sisters iPhone 5S around when I use manual mode... she even admits (and she is a loyal Apple fan) that I have a better Camera. Girlfriend continues to be pro-Apple and says that her iPhone 5 has better images (and it clearly does NOT).

Anyway this is something that has been long awaited, think we should have had it earlier. It could be ready for 4.4.1?
[Xperia Z1 General] One Twelve:
We never got to see what the camera can really do. All we saw is what the image compression program can do (!)

People think if they save in 20MP, the quality will be better, than 8MP but it isn't.

Larger sensor means you have more space for digital stabilisation via cropping.
[Xperia Z1 General] TheCraig:
Sony makes absolutely great hardware, but they really do need to get a better software team, Xperia's software has been lacking since I originally got my Xperia Play. If they had a better software team they would be unstoppable in my opinion.
[Xperia Z1 General] jonmorris:
Sony seems to do okay for some apps, like the album, Walkman player, movie player, photo editor etc.

It just struggles to do a decent camera app.
[Xperia Z1 General] danw_oz:
Spot on, we don't then have to wait for Sony to fix their image processing algorithms
[Xperia Z1 General] danw_oz:
Yep agreed, even my old HTC phone was way better than my friends iphone (of the same vintage), and after a camera shoot off with him, he left with his tail between his legs :victory:

As for 4.4.1 who knows but all we can do is hope. I think Google realise now that most phones are now stepping over the 8MP barrier that they have to do stuff to keep up from the OS perspective, I would suggest even newer linux kernels that would allow better memory management and faster internal usb throughput 10-12MB/s just does cut it today, and less so if 4k video gets implemented.
[Xperia Z1 General] jonmorris:
The Nexus 5 camera is excellent. It captures stacks of detail and works really well in low-light. But you wouldn't know that from using the native camera app, which struggles to focus and really can't cope with moving objects in anything but excellent light. Anyone would think they did all their testing outdoors in the hot summer sun, without trying to capture a picture of your active baby in a living room.

Just like the Z1, the phone is let down by bad software not bad hardware.

But now it seems Google has acknowledged this and is seeking to fix it. Perhaps it will send out a message to Sony and others that you need the expertise in producing good camera software, not just chucking in a fantastic image sensor and hoping that's it.
[Xperia Z1 General] One Twelve:
Sony won't need to do it. This means its unlikely we get to see a message like this any more.

Why did it overheat, too much CPU from image compression (along with other things in the background).

The real question is whether you can tell the difference. I have to believe a PC taking its own sweet time to batch process many images will do a better job than a mobile that needs to put out product quickly so you don't wait too long to take another photo.

How long will it take to write a 20-30MB RAW file to internal memory ? ~2 seconds

Internal memory would be faster than external.
[Xperia Z1 General] danw_oz:
Same as all other cameras they cache the images in internal memory then write as fast as they can to the memory card, this is also why many cameras have a low limit of buffered images before the continuous shooting stops for a time or slows right down.
[Xperia Z1 General] jos_031:
Now i hope they make display that support bit depth greater than 8x3 bit...

Oh wait samsung do make it.. so called super amoled...

wait but they are over saturated...

really? is it samsungs fault or googles??? for not supporting higher color depth in android and hence make it look saturated...

hey cant complain google as gpu in phones cant yet support higher bit depth..

but i use photos on pc...

I hope you have professional graphics card, as geforce or radeon graphics card does not support deep depth.. and you paid for color accurate monitors..
[Xperia Z1 General] danw_oz:
Well actually my laptop (not mac) has one of the very few IPS screens in the world that can display the full RGB gamut and is professionally calibrated with 3D, so don't worry for me ;) I got that covered.

BTW it is the monitor/screen that is the draw back for lack of colour representation not the graphics card, although I do have a dedicated 3D nvidia card, so think I am good with that one also.

And since we are ultimately talking about camera here and raw photography, the sensor on the camera is capable of capturing wave lengths of light well beyond what you can see with your eyes, so I guess we are screwed in reality by our own eyes.

I am not sure if you know what you are talking about pal. Pretty much everything you say here makes very little sense.

What I do know is that the colour representation on the Xperia Z and the Z1 (Z1 Tril screen displays more shades of colour) and both are very accurate, and the Z1 is a little too warm, but the Z was a little too cold, after processing my raw photos I actually use my phone as a sanity check that the colours and contrast are good, and trust me the phone has shown me a couple of times that I had coloured ambient light reflected on my screen while processing, if I was not so lazy this could have been overcome by having my calibration hardware plugged in at the time.
[Xperia Z1 General] One Twelve:
How to do colour calibration on the Z1 ? How to know how colour accurate it is.

I'm kinda surprised that the Z1 screen would be reliable enough to spot the underlined bit. Because its just a phone screen, the USP is small form factor, full hD and ok-ish colour reproduction for the masses. Not colour accurate or close to it. Monitors that could be trusted to do colour accuracy would be at least as expensive as the Z1 if not more.
[Xperia Z1 General] jos_031:
first even if you pay a million dollar you wont get any display that can display full gammut human can see. since it does not exist.. you can get sRGB gammut since almost all pannel are sRGB. getting higher bit for sRGB gammut is pointless since no one can percive the difference.. IPS is not good at color reproduction hence not much used for color critical equipments.. the graphics card is designed to output maximum of 24 bit (8x3). even of you make changes in os the bit depth does not changes since the output device does not support.. if you get professional graphics card like quadro it supports upto 48bit called deep field..and the point is... what is the point in taking photos with greater pixel depth if you cant see them as it is. the color temp of z1 is much more closer to 6500k than any smartphone.
[Xperia Z1 General] danw_oz:
Dude, this is a discussion about the phones camera and the possibility of future raw capability, read the thread title. This is not about screens, graphics cards etc, if you want to discuss that then go nuts in the display threads or create your own one ;)

I still do not understand your point, because you have just talked right around the bush and then contradicted yourself. No display can display the full gamut the eye can see, then bla bla bla, then what is the point of having more colours with bit depth.

So lets take it back to basics for you (dynamic range)

As a ridiculous example so you might get it. If you have a photo with one shade of say the black end of the scale, and one shade of white end of the scale captured, then you have another photo that contains 6 shades of the black scale and 6 shades of the white end of the scale. Regardless of how good or bad your screen is you have far more ability to pull details out of 6 shades of a scale than 1.

Semi pro and pro photographers mostly use the raw capabilities at the dark and bright ends of the spectrum to get more details in those areas. So in a bright sky shot, multiple shades of grey looks much better than white that lacks texture.

My point here is that the screen discussion is a completely different topic, and in reality has nothing to do with my discussion of the benefits of raw photography. In reality the screen only assists in ensuring that white balance is correct and the colours are as accurate as they can be, different topic :eek:
[Xperia Z1 General] danw_oz:
I know the phone screens accuracy or lacking points because I have a calibrated screen to compare it with in the first place. At the end of the day if it looks good on my phone the shot is going to look good on many other screens
[Xperia Z1 General] jos_031:
why i talked about display is since someone talked benifit of bit depth.. human eye can distinguish only 10 million color.. the sRGB mapping of image is addressed using 24bit.. if we increase the depth the distribution of RGB will not be a linear graph without using profeesional card and benifit does not exist.. if the gammut is extended then 255 can represent the actual sRGB 255.. and the ones up 255 can use the wider gammut...this way it does not look artificial ... you can check this by creating big color matrix in MATLAB...
[Xperia Z1 General] hasona:
Maybe I am late but I can confirm to you that the sd card r/w is around 40/40 on my SanDisk Extreme 32gb(rated for 90/50). As i understand it that is pretty much enough for RAW and 4K
[Xperia Z1 General] One Twelve:
40/40 is in Mb/s ?

Minimum performance of a class 6 is 6MB/s or 48Mb/s.
[Xperia Z1 General] hasona:
Yeah my writing was 41MB/s and reading speed was on 42MB/s with benchmarks on a 500MB file on the phone.

Generally speaking Class 6 has minimum of 6MB/s in writing speed if I remember right and a Class 10 has it at min 10MB/s

This is the one I have which gives me 89mb/s and 59mb/s on my pc in r/w: http://www.amazon.com/SanDisk-Extreme-microSDHC-Adapter-SDSDQX-032G-U46A/dp/B00D6XXL7M

edit: Ohh my I am so sorry for messing up the prefix, I am writing this on my Z1 with autocorrect :P in this case I am talking in MB/s as in Megabytes and not megabits

Fixed them :P
[Xperia Z1 General] One Twelve:
Still not clear..

Should say 41Mbit/s and reading at 42Mbit/s ~5MByte/s

Writing to internal memory will be faster than external sd card.

The PC's sd card reader will be faster than the phone's reader.

read speed (89mb/s) is twice as fast as (41mb/s)

The write speed however is only 50% faster.
[Xperia Z1 General] hasona:
No no no this is messy now. Start from the beggining:

Test on my Sandisk microsdhc 32GB Extreme card which is rated for 80MByte/s in read and 50MByte/s in write

I ran two tests one on the phone and one on my usb 3.0 connected card reader

PC test gave me the following numbers: Read - 89MByte/s and Write - 59MByte/s

If you are like me - transferring large files almost every week then the solution to a good numbers is a usb 3.0 connected card reader

Z1 test gave me the following numbers: Read 42MByte/s and Write 41MByte/s

These numbers are A LOT better than normal class 6 or class 10 cards that are not rated for any extreme usage

In no way did I state Mbit/s in my posts after editing them.

I am using MB as in filesize. 40MByte/s would equal roughly 320Mbit/s

So to clarify, the speed on a Z1 seems to max out at around 40MByte/s in R/W and NOT 5MByte/s
[Xperia Z1 General] One Twelve:
ok, very clear now.

I've not used the extreme cards before as i did not think the phone's readers and writers would be able to make use of those speeds.

Which filesystem are you using with these extreme cards ?
[Xperia Z1 General] danw_oz:
I have a Sandisk Ultra, and connected with my USB 3 UDMA/6 card reader I get close to 30MB/s read and 20MB/s write, through the phones USB starts at the previous speeds mentioned for a short while then settles at 10-12MB/s, even my Xperia Z did this. Mind you I am taking this from what the computer can send to the phone, not what the phone can write directly to the card, even though I would expect it would be similar. I have purchased the Xperia SDCARD Mounter app from Doomlord which works much better.

Good to know because I was looking at buying a 64GB Extreme because I was disappointed with the performance of the ultra.

The one thing I do not like for micro and sdcard ratings is class 10 limit or seems like. Class 10 is only guaranteed to give you 10MB/s, with these extreme cards I think they should class 20 otherwise if you have a problem card that is still only giving you slow speed they can just say well it is performing as per class 10 requirements.

I only have extreme pro compact flash cards for my DSLR, but I know for sure that my camera has an UDMA6 writing and reading capability so can get every ounce of performance from these cards. Even these cards are only guaranteed to sustain 20MB/s while recording full HD movies, the very latest 64GB is 65MB/s for Video & around 160MB/s for everything else.
[Xperia Z1 General] danw_oz:
Yeah me neither, since it can't even come close to the Ultras rated speeds
[Xperia Z1 General] One Twelve:
SanDisk products transfer speed

std does a minimum in MB/s as the class indicates eg, class 4 has 4MB/s, class 6 has 6MB/s etc

ultras do 10 -30 MB/s

extremes do anywhere from 20 - 100MB/s

This is the part i was quoting

So according to haxona, the Z1 sd card reader can go a little bit faster than an ultra or 40MB/s.

Do you have any ultra cards, haxona, what is the fastest you got with them ?
[Xperia Z1 General] hasona:
If I find one stacked somewhere in my desk I will post, I did have one I just can't figure out where I put it :P Report back tomorrow

Filesytem used on mine is exFAT

Edit: Okay so I managed to find my old ultra card and It's just terrible - I hit 10.40MByte/s in Write and 12.5MBytes/s in Read
[Xperia Z1 General] danw_oz:
Perfect so it is not just me :victory: And purchasing an extreme will have benefit :D, still a little perplexed why when I use my UDMA card reader that the Ultra card is able to sustain much higher speed than the phone.

My ultra does the same using internal phone bench mark as from my pc, write speeds all over the place, reading at 50MB/s writing average 8MB/s
[Xperia Z1 General] One Twelve:
Hah! there we go, more normal speeds :laugh:

Thing is when you're transferring, the card is in the phone, so you're choice is either microusb or wifi.

wifi tends to get used more and the speeds there are much slower ~2-4MB/s
[Xperia Z1 General] danw_oz:
Here is the best bench mark (Sandisk Ultra 32GB) I just got from a different app, the result is all over the place.

Starts off with a hiss and a roar with all tests writing at the beginning at upwards of 40MB/s, then falls back to between 9-12MB/s, here are the results for my ultra with the sdcard tester app.
[Xperia Z1 General] One Twelve:
Yeah, i've tried that app.

The question is whether you get a sustained transfer speed or not with the extreme. Small files take advantage of burst speeds, bigger files will not.

If i was loading up a 32GB card, via usb 2, time to fill it is 30-40mins.
[Xperia Z1 General] danw_oz:
If I was filling a 32GB card I remove the card and put it into my UDMA USB 3 card reader, and the card sustains high speed depending on the file sizes.

Larger files sustain full speed, until the CRC checks, small files are going to be slower because the host system needs to get the next file CRC check each file so much more handshaking going on between both file systems.

So to copy a whole lot of photos, or mp3's you will have slower through put. A movie file will sit at about 30MB/s the entire time, until it gets to the end.
[Xperia Z1 General] hasona:
With an extreme card you can definitely come over 30MB/s after 3secs with a big movie file. Thing is you need a usb 3.0 card reader and port.

And then of course it's like you say small files will have a slower speeds because of the checks on every file, that is normal behaviour on all storage devices.

Btw I couldn't get that other app to work on my Z1 it just couldn't test my sd card but from what I gathered the top on read is around 40MB/s
[Xperia Z1 General] danw_oz:
Yeah I am not so worried about read speed but write speeds
[Xperia Z1 General] hasona:
Good that the write speed is equally as good as read speed with a good card(extreme)
[Xperia Z1 General] jontr:
mine 64GB SanDisk Ultra got writing speed at around 12MB/s and reading at 39MB/s

not bad for a 64GB I guess :)
[Xperia Z1 General] danw_oz:
Sandisk Extreme 64GB on order will keep you all posted once I do some tests on it.
[Xperia Z1 General] Jimvest:
I have Samsung uhs-i pro 64GB Read is 40MB/s and write is 20MB/s
[Xperia Z1 General] hasona:
Let us now so we can compare to my tests both on PC and Phone :)
[Xperia Z1 General] danw_oz:
Right it arrived today, 64GB Sandisk Extreme, three bench marks one in phone app, mounted sd through computer, external card reader (pretty much at least twice as fast)

My guess is now that around 40MB/s is the limit of the phones interface to access the micro sd, regardless how fast your card can read or write.

Also one screenshot of Sandisk Ultra 32GB

Another bench mark tool, which shows the performance of all the ram on the phone.
[Xperia Z1 General] hasona:
So the results didn't change a lot from mine. What format did you use though?
[Xperia Z1 General] danw_oz:
Formatted in phone so extfat, fat32 not supported @ 64GB
[Xperia Z1 General] hasona:
Ahh Ohh well that's nice :)
[Xperia Z1 General] danw_oz:
Was your card 64 or 32GB?
[Xperia Z1 General] hasona:
The 64 was out of stock so I got the 32 extreme card. Still plenty of storage for my needs
[Xperia Z1 General] One Twelve:
You can do fat 32 on a 64GB card with a utility called fat32 format from ridgecorp but it will limit the size of your files to 4GB max.

it's not clear whether fat32 will be faster than exfat.
[Xperia Z1 General] danw_oz:
Yes but highly not recommended
[Xperia Z1 General] One Twelve:
Why ?

I've been using a 64GB ultra in my sansa clip+ for nearly a year now without any issues. See, i had no choice as the clip+ will only read FAT and i needed 64GB. I usually just add files to it slowly and rarely delete.

A 32GB class 4 i had which was used more regularly gave up the ghost in less than a year. No tweaks were used with that card, default FAT and normal use, download delete. Was disappointed with the life this card had, did not expect it to die so fast. Whenever you see messages from android that card was unexpectedly removed etc for no reason, it means the card is going to die sometime soon. Won't miss that next time.
[Xperia Z1 General] danw_oz:
Do a search on the internet for fat32 format partition size limitations, for me I will use a format that is supported natively eg extfat, not use a format that was never intended to do this and then rely on the software written to try to bypass this. In my opinion do it if you wish, but don't be upset if you get corruptions.