yashsingh1/xdaforums huggingface dataset:
[Tilt, TyTN II, MDA Vario III General] Alemaker:
The issue is the fact of advertising that these devices do provide an "exceptional" experience and again, by HTC's own advertising claims, with press releases of the capabiliies of the included hardware, that a reasonable person would expect such capabilities to be included, not excluded, and not to be disclaimed until after the fact of the issue being brought up by customers.

Whatever the outcome, HTC has shown that it truly does not care about their customers, period.
[Tilt, TyTN II, MDA Vario III General] unwired4:
We wont get anywhere until there is proof that the device isnt exceptional.

Its the best device I have owned (and I have owned a lot and set up hundreds for clients).

So lets see the test results. If yours is as bad as you you feel, then your benchmark result will be dramatically lower than the two we've posted here today and people's opinions of HTC might change slightly as their assertion that the Kaiser is as good as they say may not be an "outright lie"...
[Tilt, TyTN II, MDA Vario III General] Alemaker:
The issue boils down to advertising, and hopefully some action is taken, and I stand firm on my earlier statement about HTC's lack of customer service as evidenced by this situation. However, I will benchmark and post my results, as it will provide evidence, either way. Again, you sidetrack the issue by avoiding it, the advertising claims are what is at issue ,and HTC's intentional ommission of software that would be reasonably expected to be included in the hardware as advertised.
[Tilt, TyTN II, MDA Vario III General] jgermuga:
I have no idea what this means. What does it prove? If you are trying to compare against the hermes, which is it's predecessor, where are the numbers for the hermes?

I understand the memory architecture of the OS is now differnet from what it was on WM2003, but I don't think this should excuse the issue. If I thought WM5 would run faster, I would install it, much the same as many people downgrading from Vista to XP, which many people have done due to poor driver support for Vista. Oh, and by the way, most new systems shipping that are set up to run Vista run it jsut fine because they are sold as a package and the manufacturer and/or retailer make sure the Vista drivers are included.

If you are looking to run Vista on an older machine, you can take a look at the recommended hardware to see if your machine will support it. If it does, chances are it will not be as fast as older OS's and you may even have to forgo using Aero, because your graphics card is not up to snuff. If it doesn't work well or fast enoough, just go back to XP.

The point here being, the choice is up to the consumer.

For Pocket PC devices, these choices are not left up to the consumer. And before you tell me this is possible via XDA Developers, keep in mind that flashing a custom cooked ROM is not supported by ANY manufacturer, or ANY wireless service provider. In fact, it will void your warrantee. This is not the case for downgrading to XP.

Since this is the case, the only comparison I think is relevant is to pull both devices out of the box with no customization whatsoever other then factory/vendor suppied patches, so exactly as it would be supplied to the consumer, then run the programs and OBSERVE the difference. If there is no little or no difference, this is actually counter to what HTC is saying "More powerful than any mobile communications device you've seen before".

I don't know how this can be made any clearer.

Now as far as the drivers being the reason there is no OBSERVABLE difference may still be in question. As I have said before, the best way to make this comparison would be to have the same exact device with the drivers being compare to one without. I don't know that we will ever have this situation, if we did, the problem would be solved and we could all move. But we don't have the drivers, so what is the next best alternative? I'd say, compare to a device that is most similar from another manufacturer. At this point, I'd say the LG KS20 is the front runner along with proof that they do use the Ati Imageon hardware.

If you have some suggestions on how to prove the drivers result in a performance difference (for better or worse I might add) please offer them, but as far as I am concerned, the benchmark numbers are near meaningless especially in light of the devices being compared.
[Tilt, TyTN II, MDA Vario III General] liddlefeesh:
Thank you your inquery has been send

I've submitted a complaint comment. I encourage everyone to do the same.

I take it "Thank you your inquery has been send" is a good thing? :)
[Tilt, TyTN II, MDA Vario III General] unwired4:
1) The qtek9100 is the wizard. The forerunner to the Hermes. If someone has a hermes to hand could they please run the benchmarking app on that?

2) The point of benchmarking is to prove that a) the devices tested so far arent slower at the graphics/video/sound etc than predecessors and b) to comapre all of our Kaisers.

I remain convinced that those of you saying that your devices so slow that you can see icons being "drawn" on the screen will get a significantly lower score on the benchmarking tests. This will prove that you are correct that your devices are slow and that us happy kaiser owners are right that our devices aren't "crippled".

If our test results are similar, then either our expectations were woefully low or yours were exceptionally high.

And to Alemaker, I agree in principle - but the tests may yet prove that your device is slower than mine and that there is thus a specific problem with some Kaisers and not others!
[Tilt, TyTN II, MDA Vario III General] gogol:
unwired4,

For me it is quite simple.

I cannot play K-Rally, Resco Guardian, Sky-Force, and Sky-Force Reloaded with relatively enjoyable experience.

The speed is terribly slow.

This is quite the opposite when I play the above games with my old HTC Magician.

Do you think that I should forget about playing games on my Kaiser?

Bah!

If you say that "Well, I dont play games !"

Fine! Then it is not a problem with you. Be happy.

But dont say that there are no performance problem with HTC Kaiser either!

As simple as that.
[Tilt, TyTN II, MDA Vario III General] wim26:
my benchmark was also not good. it only got to 1374 in total :(
[Tilt, TyTN II, MDA Vario III General] unwired4:
I have not said there is no problem. I merely dispute that all users have the same problem to the same degree. I cannot believe that there are so many happy and so many unhappy users all using the same performance. So we need to work out which devices are worse and why?

Unless you run the tests and publish your results, we wont know.
[Tilt, TyTN II, MDA Vario III General] unwired4:
What device and rom?
[Tilt, TyTN II, MDA Vario III General] Scrog:
My scores

Q-mobile 1.2 on ATT Tilt

Graphics 1059

Others 1954

JPEG 2323

Games 1289

Sound 0672

Final: 1461
[Tilt, TyTN II, MDA Vario III General] gogol:
Why should I run the test?

I dont need to give you any proof or such.

I dont care if you know the problem or not.

In fact, you said that there are problems!

This thread is to discuss about that problem.

Problems that you have admitted yourself!

And I will not change my opinion just because you are happy with your Kaiser.

Nobody force you to NOT happy with your Kaiser, nobody ...
[Tilt, TyTN II, MDA Vario III General] wim26:
tytn ii original rom wm6
[Tilt, TyTN II, MDA Vario III General] unwired4:
So so far we have an Orange Tytn2 with Duttys v3 scoring 1643, an HTC stock rom at 1374 and an ATT Tilt running Q Mobile scoring 1461 overall.

And gogol doesnt want to help himself or anyone else.

Thanks to those of you who are trying the benchmark tests. With nearly 20 % difference between best and worse, I am not suprised some people are happier than others.
[Tilt, TyTN II, MDA Vario III General] cgcmd:
One example for poor graphic performance is the game skyforce.

The game is totally not playable when u slide the stylus on the screen.
[Tilt, TyTN II, MDA Vario III General] daniel_rh:
Well my kaiser hasn't arrived yet but I did the benchmark in my HTC Touch to compare my results with the results posted in this thread...

-The attachment "vsb2007-1, OC.jpg" and "Result OC.jpg" is with Overclocking (Up to 273mhz).

-The others are with the stock speed (no overclocking: 201mhz)
[Tilt, TyTN II, MDA Vario III General] wim26:
tytn ii

this was mine :(
[Tilt, TyTN II, MDA Vario III General] jgermuga:
My original question remains. How do these numbers correlate to real world use.

For instance, why do some codecs work better then others. Is this benchmark testing different codecs?

What is the difference between D3D, Direct Draw and Raw Frame Buffer?

And I ask these questions because I really don't know. Why should I? The complexity is way higher then the normal consumer is going to even want to understand.

Take a look at this chart.

http://www.autofacts.ca/classics/fast.htm

In particular, see how many cars with more horsepower get beaten by cars with less horsepower.

As the chart header points out, there are a whole slew of factors which affect the results. So what conclusion can we draw from this? I still say none.

So now I have invested in CorePlayer and I do in fact get better video playback performance, but it would have been nice to know I was going to have to spend another $25 to get better performance. Would this test tell me that?

And after all, why does CorePlayer do a better job than Windows Media Player mobile? Again, I have no idea. Would this test tell me that?

If you can't explain to us what these numbers mean, how on earth is someone going to explain it to a jury of non-techies?

So I will reiterate what I said before...

" run the programs and OBSERVE the difference. If there is no little or no difference, this is actually counter to what HTC is saying "More powerful than any mobile communications device you've seen before".

I don't know how this can be made any clearer."

I think what we need to be gathering are videos of stock roms performing the same functions on a set of devices that portray what and average consumer would do. Having all these benchmark numbers is just going to confuse the issue, espcially when the devices and configurations being compared are all over the map.

FWIW, here are my results - Graphics 1299, Others 2007, JPEG 2657, Games 1598, Sound 0558

My config is posted in my profile
[Tilt, TyTN II, MDA Vario III General] Teppic24:
Well, it's piss-poor English, if that helps... :D
[Tilt, TyTN II, MDA Vario III General] squeak:
UK T-Mobile Vario 3

graphics - 1258

others - 1804

jpeg - 2698

games - 1464

sound - 0577

total - 1560

I'm sure my old C64 had better graphics performance :D

Not bad for such a compact device though.

Oh .. Hi everyone! :p
[Tilt, TyTN II, MDA Vario III General] fusionpit:
This is my Titan, running DCDs 2.2.0 Clean:

Graphics - 1233

Others - 1360

JPEG - 2460

Games - 1166

Sound -1105

Final Score - 1588

Sound was a big increase over most of your guys score, but the graphics and games was just a massive hit.
[Tilt, TyTN II, MDA Vario III General] swiftgs:
Re

Why is there such a big difference in sound scores, what means that, I will also run a benchmark tomorrow!
[Tilt, TyTN II, MDA Vario III General] unwired4:
Totally agree with the comments that real world is what matters.

What I am keen to understand from as many tests as possible is the range of results and whether there is any correlation.

It is already interesting that some of you score in the 1300's on these tests and some have scored in the 1600's.

In theory these are the same devices - all runing WM6 - albeit with slightly tweaked roms.

I hope some people can also run tests on Hermes devices.
[Tilt, TyTN II, MDA Vario III General] Radeon123:
My results using an all stock rom Orange UK HTC Tytn 2, also the Orange custom cab during first boot up has never been installed.

Graphics - 1077

Others - 1811

JPEG - 2275

Games - 1238

Sound - 0689

Final score 1418 :(

Ran it twice after a soft reset with no background apps running, same score. The orange rom looks terrible compared to the results above, phone is still snappy though :)
[Tilt, TyTN II, MDA Vario III General] unwired4:
Once there's enough results I will put them in a spreadsheet and publish them in an easier to read format.

If anyone else knows of a good benchmarking test that focusses on graphics then let us know.
[Tilt, TyTN II, MDA Vario III General] getwilde:
I agree with jgermuga and others arguing this point. I'd like to add that before I bought my phone, I researched numerous PPC devices. I was split between a couple of devices, but ultimately chose to buy the Mogul (Titan). Why? One reason was because the specs showed that the Titan had a dedicated ATI graphics chip. To me, this was a very compelling feature. It was only after my purchase, AFTER my 30-day trial period had ended, that I learned that the graphics chip is not even used.

I don't understand what these benchmarks prove. If anything, they only seem to distract from what I consider to be the real issue.

At any rate, here are my benchmark results:

Sprint Mogul, stock ROM:

Graphics: 1270

Others: 1447

JPEG: 2533

Games: 1154

Sound: 0891

FINAL SCORE: 1459
[Tilt, TyTN II, MDA Vario III General] Blowfish64:
I agree. The thing that ultimately had me plunk down my cash for this thing was a comparison of how powerful the chipset was compared to the Dell Axim x51. It was almost as good, or FAR more powerful in most aspects. Now look at the benchmarks, not even close in any area.
[Tilt, TyTN II, MDA Vario III General] undac:
Try this on for laughs, run the VS benchmark while pressing the screen with your stylus.

Performance is reduced to a faulty C64 or so... ;)
[Tilt, TyTN II, MDA Vario III General] DarkDvr:
Pretty much same results, even a bit higher..

Test seemed very smooth, fps was very good as far as I can tell.

AT&T Tilt with Dutty's v3 ROM... oh, and that ati_d3d.dll that was discussed in one of the threads.. it was determined that it makes no difference, but I thought I'd mention it, since the results for me seem to be a bit higher.
[Tilt, TyTN II, MDA Vario III General] ladytoast:
I know it's not applicable to this thread really but I just tested on Dutty's 6.1 beta ROM and saw the following:

Graphics 1301

Others 2288

JPEG 2840

Games 1599

Sound 0764

Total 1758

Touching the screen during the test slows it to about 2 frames a second :(
[Tilt, TyTN II, MDA Vario III General] zyir01:
I have an AT&T Tilt with stock ROM. Ran test after soft reset, no programs running.

Results:

graphics - 1068

others - 1914

jpeg - 2190

games - 1242

sound - 0943

total - 1471

Zyir
[Tilt, TyTN II, MDA Vario III General] gogol:
*sigh*

The problem with performance of playing games in HTC Kaiser will not be solved by showing you that benchmark.

That problem will stay there in the Kaiser at current state.

It is a fact, you knew it.

I dont have to give you proof.

I dont really care with the benchmark. What's the point anyway?

What's the difference between 1300 and 1600?

Do you think +300 will play games better? Take for my problem description in this case.

Why dont you try it yourself with real application, real experience?

Play those games, compare it with your old device like HTC Magician.

You will certainly know what I am talking about.
[Tilt, TyTN II, MDA Vario III General] unwired4:
If you don't care, then take your device back to the retailer and demand your money back. They have to give it back to you...
[Tilt, TyTN II, MDA Vario III General] Boinng:
Gogol's point is perfectly valid, the benchmarks only give part of the story. What do they actually mean in RL? What does a difference of 300 points represent, when you're comparing a 200mhz device with a 400mhz one? Is it the sort of difference we should expect, or is it an over/under achievement for the Kaiser?

Has anyone run this same benchmark on a Hermes? Can we see the difference between the Kaiser, and last year's 400mhz model? That might well be more informative.
[Tilt, TyTN II, MDA Vario III General] gogol:
If that so simple, I would :)

And why on earth should I return my device just because of that 300 benchmark points???

I just explained to you that this benchmark thingie that you are advocating to the death is just pointless.

The benchmark will not give you real experience of the device.

I just feel cheated by HTC, hence described very well in this thread and HTCClassAction.org.

I dont have to explain to you (again) about this (misadvertisement), you can read it in this thread or in the class action website.

I believe you do understand english and know what I am talking about.

I just hope that HTC will fix this problem via update, the problem that they have admitted themself! But in fact, they said that they will not fix it.

So, thats the problem.

The problem is real.

The problem does EXIST.
[Tilt, TyTN II, MDA Vario III General] unwired4:
If you read my posts, you will see that I am not claiming the problem doesn't exist.

However, some users claim their devices are not just lousy at playing graphics rich games, but take "seconds" to simply draw the today screen and are almost unusable.

As my device does not suffer from anything like that, I am interested to see if certain devices are worse than others.

If this goes as far as a class action, then if certain kaisers and roms are significantly better than others, then the class action is at risk as HTC can point to the fastest devices and say the system architecture and/or missing drivers aren't the cause of the users' problems.

So lets see what people are really experiencing. Already you can see that the people who are complaining are scoring 1300's and the people who are happier are scoring 1600's so I think its a valid test.
[Tilt, TyTN II, MDA Vario III General] Boinng:
Sorry, but none of this is relevant if we don't know what 300 points actually means, or if we haven't got a clear picture of where the Kaiser should be appearing in these benchmarks. If 300 points is a vast gulf in performance then fair enough, but if OTOH it's simply a minor margin of error - perhaps caused by one or other apps in the background - then it's meaningless. In either case, the real story is this: how does the Tytn II compare with the Tytn I? Why don't we benchmark it against a Hermes rather than a lowly Wizard? And why don't we get some other comparative figures from more up to date devices?

Saying the Tytn II is anywhere between 3-600 points better than a 200mhz Wizard is, in itself, a completely meaningless result. It's effectively saying no more than that the Kaiser is "better" - well it damn well should be, but the point is, how much better?
[Tilt, TyTN II, MDA Vario III General] efty:
My results

Touch Cruise stock Rom
[Tilt, TyTN II, MDA Vario III General] unwired4:
I have asked a few times if anyone has a hermes handy to run the test.

When the benchmarking was written, they chose to put the Wizard results in as a benchmark.

I agree with the comment re what does 300 represent. But as I said, lets get as man results as possible and then compare them with the qualitative results (ie what people think of their devices).

None of this will cure the drivers issue - but if it gives xda users whos Kaisers are dog slow an option to make their device usable, then is it a bad thing?
[Tilt, TyTN II, MDA Vario III General] sabrexx:
The first thing that my professor taught on the first day of my computer architecture class was that mhz means nothing and synthetic benchmarks are not always representative of real world performance.
[Tilt, TyTN II, MDA Vario III General] Boinng:
I've posted over the Hermes forum to see if anyone wants to share their figures there - http://xdaforums.com/showthread.php?t=366555

It'll be interesting to see the comparison - although I agree with the poster above that these artificial benchmarks aren't necessarily that illustrative.
[Tilt, TyTN II, MDA Vario III General] unwired4:
Nice one.

I agree that none of us can say that the benchmarks mean anything other than one device scores higher than another - when doing that test.

But its better than shouting "my device is rubbish"
[Tilt, TyTN II, MDA Vario III General] Chainfire:
This benchmark doesn't test GDI performance at all, which is what is used when drawing normal Windows applications, so it's both completely useless as well as irrelevant for testing normal operation. 3D isn't tested at all (Direct3D / OpenGL ES) which is the other part which is completely missing on our devices.

It does test DirectDraw and I assume GAPI for the Games part which are (conveniently) less CPU/GPU intensive (in general) and bypass a bunch of layers in the drawing subsystem and communicate directly with the framebuffer. These two will always be the fastest graphics drivers of the group by far when none are hardware accelerated. Still, you can see how the screen flickers on fast animation (comparable effect as 'no v-sync' on a PC - doesn't happen with a decent implementation) which it shouldnt and touching the screen slows the benchmark to a crawl (which shouldn't happen either).

Nice try though.
[Tilt, TyTN II, MDA Vario III General] unwired4:
Is there any point testing 3g... when its missing anyway?

Does anyone know of any test apps that will test GDI?
[Tilt, TyTN II, MDA Vario III General] Chainfire:
A benchmark is never representative if it leaves out parts that are not supported by the device being tested. Those parts of the test should be there and receive 0 points, which will be reflected negatively in the 'end score'. A graphics benchmark only testing half of the graphics subsystem is never representative at all.
[Tilt, TyTN II, MDA Vario III General] Boinng:
There is if we can get comparable results from other devices. Surely that's the point of benchmarking? When the Tytn scores x hundred and the Tytn II scores a big fat zero, that pretty neatly summarises the whole issue here.

Chainfire is absolutely right, a meaningful benchmark should measure every aspect of video performance. Only then are you going to see how the Kaiser really compares - warts (and zero marks) and all.
[Tilt, TyTN II, MDA Vario III General] unwired4:
The only testers I can find for 3d and overall graphics are for ppc2002/3 devices...
[Tilt, TyTN II, MDA Vario III General] Alemaker:
Again,

Benchmarking is irrelevant to the issue at hand. You try to hijack a thread with benchmarking nonsense that makes no difference in the end and which, conversly to your stated position, would actually only do the plaintiff's side any good (if it ever comes to that) by showing that these devices perform no better than older ones, and are not the most powerful devices currently on the market, as claimed.

side note: sort off OT: anyone know why, using PIE to access this site, that I am able to login, but can't reply? When the reply page comes up, I get the window to type in , but it won't allow typing?
[Tilt, TyTN II, MDA Vario III General] XtreMe_G:
here's my result from my hermes

its running 6.1 rom though (shadow 2.0 TE)
[Tilt, TyTN II, MDA Vario III General] unwired4:
1) No hijack intended

2) There is nothing contrary about what I have said here - re-read my posts
[Tilt, TyTN II, MDA Vario III General] unwired4:
Funny how the tests are "irrelavent" and "meaningless" when those happy with Kaisers score 1650ish, a hermes (albeit a quick one) scores 1540 and those unhappy with their devices score 1300ish...

Maybe if some of the unhappy people flashed a cooked rom, their lives would become better - whilst they wait to see if a classaction works or if a developer comes up with drivers anyway.
[Tilt, TyTN II, MDA Vario III General] Alemaker:
not "funny," irrelevant to the issue.

If you care to benchmark your device, feel free, just start a new thread.
[Tilt, TyTN II, MDA Vario III General] unwired4:
How is trying to find a real way to measure the differences between your appallingly slow device and my reasonably fast device irrelevant to the issue?

You don't want to speed it up? Fine. You must be enjoying moaning about it.

In the meantime, I will happily leave the thread so others can post "why haven't we sued HTC yet"