yashsingh1/xdaforums huggingface dataset:
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
246xx Results

Alright, here's the scoop on the latest build. It's certainly not the best, but far from the worst. It's an overall average build. I like that the directory opening is very smooth. It feels as though an animated start menu like WinStart would be wonderful here. It's another one of those "you have to see it to believe it" things. If you give it a shot, you'll notice the fluidity of the opening that would just suit animated app opening.

All that being said, this is going to be a short review. I REALLY liked 24614. It pretty much had the best of everything. It even scored pretty well on the overall results. Everything was lightning quick. If it wouldn't have been for the default scroll buffer, I'm fairly confident the start menu would have scrolled on forever.

The only really bad one was the very next build, 24618. Ugh, when I say it was bad, I mean it. It took three seconds to load the start menu, four seconds to load each benchmark application. I was surprised by the results in the media tests, though. They weren't good, but they weren't the worst, either.

I don't know; it's not a bad branch, but it's not entirely good either. It's kind of another "fly-under-the-radar" branch. There isn't anything ultra-spectacular, aside from 24614, but nothing extraordinarily bad either. Most of the results were very consistent for the most part in this branch. The numbers didn't lie about "feel" for once.

The bottom line:

24630 - statistically the best

24618 - statistically the worst

24614 - my opinion the best

24618 - my opinion the worst

Is anyone interested in the blank ROMs I'm using? I can upload them if some of you want to do this testing on your own but aren't sure how to cook. Let it be known that these aren't builds to be used as everyday ROMs. There are NO EXT packages cooked into the ROMs. They are completely clean of everything except the SYS packages and the OEM packages. This means no Sense, no applications other than things like File Explorer, WMP, IE, and other Windows standard stuff. Most of the standard settings like USBtoPC and backlight control are gone as well. Keep all of this in mind before asking. These are ONLY for testing purposes.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
23148, 250xx, 282xx, and 290xx Results

I don't have a lot of builds in these last couple of branches, so don't expect much here.

23148 was crap. It's one of the worst I've tested and results back it up. Simple as that.

250xx was not bad. It started off slow, as most branches do. Overall, I think it's under-rated. It's moderately fast and extremely stable. Great branch for aspiring chefs to jump off from.

282xx followed the same pattern as 250xx. It was stable overall, with nothing outstanding. It's a solid branch with few, if any hiccups. I like this one. The results were all pretty close together, but 28236 stole the show easily.

29002 is a very nice start to a promising new branch. I only with I knew where this branch was coming from.

There will be no bottom line results for these branches. There aren't really enough to do so. 250xx was good overall and was as close as the results indicate. 282xx was pretty much the same. My opinions followed the results pretty closely.

CUSTOM ROM TESTING

I will now begin testing custom ROMs. Follow the rules in post #5 on requests for testing. I have permission from OndraSter, Jackos, dotcompt, seeM_ZA, and NRGZ28 to test their ROMs. If you don't see your chef's name in that list, then have them PM me with a go ahead before you post any results. If you have a request for a ROM you want tested, post a request here. Please be as specific as possible about which ROM it is as some ROMs have different flavors, build numbers, and dates to consider. Link to the file location when possible unless it is prohibited for some reason. I will post exactly how I will test the custom ROMs at some point this weekend. I'm glad I finally got all the builds out of the way for now, and will be going to get some sleep. :)
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] farsa:
Thank you very much cajunflavoredbob ,this thread is really useful.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] wmg316:
I would like to see results for: Energy.RHODIUM.28244.Sense2.5.Cookie.2.0.Nov.13

I would also like to see results for: Energy.RHODIUM.21684.Sense2.5.Cookie.2.0.Nov.08
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] Vico100:
Could you do the "The bottom line" thing for the 282xx branch, thnxs :D

Also should newer build date mean better? If I build 28236 compared to 28244 would there be less bugs and things missing?
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
I'm outlining the guide to how I'm going to test the custom ROMs. There's a bit more detail to it seeing as most users will look at it as the "end-all-be-all" and flash one of the top ones. I can tell you with certainty, however, that 28244 is going to score much better than 21684. I used both of them and did a quick bench while I was using them. 28244 is definitely faster. The newest one, 23148, is one of the slowest in a long time, though.

I decided not to do a bottom line for the last three lines because there weren't enough builds or variances for me to consider. My bottom line picks were consistent with the results. I liked 28236 best and 28244 would be the worst. It's not that simple, though, as I liked the entire branch. 28244 was statistically the worst, but really they were all very good. 282xx is a very fast and stable branch. You can't really go wrong with any of them. 28236 scored fourth best overall in the masterlist. Try both of them and see which ones you like better. I'll even bench them both if you want.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
Custom ROM testing

Ok, so I'm going to need some input on how you guys feel I should be doing this as custom ROMs are not universal in terms of what they offer. Some have Titanium, some have TF3d, some have Sense 2.1, and most use Sense 2.5. There needs to be a fair way to test them. I'm going to be using the same tools essentially, but I'm trying to think of anything that I should be doing differently, like swiping between tabs in Sense before testing, or something similar to show the true degradation of the RAM and CPU, or if I should just install and run the apps right away without touching anything else as I did on the blank builds. Any thoughts?

I'd appreciate any thoughts or suggestions you guys might have at this point. I will certainly try to have the first round of requested ROMs done by Thanksgiving.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
I'm working on the first batch of custom ROMs. I'm doing the CHT2.0 versions of NRGZ28's Energy ROMs. I will not be finished before bed tonight, but check back tomorrow evening if you haven't eaten yourself into a coma. (Tomorrow is national eat-too-much day in America.) I'm doing the 21916 versions right now. The 6.5.3 versions will be done tomorrow evening if I have time/am still conscious. I like to avoid retail stores on black Friday, so I will probably spend the day finishing these. Sometimes I think the biggest reason men get together with women is so they can do our shopping. :D

EDIT: I also decided to do the testing EXACTLY like I did the blank ones, by opening the start menu as soon as possible upon the ROM booting. This means, I'm not going to slide through tabs or anything like that seeing as there isn't a fair way to do so.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] -Jonny-:
I'm looking forward to some of these results. I think it's fair not going through the tabs - as some ROMs will have different tabs available, and we all like to add/remove our own anyway, and will have to put up with the effect the tabs we like has on the ROM overall anyhow. (I think you'd be moving into the 'testing apps' territory if you went down this path.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
You're probably right. I was thinking about it originally because of the way Sense likes to cache every little thing you do. Sliding through tabs drops the free RAM by a considerable amount. It would be more realistic to do so, but not practical for this testing.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] monomer888:
CFB, when you summarised the 236xx builds, you talked about CPU management. I've been meaning to ask... how are you defining CPU management? :)
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
Well, basically, I don't have any way to really define it. You can tell that when using it, the horsepower isn't really there while opening things, or scrolling. It didn't feel like the branch was just having slow scrolling issues or anything. It was more jumpy. Even during the tests, there was stutter in the OpenGL test, as if it were trying to do multiple things at once. It was just very poor performance. I figured it was most likely poor CPU management since it wasn't build specific. The entire branch had this problem, meaning that the branch core was not really up to the task of multitasking. I hope that clears it up.

23149 is almost done testing. Look for it in the spreadsheets in about ten minutes. It's fairly average, so don't get your hopes up. That's all I'm going to be doing until tonight or tomorrow. Gotta spend time with the family today. :)

EDIT: Results are posted.

Happy Thanksgiving!

-Or just happy Fat Americans day to those of you in other parts. :D
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] monomer888:
Ah, thanks for clarifying!

Happy Thanksgiving. :D
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] Vico100:
Can you if possible test more 235xx builds (6.5.5)
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
I'll test them if you point me to where they are.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] Vico100:
http://joojoobee666.ppckitchen.org/Kitchens/

23551

23554

23557

23559

23563

23566

23568

23569

(I have these, but they are last year builds, i can upload those if needed)

23541

23529

23518
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
If you want to package those up for me, I'll try to get to it tomorrow. As they are, I can't import them into my kitchen. I use osKitchen. I normally use MSXIPKernel packages. Those are the ones with Common, LangDB, et cetera folders. I've used builds from that repository before, but it takes too long to get them together to be able to use for it to be worthwhile for me. In the meantime, I'm off to bed finally. :)
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] Vico100:
Whats your resolution?

Edit: found WVGA
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
WVGA - 800x480
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
New results will be delayed

I'm sorry to have to report this. I spent most of the morning fooling with my PC, but I don't have a spare power supply for this box, so I won't be able to continue testing until I get the new PS in the mail. I'm having a cheapo express shipped while the good one is backordered. I won't set a time frame on when the results will be up, but for those using NRG's ROMs, the 21683 build is my favorite so far. It's very fast and stable and is my recommendation right now. If you use the 21916, then the 13 November one was my favorite.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
Custom ROM Results: Part 1

I finally took care of the NRGZ28 Energy ROM testing. Let me know which chef you'd like me to do next!

I also benched 21685. If you are a chef, please do your users a favor and cook this build into your ROM! It has the speed of 28244 with none of the bugs. Me gusta mucho.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] wmg316:
Well...NRG tweeted: "Well I just tried 21695 for the last 24 hours and what piece of steaming pile that is... back to 23XXX..."

I assume he was talking about 21685??

Looks like us Energy users may be out of luck on that one...unless you can talk him into it! ;)
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
I'm afraid I don't have much sway as to what any of the chefs cook. I really liked 21685, but perhaps there was something that he saw that I did not. Keep in mind that I'm not testing every aspect of the build, too. I'm playing with it in my ROM today, and I haven't seen any issues so far.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] BigWopHH:
NRG has the typical software developer's mentality. He quickly tests a build with all of his tweaks, etc., encounters an issue, and immediately it must be a problem with the build and not worth researching. It can't possibly be something he did--must be the build! :)

Meanwhile an objective source such as yourself, that actually tests the builds using a semi-sceintific method, finds a cetain build to be superior but the masses who don't possess the knowledge to cook thier own roms are going to miss out. Oh well.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
Actually, this time, there actually seems to be merit to his testing. In chef central, a bunch of the chefs are complaining about this one, saying that a lot of the problem is related to contacts and ActiveSync. This isn't anything I would test out, so perhaps they're on to something.

I didn't see any of the little glitches that I did in 28244 like the screen jumping or anything like that. 21685 was just as fast as 28244, but I did not notice the problems that the chefs were having because I did not use those features. Remember, I'm only testing the builds for speed and video quality. :)
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] wmg316:
Well...I haven't flashed anything since NRG's 28244; it has been bug free for me...and the fastest I've had on my device.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
Indeed. I liked that one too. I'm not sure what the issues were on that one. It was hit and miss with other chefs too. I cooked it into my ROM, and I saw no real problems. A lot of chefs reported similar random issues like NRG noted. I saw no problems with 21685 either, but several chefs are reporting problems with contacts, ActiveSync, and Outlook. I haven't cooked it to test it, but the results for the build are only for the applications, video, and file opening.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] crazyC:
regarding 21685

Immediately I cooked and flashed this I had issues with contacts not syncing from the exchange server properly. AndrewSh did then advise to use an older pimstore.dll which sorted that, but I don't think it liked my dialer and hung after a call.

As with most things, I guess it depends on your setup and extra EXTs.

I have fallen back to 21683 which I have run for weeks with minimal issues.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
Good feedback. I wish more people would chime in and post things like this. It's nice to hear opinions of how the build actually fairs in real use.

I only test the builds using the benchmarks and whatever means I have to in order to install and use the programs. I don't go through each one and test every feature. I think some people mistake that for what I'm doing. I have NO EXT PACKAGES in the builds I test. This allows for a semi-unbiased view of the build.

Does anyone have a wish list of ROMs they want tested? I've got dinner plans tomorrow, but I'm free all day Sunday (except during the Saints game) and in the afternoon on Saturday if you guys have recommendations.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] OndraSter:
Do mine, as there are just two and they wont be updated anymore :).

Thanks
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
I'll get right on it tomorrow morning! Thanks!
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
29003 and LBFAR ROMs

29003 just came out. If you like 29002, then this is pretty similar. I flashed them back-to-back to get a feel for it. 29003 is a bit faster with scrolling, but no noticeable improvements otherwise.

I tested Ondrsster's LBFAR ROMs. I did the 23569 build first. I noticed it changed my splash screen. I personally don't like this as I flash quite often, but I suppose it's about preference there. The initial start up was extremely fast. I ran into my first problem right here, though. There is a customization screen that appears as soon as it boots for the first time to, I assume, run cabs or settings. I was stuck on this screen with the progress bar unmoved for about 10 minutes before I gave up and soft reset. I got the same problem this time as well. Thinking it might have been a bad flash, I did task29 and flashed again. Same problem. I soft reset one more time, and this time the progress bar began to move. It installed whatever it was doing and soft reset automatically. Upon this reset, Windows picked up the device as a USB CDMA controller and installed the driver for it. I'm not sure why, as this was supposed to be a GSM ROM. Even the chef uses T-Mobile to my knowledge.

I then went into the start menu and file explorer to install the benchmark cabs. This ROM is pretty fast, but could use an update. It's still running TF3D, which isn't bad, and this cuts down on its memory footprint. I ran the SPB Benchmark test which it did reasonably well on, but it did not have the mod to allow for unsigned programs to run. I was not able to do the OpenGL test on this run. I don't have the unsigned program cab anymore, and didn't feel like tracking it down.

Overall, the ROM is very fast, but very outdated. The lack of support for unsigned programs puts it at a huge disadvantage when compared to other ROMs. It would be nice to see this ROM revived and caught up to speed, but as it is now, is a disappointment as far as the speed and flexibility go. Because of the problem with OpenGL, I will not add this one to the list as it isn't complete. I might do this one again at a later date, but I'm pressed for time right now.

I'm working on LBFAR 21899 right now and will update in a bit. So far this one is far less troublesome. It started up normally and ran the customization thing again, but worked on the first try. After a soft reset, I went about my usual routine. Directories are a bit slow to open. This could stand an update as well, but not as much as 23569. I was pleased with the start menu and speed of scrolling. It was a very fast build overall, however. If you like speed, then this is a good one to use provided you prefer TF3D to Sense of course.

Results posted. The 23569 version is incomplete as I accidentally erased the coreplayer benchmarks. I'll try to redo them tomorrow.

EDIT: Sorry, I forgot to update this. The 23569 results are now complete for LBFAR ROM.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] farsa:
Any benchmarks for build 23150 and 23151?
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
Working on all the new builds tonight and tomorrow. Sorry I'm behind. It's been a very busy week for me. I've got three new builds to test, I believe.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
First off, I will no longer post the spreadsheets separately from the zip files. It's no big deal, the zip files are smaller than the spreadsheets separately. When I have an update to this thread, I will list the new spreadsheet on that update post, but not on the second post. Hopefully this makes sense. Basically, I'm going to post the spreadsheets for the updates on this post today, but only post the zipped files on the second post of the thread. Simple enough, right?

Okay, today I've got the results for 21686, 23150, 23151, and 29005. None of them were out of the ordinary in my opinion, but 29005 was certainly a big improvement over 29003.

21686 was very quick with scrolling. I didn't have to apply much pressure for it to go scooting along.

23151 was my favorite out of the four I tested today. Directories opened quickly. There was little to no lag in opening apps, and the scrolling was much smoother than 23150.

Again, guys, sorry for the delay in posting here. It's been a busy week, and I expect the coming week to be even worse with the holidays upon us. I'll do my best to stay on top of things, but I'm not promising to have the builds done during the week for a while. I can say for sure that I can do them on the weekends, though. Hopefully this will be temporary.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] wmg316:
CFB...have you flashed NRG's 29005? It seems pretty quick...and your stats show it to be a bit better than 28244?? (percentage wise). I was just curious what your thoughts were comparing the two.

I flashed 29005 two days ago just because I hadn't flashed anything since 28244 and it seems pretty quick. I plan on flashing back to 28244 tonight to compare...but as I said...just wondering what your opinion was.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
I'll try to test them out when I get home tonight. I really should have done the benching when I actually flashed them earlier this week.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] wmg316:
Nevermind...that is if you don't want to go through the trouble. I flashed back to 28244...and IMO...there is no comparison!! 28244 feels so much snappier and smoother. 29005 was fast; but 28244 beats it hands down on my device.

EDIT: Of course this is just "feel"...no benchmarks done.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
I'll do my best to get them done this weekend. It's been a busy few weeks at work lately. I haven't been finding much time for XDA. 28244 is one of the fastest current builds I've tested so far.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] Cosmic Ranger:
Relative merits of builds

Thanks for the testing. It is interesting to see how the builds measure up. As was stated though, the chefs are looking at a broader range of performance indicators (perhaps subjective vs. objective).

Which leads to my question. I have searched for a thread discussing in detail the relative merits of the builds in the eyes of the chefs but can't find. Does anyone know of such a thread?
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
You could poke about in the Chef Central forum. Chef's look at benchmarks as well as other things such as the overall feel of the build. I still cook 28244 into my personal ROM because it's fast, stable, and I don't need to overclock.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
Thread poll

I have added a poll to the thread. Please take it if you can. I am thinking of moving the thread to Chef Central if enough people are using this with other devices.

I will likely begin redoing some builds test results soon. I will upload the newer build results tomorrow, hopefully.

I'm sorry for being absent for so long. Various circumstances including the holidays have kept me away from the site. I should be around more as the holidays draw to a close.
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] empds:
Hi there,

what about custom ROMs?

I read somewhere in this thread that you had permission from several chefs to test their ROMs and I would like to see results of my favorite ROM which is deepshining from dotcompt (touchstone flavour).

Is this possible?

Thnaks in advance
[Touch Pro2, Tilt 2 Windows Mobile ROM Development] cajunflavoredbob:
I know, I know. I feel bad because I haven't been able to get out even the new blank SYS builds. I am about to begin flashing those, though, so it will get them out of the way. The custom ROMs will begin again shortly after. I'll get on DeepShining ASAP, too. I'm not going to set a time-frame, because I've still got some jobs to finish before New Year's, and I've also got some personal issues to sort out. I'm trying to find time for XDA, but benchmarking is going to be slow until the holidays are over.