yashsingh1/xdaforums huggingface dataset:
[Touch Diamond, MDA Compact IV ROM Development] HerrVorragend:
ok guys here is what I've found out...

manile.exe has gat a reference to mode9.dll... and that dll contains a class named "LibraryManager"... that class has a funktion called "ConvertPathToLibraryName"... perhaps that function delivers the original filename.

I'll try to write a small app, that calls that function...
[Touch Diamond, MDA Compact IV ROM Development] sushilange:
There is also "mode9InitializeHashPaths" in manila.exe

"1d1ee632_manila" is "HTC\Scripts\orangecalllogtransitions.luac"
[Touch Diamond, MDA Compact IV ROM Development] ryujakk:
Apparently some people extracted PNG's from the QTC files... I dunno about that for sure, but with "direct file editing," as mun_rus put it, I've managed to display the manilla icons on my desktop PC, complete with color and alpha transparency. However, they are smaller than what appears on the Diamond, and I still have 3/4 of the bytes that give me weird stuff (probably what's needed to enlarge the icons to their correct size) So evidently I've got something wrong... but I'm on the right track (16 bit RGB more precisely.) I'll post more details when I get home.

A QTC read and write program is just around the corner, so keep working on this!
[Touch Diamond, MDA Compact IV ROM Development] HerrVorragend:
oh my god... "orangecalllogtransitions.luac" has 29 charaters... no idead how to encode those masses of informations in 8 bytes

is here anyone with detailed assemberl-knowhow?
[Touch Diamond, MDA Compact IV ROM Development] sushilange:
@ryujakk: GREAT!!!
[Touch Diamond, MDA Compact IV ROM Development] sushilange:
You can call me just Sascha :D

It is a normal hash-algorithm, but we have to know how this hash-algorithm is workin.

When you use MD5, etc. you also get only a fixed number of characters, not matter how long the string was before.
[Touch Diamond, MDA Compact IV ROM Development] HerrVorragend:
I know that... but a hash-algorithm is one way only "filename -> to hash"... so when it is a hash-algorithm you can't get out the filename from the hash
[Touch Diamond, MDA Compact IV ROM Development] ryujakk:
But it is a hash nonetheless ;) There is absolutely no way you can get the complete original filename out of an 8 byte code! For the simple reason that there are many more possible filenames (infinite actually...) than possible hash codes (exactly 4294967296 to be precise :p) To find the correct QTC to display, the Diamond calculates the filename's hash code, and then opens the correct xxxxxxxx_manilla file. No need to get the filename out of the hash code, it's a one way operation. Now we just need to find out exactly WHICH operation it is!
[Touch Diamond, MDA Compact IV ROM Development] HerrVorragend:
Ah... OK... that make sence! ;-)

So please correct me if I'm wrong... we actualy know the filenames but we do not know how to get the xxxxx_mailna from that. do we know all the filenames? If yes... were can I find them?

btw. it is not impossible to compress those filenames to 8 bytes... if written a compression-algo for one of our products which has an comparable compression-rate...
[Touch Diamond, MDA Compact IV ROM Development] sushilange:
@HerrVorragend:

Here are some:

<Manila Mode9Path="HTC\manila.mode9">
<PreloadImage Path="HTC\Assets\Images\Common\dottedline.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\transparent.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\alpha-hitbox.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\down_arrow.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\up_arrow.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\selector_side_glow.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\selector.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\selector_side.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\selector_glow.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\black.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\white.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\Line_Horizonal2x2.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\TitleBar.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\Transparent_Vertical2x64.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\TitleBarLevel1.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\TitleBarwhite.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\TitleBarLevel0.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\TitleBarLevel2.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\GridSingle.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\grad_top.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\ThumbnailList_Frame.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\Grid.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\ImageMask.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\grad_bottom.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\Background.qtc" />
<PreloadImage Path="HTC\Assets\Images\Common\VGA\largeframe.qtc" />
<PreloadImage Path="HTC\Assets\Images\Menu\menu-background.qtc" />
<PreloadImage Path="HTC\Assets\Images\Menu\line-seperator.qtc" />
<PreloadImage Path="HTC\Assets\Images\Menu\left-side--hadow.qtc" />
<PreloadImage Path="HTC\Assets\Images\Menu\right-side-shadow.qtc" />
<PreloadImage Path="HTC\Assets\Images\Menu\darken-gradient---middle.qtc" />
<PreloadImage Path="HTC\Assets\Images\Menu\menu-arrow.qtc" />
<PreloadImage Path="HTC\Assets\Images\Menu\darken-gradient---bottom.qtc" />
<PreloadImage Path="HTC\Assets\Images\Menu\darken-gradient---top.qtc" />
<PreloadImage Path="HTC\Assets\Images\Menu\selector_side_top.qtc" />
<PreloadImage Path="HTC\Assets\Images\Menu\button-top.qtc" />
[Touch Diamond, MDA Compact IV ROM Development] ryujakk:
OK, this is where I'm at.

I have series of 8 images.

Obviously, the four black and white images on the left are alpha masks. The sixth image is pretty self obvious. As for the remaining 3, I don't know how to integrate them... But they must have information in them to be used somehow!

As for the byte code:

the first 4 bytes: QTC1 -> signature

bytes 9 and 10: image length (or height, forgot which)

bytes 13 and 14: image height (or width, forgot which)

byte 17: is at 0x14 for "normal icons", and at 0x15 when there are two images in an icon.

bytes 18 to 32: probably image metadata

bytes 33 on: image data
[Touch Diamond, MDA Compact IV ROM Development] mun_rus:
Wow! Great reasearch! Thanx!
[Touch Diamond, MDA Compact IV ROM Development] ykat:
i agree.

this is how these systems works.

one more thing i read something earlier about hash-tables for file naming..

i am still searching for that.

I think this hash table and algorithm is also responsible for not useing two filenames at the same time (duplicated filenames)
[Touch Diamond, MDA Compact IV ROM Development] cmonex:
mun_rus asked me to look at mode9InitializeHashPaths

I can analyze the disasm no problem, can rewrite it in C++ if anyone wants.

but I haven't finished the complete analysis yet, so I can't yet see why a hash function would need to

- call these (some of these undocumented) APIs: SetKMode, SetProcPermissions (with arg 0xFFFFFFFF, to allow access to any process), GetProcessIndexFromID, KernelLibIoControl,

- read kernel structs such as process array and the module list, then the import table from two specific modules (but only if the process calling this export loaded these modules) looking for coredll.dll?

weird thing.
[Touch Diamond, MDA Compact IV ROM Development] ykat:
the hash again

look at the picture: isn't it familiar?

windows does something similar...

manila works with hex range, windows with full alphabetics

it is absolutly not the same, just a flash..
[Touch Diamond, MDA Compact IV ROM Development] AssimilatorX:
Hello i just start doing some research around so i find the internet tab i quiet useful we could use as program launcher....

something like that

<?xml version="1.0" encoding="utf-16"?>

<InternetPortal>

<IncludeUserFavorites>false</IncludeUserFavorites>

<OperatorLinks>

<Link

Text="Resco Explorer"

Executable="\Program Files\Resco Explorer\explorer.exe" />

<Link

Text="Resco Explorer"

Executable="\Program Files\Resco Explorer\explorer.exe" />

<Link

Text="Resco Explorer"

Executable="\Program Files\Resco Explorer\explorer.exe" />

<Link

Text="Resco Explorer"

Executable="\Program Files\Resco Explorer\explorer.exe" />

<Link

Text="Resco Explorer"

Executable="\Program Files\Resco Explorer\explorer.exe" />

<Link

Text="Resco Explorer"

Executable="\Program Files\Resco Explorer\explorer.exe" />

<Link

Text="Resco Explorer"

Executable="\Program Files\Resco Explorer\explorer.exe" />

<Link

Text="Resco Explorer"

Executable="\Program Files\Resco Explorer\explorer.exe" />

</OperatorLinks>

</InternetPortal>
[Touch Diamond, MDA Compact IV ROM Development] D-MAN666:
Hello, guys!

I've done some research, and here's what i found:

1. QTC files

They're non-compressed bitmaps, but encoding used there is completely retarded. :)

As ryujakk *almost* correctly stated, header is:

0x0000: 51 54 43 31 01(?) - magic bytes signature

0x0008: xx xx xx xx - UINT32 image width

0x000c: xx xx xx xx - UINT32 image height

0x0010: .... some metadata goes here, maybe encoding type too, dunno yet

0x0020: image data

0x......: last ~500 bytes are padded with zeros for some reason

Here's the most interesting part - image data. First of all, the image is chunked. Each 16-byte chunk refers to 4x4 image piece, which are built from left-top corner (i.e. last chunk refers to bottom-right corner of image).

Next, chunk format. I've found only one encoding format so far, and there may be more, so keep it in mind.

0x0000: xx xx xx xx xx xx xx xx - transparency data. 1-byte/pixel. But you may ask: hey! there're 16 pixels per chunk! Wtf? Sorry guys, QTC image is actually halved by width and restored via interpolation during decoding.

Bytes are drawn in this order:

[i2][0][1]

[i2][2][3]

[i2][4][5]

[i2][6][7]

"i" means interpolated pixel, "i2" means pixel, interpolated between 2 chunks

also, FYI, tflo3d wrap's image around during edge interpolation

0x0008: xx xx - color non-uniformity info. see next. also, dunno how it works yet

0x000a: BB RG - key chunk color in RGB format (8 bits for blue, and 4+4 bits for red+green). Yes, pixels in chunk have shared color, modified by previous 2 and next 4 bytes

0x000c: 21 43 65 87 - ligthness values for each drawn pixel (number corresponds to pixel #), 4bits each

Here's it but it's not that simple - image is interpolated and additionaly anti-aliased, so results are pretty unpredicatable and i don't even think about PNG/JPG -> QTC yet

Also, i wrote a small Flash (sorry guys, i'm flash coder) QTC Viewer - it doesn't read colors/ununiformity properly, but it may be useful for exploring manila. It's attached, just put it next to "image.qtc" file and run (flash player 9+ required). Of course, i will write full-blown standalone converter (using Air) after i understand what's going on there.

Ok, that's all for QTC :)

Update: see updated viewer below
[Touch Diamond, MDA Compact IV ROM Development] D-MAN666:
Ok, now regarding other things:

2. LUA

Lua scripts in TFLO3D - are *normal* LUA 5.1 scripts, but in Unicode format and without Unicode signature, that is. If you put it in (0xFFFE), you will see a normal LUA bytecode format. But it won't help for decompilers - none of them do support unicode. So we'll have to write our own decompiler (or disASM - for the first time). Thankfully, LUA bytecode is pretty simple and straightforward.

3. Filenames

Yah, hash means "one way function" and we can't restore filenames, but we can get FILENAME->HASH table, using QTCViewer and looking into XMLs, than find come cryptoanalyst among us (basically, somebody who's working on keygens :)) and ask him to find a hash function from the table.

The only problem here is path format, coz in XMLs it's HTC\Blablabla\Blablabla2, but in GLES'es - .\Blablabla\Blablabla2. Who knows what is fed to Filename2Hash function :(

4. GLES

No idea yet. :(

So, summarizing all above, TF3D is a *completely* customisable and extendable platform - a paradise for developers :)
[Touch Diamond, MDA Compact IV ROM Development] D-MAN666:
Okay, it's me again :)

Here's an update on things:

1. LUA

Only header & strings are Unicode - the rest is plain binary.

2. QTC

I've cracked down this QTC thingie :)

Here's an updated info on chunk structure:

0x0008: xx xx - Key color 1. Color is calculated by adding values from 4 LUTs (lookup tables) for each 4 bits. Each LUT holds RGB values

0x000a: xx xx - Key color 2. Same as key color 1, but different LUTs

0x000c: 21 43 65 87 - mixdown coefficients for each drawn pixel (number corresponds to pixel #), 4bits each. see next

As i wrote earlier, pixels in chunk have shared master colors. The resulting is calulated as:

color = mixdown * key_color_2 + (1 - mixdown) * key_color_1

Note 1. Mixdown value is normalized to 1

Note 2. Mixdown operation s/b done for each color channel distinctly

Please also check an updated viewer - it should work flawlessly (it doesn't do 100% the same interpolation, though). Please report about any errors
[Touch Diamond, MDA Compact IV ROM Development] ryujakk:
Wow, great work! I don't know how you came up with that "magic" formula, but I can't wait to get back home to test your flash reader! I was really stuck, as I had no clue where to go since my last post. Now all you need to do is write a complete two way converter :)
[Touch Diamond, MDA Compact IV ROM Development] Ximoon:
Great job indeed !

So anyone can try now to write some third party LUA script to replace existing ones, or is there still some limitations ?
[Touch Diamond, MDA Compact IV ROM Development] D-MAN666:
Well, via some trial and error i've found a "Messages" icon filename, then i was puttin' in some test patterns looking for regularities. :)

Re converter: the format is lossy as hell (only 2 master colors per 4x4 chunk, 16bit master color resolution and halved horizontal resolution), so PNG/JPG -> QTC is not such a simple thingie. And the hardest thing is reverse LUT colors lookup. But if someone has an idea how to do that - please PM me, and i will give out LUTs to you. ;)
[Touch Diamond, MDA Compact IV ROM Development] Ximoon:
Same number of zeros for all files ? Wouldn't it be to match some regular size (ex : multiples of 512 bytes, 1ko, etc) ?
[Touch Diamond, MDA Compact IV ROM Development] D-MAN666:
Yeah, that was my first thought, but, no, padding size is variable and filesize is not always regular. (btw, for some reason, qtc files in manila are grouped into series of files by size - 512 bytes, 1536 bytes, 66048 bytes, etc. and padding size is variable among them)

Moreover, i've just removed padding and file seems to be working ok :) so we don't have to worry about it.
[Touch Diamond, MDA Compact IV ROM Development] bepe:
Not sure if this could help somehow...

This is one qtc and the original image.
[Touch Diamond, MDA Compact IV ROM Development] idsk:
did someone try to put non-compressed stuff?

i mean adding a new tab by linking directly to a .lua and/or a .mode9 that themselves links to png

because we can see between the very first unofficial version of manila and the one in the diamond, the m9 are in plain text and the mode9 are in a kind of semi plain text, and inside regular properties are replaced with few binary bytes, the same for lua, and if you compare the compressed/uncompressed we see it's exactly the same contents

i know the filenames "algorithm" is another problem,

{ but when you look in disassembly, the names are calculated with many simple mathematic operations between each characters values (like 'h' + 't' x 'c' / '/' etc...) it's not long calcul and maybe i think someone can easily reverse this calcul

}

but did someone tried to rename a xxxx_manila file to its original filename to see if it's still loaded?

these are few question i thaught about, but don't have my diamond now and i wanted to try that..

i think you will understand what i mean only by viewing the 2 linked files (1c684cd8_manila and Home.m9 are the same files, the first is compressed, 103Kb, and the other is uncompressed, 267Kb)
[Touch Diamond, MDA Compact IV ROM Development] D-MAN666:
Well, they're not compressed, they're compiled :) But thanks a lot man - it should help much! :) BTW, can you please upload other ones (or the unofficial manila completely)?

Re path algo - we don't need to reverse it. We need to know it. Do you? ;)
[Touch Diamond, MDA Compact IV ROM Development] D-MAN666:
A small update:

1. Mode9.

Thanks to idsk, i was able to parse .mode9 structure. It's an exactly a binary representation of .m9 XML format. There's a bit of retardness, though, again :)

The format of file is the following:

ROOT_TAG

END_TAG

The format of root and nested tags is following:

0x0000: xx xx xx xx - tag type (UI32)

0x0004: xx xx xx xx - tag header (to be researched)

0x0008: xx xx xx xx - data length (UI32)

0x000c: ............... data (actual data or nested nodes) goes here

Parsing tool to be posted soon.

2. Unofficial Manila SDK is on it's way, but we badly need:

- WM coder to decompile mode9.dll ConvertPathToLibraryName function.

- Win32 coder to write tools like PNG->QTC, QTC->PNG, Mode9 (de)compiler, etc. coz if i do em on flash, they will be hell slow (flash is kinda slow with bitmap per-pixel and binary per-byte operations).

3. QTC padding.

My bad and you're right, Ximoon :) QTC files are padded up to multiples of 512. For faster loading, i reckon.

4. 1.93ROM Manila. Home.mode9 file is missing but it works. But when i copy it over (from older version), home tab stops showing up (so, it's still referenced from somewhere). I wonder how come :)
[Touch Diamond, MDA Compact IV ROM Development] idsk:
i was also on the parsing of mode9 files!

but wow D-MAN666 you got a fast brain

i'm interrested in your algo of the qtc files, and it would be great a qtc viewer/converter with a gui

and better, a manila "cooking tool" with an embedded xml editor, qtc converter and cabber/updater functionnality,

with your searches it should be named d-manila hehe

now ConvertPathToLibraryName have to be understood
[Touch Diamond, MDA Compact IV ROM Development] sushilange:
D-MAN666 and I will make a Win32-Application for viewing the manila-files (and maybe in the future for editing them).

Also I will add the possibility to create a theme-file for my Diamond TF3D Config ;-)
[Touch Diamond, MDA Compact IV ROM Development] MonteCristoffOn:
Great news sushi !

And if you need some help (design, graphism, ergonomy, ...), don't hesitate to contact me.

;)
[Touch Diamond, MDA Compact IV ROM Development] D-MAN666:
sushilange, idsk - PMed you

regarding .mode9 - here's a preliminary listing for some component - maybe someone will find it interesting to dig :)
[Touch Diamond, MDA Compact IV ROM Development] holyfox:
Excuse me, is there anyone can tell how to make transparent glass clock?

like this (it not manila but spb)
[Touch Diamond, MDA Compact IV ROM Development] D-MAN666:
Ok, here's the mode9 parser. :) A couple of datatypes is unknown yet and we need LUA (de)compiler to work on scripts, but that's a giant leap ahead anyway.

Have fun!

holyfox, you should edit corresponding .qtc files
[Touch Diamond, MDA Compact IV ROM Development] nushuth:
i found this:

3ca01154_manila

seems to be a configuration file for manila color & appearance.....
[Touch Diamond, MDA Compact IV ROM Development] D-MAN666:
All right,

looks like we have a decent viewer for QTCs. Now we need to make a png2qtc converter.

The task is kinda complicated and it will be greatly appreciated if someone could help us with some tasks i will post eventually.

Task 1. Looking for someone with good math knowledge (or with MathCAD installed ;)) to solve this thingie:

- Find an x, with which a following expression has a minimum value (of F):

F(x) = 30 * (r - R)^2 + 59 * (g - G)^2 + 11 * (b - B)^2

where

r = x * r1 + (1 - x) * r2

g = x * g1 + (1 - x) * g2

r = x * g1 + (1 - x) * g2

Basically, we need to find a function MinX(R, G, B, r1, g1, b1, r2, g2, b2) which value will turn F(x) into minimum value, and 0 <= MinX <= 1

Thanks in advance,

Dmitry
[Touch Diamond, MDA Compact IV ROM Development] sandeman684:
I guess in the last equation all r and g should be replaced by b.

In that case, according to Maple, the solution of dF(x)/dx = 0 is:

x = (-30*r2*r1+30*r2^2+30*R*r1-30*R*r2-59*g2*g1+

59*g2^2+59*G*g1-59*G*g2-11*b2*b1+11*b2^2+

11*B*b1-11*B*b2)/

(30*r1^2-60*r2*r1+30*r2^2+59*g1^2-118*g2*g1+

59*g2^2+11*b1^2-22*b2*b1+11*b2^2)
[Touch Diamond, MDA Compact IV ROM Development] D-MAN666:
You're absolutely right! :)

Thanks a lot, i will give it a try.
[Touch Diamond, MDA Compact IV ROM Development] D-MAN666:
An update on .qtc format:

In fact, there's no interpolation there. All 16 pixels are stored in chunk data, with 4 bits for alpha and 2 bits for mixdown. :)

Here's a couple of updated .QTCs (produced with the new knowledge) for anyone interested to test right in their Diamonds.

So, QTC converter is almost done, but before we build theme editor we need your help here.

Have fun! :)
[Touch Diamond, MDA Compact IV ROM Development] Smartstradamus:
tested your edited people qtc's and they work fine on my 193 rom,nice and crisp images.

Look forward to the finished results and cant wait to get to edit them.
[Touch Diamond, MDA Compact IV ROM Development] herg62123:
it may not help out with editing manila but i am trying to help.

while i was digging in ship rom for att touch pro i found this cab file for manila.

it may help it may not. as far as i can see it is a reg file but might have info to help out.

if it doesn't than sorry for a lame post.

Reg Cab File Found Here
[Touch Diamond, MDA Compact IV ROM Development] cookie1977uk:
Looks like the pro manilla support landscape , now i like that lots
[Touch Diamond, MDA Compact IV ROM Development] l3v5y:
In landscape, it's just a launcher.
[Touch Diamond, MDA Compact IV ROM Development] Riel:
How does the home-screen work?

It's not just an image, since orange has other functionality: move the clock, put alarm next to it, instead of a 'static' htc-clock.

Can there a script be written for home page, or is it flash or java or something like that?

Since you may not want to edit the clockimages, but you can write your own homepage functionality ..
[Touch Diamond, MDA Compact IV ROM Development] l3v5y:
Awesome!

So, you can now rewrite and compile scripts?
[Touch Diamond, MDA Compact IV ROM Development] Ximoon:
Great work. I guess I now have to learn LUA. Or maybe it would be possible to have tabs in any language ? :D
[Touch Diamond, MDA Compact IV ROM Development] riri:
TF3D Internet tab

Hello,

Somebody knows he it or it is necessary to modify to change the list of the favourites link on the internet tab of the TF3D
[Touch Diamond, MDA Compact IV ROM Development] D-MAN666:
l3v5y, theoretically - yes, coz i know the format at whole. But implementing a decent compiler, etc. takes much time.

Riel, it's very similar to Adobe Flash. MovieClips, gfx objects, place on scene, animations and scripted language - very alike, but with 3d support and LUA language (not AS).
[Touch Diamond, MDA Compact IV ROM Development] AssimilatorX:
HTC will fall

I just cant understand wy people want pink manila, but i guess after the secret been solved so we can editing the manila interface HTC will release an official editor or devpack for it because they lost the coding war.... :)
[Touch Diamond, MDA Compact IV ROM Development] D-MAN666:
I seriously doubt that. Look at Sony, for example, who never released any official SDKs, though PSX, PS2 and PSP coding is blooming :)

Also, i don't think there was any "war" at all. They put no protection on the TF3D data, just converted everything into neat performance-effective binary format and didn't release specs. Therefore, i don't think they "lost some fight", moreover, i believe they are our "allies", coz they created a nice extendable platform with great possibilities ;)
[Touch Diamond, MDA Compact IV ROM Development] AF241:
lol. Sounds like you are enjoying decoding this manila thing. Good. Just hoping we get a chance to make our own manila theme soon.

All the best mate