yashsingh1/xdaforums huggingface dataset:
[Galaxy S I9000 Android Development] AdamOutler:
Yeah, we want non-show-stopping notifications... warnings are great.

In my project mythicalLibrarian (matches against an online database to rename a digital TV recording to eg.. Simpsons S10E17(Bart does something).mpg) I was able to make fuzzy logic matches using the "grep" and "agrep" library. mythicalLibrarian is fully aware of it's matching capabilities and assigns a value of -6 to +4 to each match. if the value is less then 1 then it is assigned a "doover" status because obviously information was out-of-date or non-available at the time of the match... too many fuzzy logic checks were made to meet the matching criteria.

I'm thinking something like that against known names may work for this... Kernel should contain a word like "zImage", "Kernel", "linux", and can contain an extension like "bin", "img"... So say you have a name like 30z111-Lin-Img.tar.... this would have a matched value of 3/6 on zimage, 1/6 on kernel, 3/5 on Linux 0/3 on bin, and 0/3 on img... total Name value would be 3/5 suggesting a non-discriptive filename, or unrecognied naming pattern. Extension value would contain 0/0. If 2/3 of each do not mach, pop up a warning saying the name pattern is not recognized for this type of file, or the extension is not recognized...

Working with fuzzy logic can be a PITA. I have 960 revisions on mythicalLibrarian to reach the state it's in now... However if you'd like to pull some of my knowledge, I'd be more then happy to help.

Btw... it's looking like "Heimdall One-Click packager" will support command line options to generate a package.. You say..

heimdall-One-Click.jar /path/to/MyROM.TAR.GZ

and "Heimdall One-Click packager" will deploy "Heimdall One-Click" and then fill it with the Tar.GZ file. So if you'd like to offer a "3rd party packaging" option, then that would be fully possible if you can download a file from "Heimdall-Frontend".. It looks like "Heimdall One-Click" will add 20megs to a generated Tar.GZ file.
[Galaxy S I9000 Android Development] Benjamin Dobell:
I've pushed a few more updates to the Git repo.

Included is an attempted fix for the SGS2 failure to end session issue. This is totally untested and just a guess off the top of my head. If my faithful SGS2 users are able to compile the latest version and test it out for me that would be fantastic.

If these minor changes don't fix the SGS2 issue then I have another idea as well, the other idea is probably more likely to work actually. However I'd like to test the current code out first rather than implementing the other "possible solution" right away. By implementing the two ideas separately if the problem does get resolved then I'll be able to work out which changes actually fixed the issue.

EDIT: In case you're worried about the SGS2 changes causing problems, I'll elaborate slightly. I've just added some device information queries in before performing non-flashing tasks. These queries are performed when flashing but weren't previously being performed when not flashing. Basically I'm trying minimise any differences between a flash and a non-flash.
[Galaxy S I9000 Android Development] jerome.d:
High five, Benjamin, it works flawlessly now: no more error in session ending when printing/downloading pit files, no more wipe of internal memory.

I do not know if you have changed the way pit files are created, but I notice that the pit file I have downloaded now is different from the pit file I downloaded with the previous version (same size, but "diff" says they are different). Note that, since then, I have upgraded my firwmare (but I do not believe this should change the pit files, I did not check "repartition").

One possible improvement, however: when in the "utilities" tab, if one prints or download the pit file, afterwards the phone automatically reboots out of download mode. It would be preferable, I think, that these operations do not lead to a reboot of the phone. Usually, if you want to upload a pit file, this is to create a package and then flash the phone. You do not want, after the download of the pit file, to have to wait for the phone to boot, then switch it off and re-boot into download mode.
[Galaxy S I9000 Android Development] AdamOutler:
Heimdall will never, and should never modify the Partition Information Table. The PIT is used as a mapping of the device. It is likely that you're using a new Partition Information Table on your device.

I tried to verify an MD5 sum with the partition information table on my device.... I could not do it.

Benjamin Dobell,

make[1]: Entering directory `/home/adam/Desktop/heimdall/Heimdall/heimdall'
make[1]: *** No rule to make target `../libpit/libpit-1.3.a', needed by `heimdall'. Stop.
make[1]: Leaving directory `/home/adam/Desktop/heimdall/Heimdall/heimdall'
make: *** [all] Error 2

I can't build libpit.
[Galaxy S I9000 Android Development] jerome.d:
Between my first download of the pit file and the second, I did nothing but flash some kernels and firmwares (using Heimdall). So I do not understand how it can have changed.

Since pit files are binaries (not readable by humans), I am not able to look precisely for what has changed in it. I can only say that "diff" tells me that the two pit files are different (perhaps some slight modification on how they are created by Heimdall will explain the difference).

You have to build libpit before building heimdall: go to the "libpit" directory (it should be at the same level as heimdall/) and "./configure; make". The REAME under Linux explains this, but there is a typo: it says

4. Enter the following commands to compile libpit.

cd libusb-1.0

whereas it should indicate "cd libpit", not "cd libusb-1.0".
[Galaxy S I9000 Android Development] AdamOutler:
^^ That worked...

adam@Adam-Desktop:~/Desktop/RomPackages/I897UCKF1$ heimdall download-pit --output ~/Desktop/part.pit
Heimdall v1.3 (beta), Copyright (c) 2010-2011, Benjamin Dobell, Glass Echidna
http://www.glassechidna.com.au

This software is provided free of charge. Copying and redistribution is
encouraged.

If you appreciate this software and you would like to support future
development please consider donating:
http://www.glassechidna.com.au/donate/

Initialising connection...
Detecting device...
Claiming interface...
Attempt failed. Detaching driver...
Claiming interface again...
Setting up interface...

Beginning session...
Handshaking with Loke...

Downloading device's PIT file...
PIT file download sucessful

Ending session...
Rebooting device...
Re-attaching kernel driver...
adam@Adam-Desktop:~/Desktop/RomPackages/I897UCKF1$ md5sum ~/Desktop/part.pit658dd97b94a66a25523ee2e58edf2432 /home/adam/Desktop/part.pit
adam@Adam-Desktop:~/Desktop/RomPackages/I897UCKF1$ md5sum ./*.pit1d927b36d2fa807a22e64fc86b445130 ./Kepler_odin_new_part_JE3_S1JE4.pit
1d927b36d2fa807a22e64fc86b445130 ./part.pit
adam@Adam-Desktop:~/Desktop/RomPackages/I897UCKF1$ diff ~/Desktop/part.pit ./part.pit
Binary files /home/adam/Desktop/part.pit and ./part.pit differ
adam@Adam-Desktop:~/Desktop/RomPackages/I897UCKF1$ diff -B -b ~/Desktop/part.pit ./part.pit
Binary files /home/adam/Desktop/part.pit and ./part.pit differ
adam@Adam-Desktop:~/Desktop/RomPackages/I897UCKF1$

And that shows a very very different file. the diff -B and -b ignores changes in whitespace and blank lines... Something is wrong here.
[Galaxy S I9000 Android Development] jerome.d:
Glad I could be of use.

The difference might be slight. I am not sure -B and -b work for binary files. I tried "wc" and the results are quite different also:

6 (lines) 17 (words) 4096 (bytes) pit_sgs2.pit
3 (lines) 11 (words) 4096 (bytes) pit_sgs2_bis.pit

The first one is the one I created last week-end, the second one is the one created with the last - and "unbugged" - version of heimdall. Note that I have used the first one for all my flashes, and it seems to work fine.
[Galaxy S I9000 Android Development] AdamOutler:
I opened it in a hex editor....

It looks like they're identical, however it looks like at address 6CF, the original PIT stops and Heimdall retrieved part.pit continues on with a bunch of zeros until it reaches address FFF.

Attached are the two pits.

Kepler_odin_new_part was provided by Samsung.

part.pit was pulled from Heimdall

This may have to do with dumping the entire PIT block out of memory. In which case its still a valid file. Boot.bin will always be one block and part.pit will always be one block. This should cause no adverse affects.
[Galaxy S I9000 Android Development] jerome.d:
A precision: I did not compare a pit from odin or samfirmware with a pit from heimdall. I compared two pits from heimdall, one retreived with the preceding version (the one leading to the "session ending error") and one with the new version (without the "session ending error"). And they are not the same.

However, I used the first one successfully for my upgrade to XXKG2 (new ROM for SGS2).
[Galaxy S I9000 Android Development] AdamOutler:
My test was what went in the phone and what came out. Though different, likely acceptible. As long as it flashes on yours it should flash on all... I must remember to grab my pit for testing later...
[Galaxy S I9000 Android Development] Benjamin Dobell:
In regards to the PIT files, the main difference is that PIT files downloaded from your device are always 4096 bytes where as files downloaded from samfirmware etc. give the appearance that they're less than 4096 bytes, even though they're not.

In the above example you can see that the size on disk is 4096 bytes even though only the first 1744 bytes are editable. So once they're transferred to the device they're just treated as 4096 byte files, which is why they'll be that big when you download them from the device.

Heimdall transfers PIT files to a phone in their entirety (totally unchanged). So any changes that take place between flashing and downloading are the device's doing. Although in my personal experience I haven't seen the device make any changes (other than the 4096 byte thing). PIT files often include a fair bit of junk anyway so I wouldn't stress too much if the file you retrieve from your device doesn't perfectly match the one you flashed.

Also when flashing via Heimdall Frontend a PIT file comparison is made and you will alerted and the flash safely aborted if the local and device PIT files don't match. This comparison isn't simply a raw file comparison, in order to avoid comparing junk data and file lengths. Instead Heimdall just checks that the information contained in the device's PIT and the one selected when flashing match. This is of course done using libpit :)

That should hopefully have been fixed in the latest git update. I happened to notice it myself when fixing a few other typos.
[Galaxy S I9000 Android Development] AdamOutler:
Doesn't size on disk have a minimum size of the block size you use when partitioning? I've noticed alot of files have a large size on disk. So what's the difference between zeros and nothing at all?
[Galaxy S I9000 Android Development] Benjamin Dobell:
Yeah, sorry should have elaborated.

All files have a minimum file size depending on your partition block size, so technically a PIT's on disk file size doesn't need to be 4096 bytes. In some files there would be a difference between all zeros and nothing, however in the case of PIT files it's treated as the same thing. A PIT file's header indicates the length of useful data in the file, anything else is ignored.
[Galaxy S I9000 Android Development] AdamOutler:
Cool. Maybe in 1.4 we could have an exact replica. Not worth adding that feature when youce moved to beta cycle.... that's more of a pre-alpha change.

Btw..... any timeline on 1.3 backend? Im nearing completion of all desired features. I should be working on the packager soon.
[Galaxy S I9000 Android Development] Benjamin Dobell:
It is an exact replica of the file stored on the device, the device stores the file as a 4096 byte file. Heimdall simply asks the device for the file length and then then each part of the file, the device returns a file length of 4096 bytes. There would be no way for me reconstruct the original samfirmware PIT from the file retrieved from the device without knowing the file length of the original PIT.

In saying that, the PIT file the device uses is 4096 bytes in length and not the "original file" from samfirmware. So in theory the PIT file downloaded using Heimdall is more correct than the files distributed by samfirmware.

EDIT: You can expect an official 1.3.0 release (both command line and Heimdall Frontend) in the next couple of days. I'm going to skip the release candidate and go straight to a final release. All I need to do is add the file extension warnings, as discussed earlier, and then build the various binary packages.
[Galaxy S I9000 Android Development] AdamOutler:
What about the "the length of useful data in the file" in the header? If you simply truncated at that point, or a byte or two after, the MD5 sums might be able to match up? Unless Samsung puts an arbitrary number of "00"s after each file.

Great! I'm looking forward to it. All that's really left for me is to tidy up features. Things are all pretty much done with Heimdall One-Click. I've got a "donate to Heimdall" and "donate to Adam Outler" in the about window. I'd hope that people click them both.

Some time after release, I'll be doing an instructional video on packaging a heimdall Tar.GZ, using Heimdall One-Click, and then a long-duration, speed-time, repeated flashing video to demostrate the safety of Heimdall and Heimdall One-Click.

It feels really good to start wrapping things up on this project.. It started off with me thinking that I could write a program to operate a Command Line app... I was thinking it would take something like 2-3 hours. It ended up taking over a month, with new controls and features being added daily because I'm ****ing OCD about any project I take on.
[Galaxy S I9000 Android Development] Benjamin Dobell:
Unfortunately they do :(

The header actually just states how many PIT entries there are (usually 14) from that I'm able to determine the number of useful bytes. However there's a bunch of junk afterwards, sometimes zeros, sometimes random bytes.

Thanks, that's very much appreciated.

Haha, I know how you feel.
[Galaxy S I9000 Android Development] AdamOutler:
Tested one-click UnBrick on a buddy's Samsung Infuse 4g successfully 10 minutes ago. One more to add to your list I think.... I will flash firmware later. Its a non-galaxy phone.
[Galaxy S I9000 Android Development] Benjamin Dobell:
Thanks, I really do need to update the list! There are probably 5 or 6 devices that I've been told Heimdall works for but haven't got around to sorting it out.

I suppose it depends on how the phone is marketed, but I'd personally say the Infuse 4G is a part of the Galaxy S line-up based on the hardware and software running on the device. Same goes for the Galaxy Tabs and phones like the Epic 4G or Droid Charge, although the Droid Charge doesn't seem to work with Heimdall.
[Galaxy S I9000 Android Development] AdamOutler:
About that.... Doesn't HTC own the "Droid" name or something? that would be like HTC putting out a "Galaxy Charge". I kinda wondered if it was based on the HTC system...
[Galaxy S I9000 Android Development] Benjamin Dobell:
Verizon "own" the Droid name, not HTC.
[Galaxy S I9000 Android Development] AdamOutler:
Do you have that code still? This needs to be reported to the Free Software Foundation. http://www.fsf.org/
[Galaxy S I9000 Android Development] Benjamin Dobell:
I've sent you a PM about this. Just to be clear to anyone else reading, I won't be redistributing the code myself. I'm not a lawyer and that sounds like the kind of thing could get me into a legal mess.
[Galaxy S I9000 Android Development] bmxje:
Ive got a problem trying to flash a kernal onto a SGSII in OSX 10.6.

If i give this command:

sudo heimdall flash --kernel zImage

It sais:

new-host-11:desktop tazzie001$ sudo heimdall flash --kernel zImage

Heimdall v1.2.0, Copyright (c) 2010-2011, Benjamin Dobell, Glass Echidna
http://www.glassechidna.com.au

This software is provided free of charge. Copying and redistribution is
encouraged.

If you appreciate this software and you would like to support future
development please consider donating:
http://www.glassechidna.com.au/donate/

Claiming interface... Failed!

Can someone help me? I looked everywhere for a solution!
[Galaxy S I9000 Android Development] Benjamin Dobell:
Could I grab a little information about your phone/device? Is this a GT-I9000 we're talking about? What version of Android is it running?

If I had to guess I'd say your device is probably running a broken bootloader i.e. A secondary bootloader that contains the bug in it's USB stack implementation. This means that the device can't be detected on an OS X machines, even if you run a virtual machine on top of OS X's kernel.

If this is the case then unfortunately the only solution is to upgrade your secondary bootloader using Windows or Linux. You won't be able to use a virtual machine but bootcamp is sufficient.

EDIT: If your device is an Epic 4G then it's possible that Samsung never released a working secondary bootloader, although I'm not 100% certain if that's still the case so it might be worth asking around.
[Galaxy S I9000 Android Development] jerome.d:
Although not the poster, I can be of use: bmjxe said he was trying to flash a SGS2, which is GT-I9100.

Don't know for the rest... I flashed my own GT-I9100 three times using heimdall (two of them to flash just a kernel, once to flash a whole firmware), and aside the problems related above, which are now resolved, I had no issue.

BTW, I am thinking: maybe it comes from a previous version of heimdall, which had some issue re-connecting to the SGS2 (see the internal memory wipe, etc.)?
[Galaxy S I9000 Android Development] AdamOutler:
Whatever happens, do not try to tell heimdall to flash 3 or more times within .2 seconds... it ends badly... I've got a softbrick that one-click unbrick can't unbrick.

any ideas?

[ 339.470449] usb 1-1.5: device not accepting address 10, error -32
[ 339.470690] hub 1-1:1.0: unable to enumerate USB device on port 5

I clicked the "flash" button several times. which i had the "disable" disabled
[Galaxy S I9000 Android Development] jerome.d:
How can you try to flash 3 times within 0.2 seconds? Do you simultaneously launch the process in three different shells??
[Galaxy S I9000 Android Development] AdamOutler:
duplicate post
[Galaxy S I9000 Android Development] AdamOutler:
Yes... 3 separate shells.

[ 339.470449] usb 1-1.5: device not accepting address 10, error -32
[ 339.470690] hub 1-1:1.0: unable to enumerate USB device on port 5

I clicked the "flash" button several times... I had the "disable" disabled for debugging.
[Galaxy S I9000 Android Development] AdamOutler:
Man I'm a dummy....

I was thinking I was looking at the connect/disconnects from my phone.... That's my UART interperater.... damnit.. Alls good now. The USB cable came slighly loose at the same time I launched my 3 clicks of fury..... that's what I've been troubleshooting...

No problems with heimdall..... Just tried again...

vSirete..t.
ing up interface...
CCalilamimiinngg inte niftreacerfc.ae....
Ste.t
nig up interface...
Setting up interface...
Claiming interface...
Setting up intACltreface...

itBmageeipminning session...n
t gafniileteradfce...
. Detaching driver...
Claiming interface again...
Setting up interface...
RSCe-laitiaenmctaittgiihn nt gngre efkrlnau cpde.erii v..
Attempt failed. Detaching driver...
Claiming interface gaer...
nterface...
ain...
Re-attaching kernel driver...

no problems.
[Galaxy S I9000 Android Development] AdamOutler:
Epic beta testing with 1.3

[YOUTUBE]DcknyMZUfhk[/YOUTUBE]

You could call this "heimdall no-click"
[Galaxy S I9000 Android Development] Benjamin Dobell:
I read your post a few times but I didn't quite understand what you were saying, that happens a bit actually. :confused:

Were you running Heimdall Frontend or just Heimdall command line? If you were running Heimdall command line (or your tool) what was the exact command used to flash your device?

Based on the output I can see Heimdall command line wasn't given the --repartition parameter. I can tell that because of:

Well either that or Heimdall command line failed to recognise the --repartition parameter (I haven't seen this before but I'll take a look). Obviously if you have the exact command used that would be immensely helpful.

However, regardless of whether you repartitioned or not your device shouldn't be hard-bricked because you flashed a primary bootloader, secondary bootloader and param.lfs.

What I would recommend doing is comparing the checksums of the files you flashed with the files they're supposed to represent (i.e. those downloaded from samfirmware or wherever they came from). The reason I suggest this is I've read about users having problems flashing dd'd firmware files with Odin (depending on their device). I haven't seen it happen with a GT-I9000 (and I've done it myself) but I think I read about it happening with a US model phone. I don't think it was a Captivate, but it might be worth looking into nonetheless.

Anyway I think it's definitely worth looking at those three files in detail because the flash went through without error (USB or otherwise) so the files were transferred to the device their entirety. Regardless of whether you meant to repartition that action shouldn't have resulted in a hard-brick with valid files being used.
[Galaxy S I9000 Android Development] AdamOutler:
I've saved everything and I'll be doing some data forensics on it. I was using Heimdall One-Click and I can reproduce the fault if it was on my end. I'll report back when I get more information.

The reason I say repartitioning may have been to blame is if the size of the PBL, PIT, or SBL partitions change, they may get chopped off.

btw.. they release a new bootloader with each version of my phone's software. I was running the 2.3 bootloader (which will crash if you feed it a 2.2 or 2.1 rom) and I was trying to flash back to 2.2 (Andromdea3).
[Galaxy S I9000 Android Development] Benjamin Dobell:
I'm not certain but I think that may result in a "Failed to receive end of file transfer packet" error. Nonetheless it's definitely worth looking in to.
[Galaxy S I9000 Android Development] AdamOutler:
repartition was not commanded by me. However, that still does not make the package valid. I must check that.

[heimdall, flash, --pit, /tmp/adamHeimdallOneClick/part.pit, --0, /tmp/adamHeimdallOneClick/boot.bin, --3, /tmp/adamHeimdallOneClick/Sbl.bin, --21, /tmp/adamHeimdallOneClick/param.lfs, --6, /tmp/adamHeimdallOneClick/zImage, --22, /tmp/adamHeimdallOneClick/factoryfs.rfs, --11, /tmp/adamHeimdallOneClick/modem.bin]
[Galaxy S I9000 Android Development] AdamOutler:
This is a good think for everyone to know. You can copy the entire filesystem from the unit, but stay away from PBL and SBL.

I thought I should post this here

Good to know information..... "You cannot dd the PBL/SBL from another phone. It will not work. The boot.bin and sbl.bin are not in the same format as a dd image."

Here I was thinking everything would work fine. Maybe I can come up with a way to test and see if it's a DD image or a bootloader file? At the hex level, everything seems to be pretty much the same between a DD'd file and a bootloader.

It may be that the bootloaders are distributed in an incomplete format so the processor can adapt them to itself? I don't know, it's just an idea. The size is the same, but some areas in the files are different... it would make sense that the processor would verify the files somehow, but I don't understand why they would change from input to storage.

I wonder.... The filesystem can be dd'd properly, and Heimdall "dump" does not copy it properly... Could it be that the Heimdall "dump" command will translate the bootloaders properly?
[Galaxy S I9000 Android Development] GuestK00150:
Hey guys!

Can confirm a working heimdall installation on os x 10.7 lion with the known issues from my old post! Really good job ;)

THANKS