yashsingh1/xdaforums huggingface dataset:
[XDA-University] ab-samy:
Quote:

Originally Posted by ab-samy

Dear Red Devil!

Thanks for your great tutorial for repo, i read it it's very useful.

i setting up my laptop (ubuntu 10.04 64bit installed) to compile android 4.2.2_r1.2 from source, i am confused what will be the actual size of this branch because i am syncing more than 2 day with speed of 1MB connection and download 13.5 GB.

Sorry for poor english.

THanks

Answer by Red Devil:

Hi,

If you had read my guide a bit earlier maybe it would have helped you a bit.

You can remove unecessary stuff from the sync like Darwin toolchains etc.

Now continue the sync. It will be around 15-20 GB.

To know how far you have synced count the number of projects.. like it will show

Projects fetched : 220/250 for eg:.

Once 250/250 is done it will sync work tree and you will get youre source.. Post your doubts on thread.. People with same question will get answers.. nothing to be ashamed of

Cheers.

---------- Post added at 12:10 PM ---------- Previous post was at 12:02 PM ----------

:D i must download 13GB more.

Fetching projects: 51% (168/328) :mad:
[XDA-University] percy_g2:
Great guide :angel:
[XDA-University] Red Devil:
That is not the way ... the size of the projects may be much smaller or larger ... I cant say how large.

There is no way to find out exactly how big the source will be as android/CyanogenMod/AOKP/PA etc keep adding/deleting stuff from their manifest and also make additions/ deletions to the repos defined in the manifest !

So what I suggest is to use local_manifests and try to remove unwanted projects .. read post 2 #1.. :)
[XDA-University] <JohnCarter>:
congo [MENTION=4918174]Red Devil[/MENTION] for the xda-portal feature.. ;)

btw i wanted to know that i had an old compressed cm9 source which wasn't downloaded through git,, so will reference command help me download the newer android version without having me to download the whole source again ?
[XDA-University] Red Devil:
Yes why not. :)

Just see some of the common projects and check if they are getting redownloaded .. during the sync ..

If they are still getting redownloaded you can copy paste the downloaded projects from your source to the newly initialized .repo/projects/
[XDA-University] Shaaan:
This was needed. Good work.

OT

Can compile from source. Needs to learn how to draw a straight line in paint.

/OT
[XDA-University] <CyberAnarchy>:
Thanks for the awesome guide man.

BTW i have a problem.

When i use the local manifests, the number of projetcs is unchanged.

Please help this n00b out!

im compiling aokp aswell so the local_manifest is same as yours but i cut out the part where you put in the code to add your device tree and kernel

EDIT:Solved
[XDA-University] rkpeterson:
Great guide, much appreciated!
[XDA-University] assassin0408:
still sync all the project even i made local manifest with remove-project.. where exactly the local_manifest.xml file should be put?
[XDA-University] Red Devil:
mentioned in the first post.. with pictures. !
[XDA-University] assassin0408:
follow exactly the tuto..but still all repo is synchronizing now.. :crying:
[XDA-University] Red Devil:
did you stop and restart your sync ?

There are 2 different ways of placing the manifest ... check them out
[XDA-University] assassin0408:
nope.. didn't stop it.. ok i will check it.. and i more thing.. i want to try building aokp for s advance which is not in official devices list for aokp. i have device tree, kernel tree and hardware sources(which is not fully working as ste novathor didnt release the sources code) for s advance.. any suggestions or tips for me? :p
[XDA-University] Red Devil:
well you can ask that in youre device specific forums. :)

I can't possibly know about each and every device ! :(

So see if these general tips help you others you can ask Developers of Janice . :)
[XDA-University] freakboy13:
Excelent guide as the comments express

Not just the guide itself but how it was written. You should write more tuts.

I have suggested rc status for you.

Hope ya get it.

:thumbup:

:beer:

sent from my T.A.R.T.I.S

(Time And Relative Tarts In Space)
[XDA-University] assassin0408:
done sync all repo..but the sources not appear in directory! i can confirm it downloaded GB's of file..
[XDA-University] Red Devil:
Probably you're sync isn't done yet. :(

Wait for the whole work tree to get sync-ed.
[XDA-University] assassin0408:
so what should i do to continue sync?or should i start again???demm! :(

this is after repo sync is 'done'.. but i have moved those folder to other place becouse i want to sync other repo..
[XDA-University] Red Devil:
The sync will automatically stop after the all the downloading is done and the folders will automatically appear. the folders that appear are called the work-tree.

Be a bit more patient and wait for the sync to get over :).

I am not sure from you're picture what exactly you have done.
[XDA-University] assassin0408:
The work tree should be outside the .repo folder right?in .repo folder all projects list are there but the sources are not in working folder.. :(
[XDA-University] Red Devil:
Yes sir,

Delete the folders outside the .repo. . and continue sync. Your sync isn't done yet :(
[XDA-University] assassin0408:
to continue sync just type repo sync? or repo sync -f?
[XDA-University] Red Devil:
Just the normal repo sync. Patience is virtue. :)
[XDA-University] ab-samy:
Dear [MENTION=4918174]Red Devil[/MENTION]

I want to compile aosp 4.2.2_r1.2 for Samsung i9100 can i delete other device from manifest.xml because i download over 27GB still syncing not completed?
[XDA-University] Red Devil:
well that's what the guide is all about ! to remove stuff from manifest using local manifest !
[XDA-University] Galaxo60:
[MENTION=4918174]Red Devil[/MENTION]

I have spent the last hour reading all the guide and trying to understand all the stuff you wrote. I just want to say is it really useful, but still have some questions if you won't mind to answer:

1) Which one would be the repository link from building JB for Nexus 4?. I read the other article, but only found PAC-Man, CM and BeerGang. I made a search on Google and found it could be this one, but I'm not quite sure:

repo init -u https://android.googlesource.com/platform/manifest

2) As far as I understand, the way to go is downloading all, and then remove projects of devices we don't want. But still my questions is those projects would be deleted or just ignored. I'm a little concerned about space, so the question would be if they got deleted from hard disk when we remove them on the local manifest, or do we have to deleted manually, or it is not a good idea to delete anything.

Thanks so much for the great guide anyway, I just want to start with some Android Development and maybe with some luck, make my own custom rom for my phone.

:victory:
[XDA-University] Red Devil:
Hey, glad you liked it. :)

1) it is correct but you didn't specify any branch. 4.2.2 r1.2 is the latest .

2) local manifest doesnt download removed projects at all. That's the point of the local Manifests !!

I have mentioned a few stuff you can remove ... Read post 2 #1 ... Be careful though.
[XDA-University] Galaxo60:
Does all this guide works with Oracle JDK 1.7?. First, when trying to to make the build, warned me about the java version. I found a workaround for that editing the main.mk file, but still getting errors. It is recommended to just use the java 1.6 or can we achieve building with java 1.7?.

I'm building aosp, but it seems that building CM10.1 is easier. What would you recommend for a totally noob in this terms?.

Thanks

:victory:
[XDA-University] Red Devil:
Hi , :D

Though the repo tool has nothing to do with Java, Java 6 is most preferred. I don't know what guide you are following to build AOSP , but This clearly mentions you need only Java 6 :)

To be honest , CM is much easier to build than AOSP, just cause its very well maintained IMO and the device trees are already ready-to-build ( in most cases ) as Nightlies are kept on rolling out on a regular basis. In case there is an error during the CM build process you can mention it on the threads /pm / email ( yes in that order ;) ) the developers who are maintaining CM for your phone ! :)

I personally started off with CM. I would recommend the same :)
[XDA-University] mssngr:
Hi, I hope this isn't a stupid question. I have compiled a few working CM 10.1 builds which I then successfully patched with PDroid. I just followed the guides and don't fully understand how it all works. I found your guide a very nice read!

But one thing I just don't seem to understand is how to start a specific build like for example CM 10.1 RC5, CM10.1 M3 or the new CM 10.1.2. I think it has to do with the manifest but I just don't get how it should work. Any advice would be much appreciated.
[XDA-University] IronFistYo:
HElp needed.!

Hey.

I want to download CM-9.1.0 sources to compile for HTC Explorer.

Here i the default.xml

Here is my version of local_manifest.xml

I think i need to add the remote.

I see in the default.xml file there is remote added like this-

<remote name="github"
fetch=".." />

Do i need to copy the same line to my local_manifest or something needs to be changed.

Correct me if something is wrong.

Can i use this as is or some more modifications need to be made to this.

I yes then what modifications do i need to make.

Thanks in advance
[XDA-University] Red Devil:
Yes you need to add something like

<remote name="github"
fetch="www.github.com/" />

and remove

remote="aosp" revision="refs/tags/android-4.0.4_r2.1" />

just a tip : use pastie.org / pastebin.com for pasties .. makes life for the reader somewhat simpler :)
[XDA-University] IronFistYo:
Will this work..?

Thanks for the reply .
[XDA-University] Red Devil:
hey you forgot to add remote github to lines of the extra projects you want to fetch . Read the guide a bit more properly.

<project path=" directory in work tree" name="name of the repository in remote" remote="name of remote defined earlier" revision="branch

of the repository" />

<remote="aosp" revision="refs/tags/android-4.0.4_r2.1" />

is not needed really if you are not fetching anything from AOSP.

Check out post 2 #1 for more stuff you can remove from your manifest.
[XDA-University] CNexus:
[MENTION=4918174]Red Devil[/MENTION]

The -jX parameter in repo sync (and other repo commands) is not solely based on your internet connection. It also depends on how many cores you have on your machine. If you have very few or only 1 core, the machine will be unable to handle running parallel threads for say, 32 jobs even if the internet connection is fast. This goes for other commands as well where you can pass a -jX parameter. It should agree with the specs of your machine or it will run even slower than if you were to do something like -j4.

Sent from my buttered S3
[XDA-University] Red Devil:
As that part of the post refers to

to the local manifest and it depends on your internet connection and not on the number of cores in your PC.

I havent mentioned anything regarding the number of threads to be used in the build process in this guide.

The repo sync number of threads can go upto 32 / 64 on how well your isp manages the simeltaneous downloads of the file .

Only when you put in the -jX in the build command do the cores matter !

Correct me if I am wrong , or have I misunderstood you ?

I personally have tried to use -j32 on a particularly old pc and makes difference without blacking out / overheating my laptop during the repo sync and it manages it quite well .
[XDA-University] IronFistYo:
Check This

So i think finally this will work.

Thanks for the awesome thread and helping and quick reply.:good:

PS: I tried this one but it didn't work.

It was giving fatal error about the remote github attributes.

So i removed the remote from local manifest.

Now it's syncing perfectly.

Thanks for the helps anyways
[XDA-University] hazedabeast:
This thread is the sickest thanks ....it helped me a lot!!
[XDA-University] Red Devil:
Thanks ! Queries are welcome. Refer it to friends too .. For those who might need it. ! :D
[XDA-University] UtkarshGupta:
i have 2 repos that i sync.

1 - domination

2 - cm

now intially i only had domination. (in ~/android)

then i added cm with reference flag to domination. (in ~/cm10)

now domination team decided to work with pacman team so domination repo is of no use anymore.

but i still want cm repo but i wanna delete domination repo.

but if i do that, i will have to sync all the local objects again or all the files which were previously obtained from domination repo.

is there any way to keep only cm repo and be able to delete domination repo.

i know i can wipe out all the folders from android (except .repo) but then files from other repos will still be left behind but my motive is to free up as much space as possible.
[XDA-University] Red Devil:
What you can do is manually copy paste the projects that are missing from your CM's repo from your Domination's repos.

Which repos ? You will have to manually search through : the sync log.

repo --trace sync -c -d -jX

might help you track what is missing. Tough luck though.
[XDA-University] Red Devil:
Just a tiny bump
[XDA-University] thewisenerd:
[Q]Getting patches using repo???

As I write this, sharing your agony (slow internet speed (~50 kBps)), I was wondering if there was something which would help me patch the existing, already-downloaded source code without having to download the whole project again...

Example:

1. I downloaded source.

2. Someone committed changes to repo, say 5-6 commits.

3. I do repo sync again, it is naturally gonna re-download the project. I don't want the whole project to be downloaded again. I want something that automatically downloads the *.patch files and patches my source, so that I can keep it updated without using much of bandwidth.

Is this even possible??? :o

p.s. cherry picking, yeah, I read, but doing it for each and every commit? there ought to be another simple way...
[XDA-University] Nihar.G:
It doesn't re-download the whole project.
[XDA-University] thewisenerd:
But then... how would it sync the tree to master (if it doesn't download the whole project again)???
[XDA-University] Red Devil:
It will take a reference of the last commit that is present in your local source and then download the commits that newly have been committed... that's how git works !
[XDA-University] thewisenerd:
So, only the first repo sync is a headache (i.e. requires a lot of time)! :highfive: :laugh:

You just made my day! ;) :p

P.S. Was waiting for CM sources to get stable with the fear of frameworks_base being changed every few hours. And, it takes a 2 GB download :cyclops:
[XDA-University] Red Devil:
Bump..
[XDA-University] AGONTUK:
Great guide bro, very useful for beginners :good:

Here'e my problem. I started to sync a repo then in halfway I found out that it was outdated. So I stopped syncing, created a new folder and began syncing from a new source. I used --reference from the previous folder since it has downloaded half of the folders.

My sync completed succesfully and I tried syncing again, it gave me no errors. Then I deleted the referenced folder (half synced folder) since it's useless. Now when I start sync in my new source folder, it's giving me all kinds of errors :(

is it becasue I deleted the referenced folder? What should I do ?
[XDA-University] Red Devil:
Thanks man.. really appreciate your comments. :) :svetius:

Actually it was because you deleted the reference folder it is giving you errors. It symlinks the referenced source to the current downloaded directory to which you are freshly repo syncing. I found out I have mentioned it copies it to the new folder. It was my bad. Ill correct that right away. I didn't notice it till you mentioned it now.

Thanks.

What you can do now is to reference whatever you have now and start another fresh repo sync. That must do the trick.

BTW.. if you ever get a problem with the repo tool getting updated again or what not.. just delete the ~/bin folder and redownload the repo tool.. It wont make any difference.
[XDA-University] AGONTUK:
I tried but it also gives errors. Maybe because it checks for reference in my current source folder and gets redirected to the deleted source :(

I guess I have to start fresh again. So I have to keep my original source (that I downloaded fully) no matter what ?

Another question, you mentioned to set CCACHE value to 50~100GB. But I have created 40GB drive for ubuntu. Is this gonna be a problem ?
[XDA-University] Red Devil:
You can still scavange the source for some downloaded repos and manually copy paste them while syncing it the latest time. Check the .repo/projects.

CCACHE can be set to any value. To make a significant difference it can be set to the recommended value. Else too it makes the compilation speed a bit faster. You can try setting it to a value you like below 50 GB too.

One more thing. Try to allot more space to Ubuntu .. 40 GB may or may not be enough for the whole source + out folder.. after compilation. Its better you have at least around 75-100GBs.. If I were you.. I'd just bomb everything.. Just take a backup of all the downloaded packages of linux .. put it on an external Hard Drive.. start again by installing linux.. and then allot around 15-20 GB for linux.. create another partition of around 100 GB and then install all packages.. and repo sync from the new partition. EXT4 is the best partition type.
[XDA-University] AGONTUK:
OK, sync is complete. I still need some basic info about building....

1) I ran some commands like vendor/cm/get-prebuilts, . build/envsetup.sh, make -j4 otatools etc. There are lots of messages showing up in the terminal and I noticed some warnings like "not found".

How can I be sure that this initial processes are completed successfully?

2) Say I modified files, created new ones. Now I want to revert back to the original source that I downloaded. Can it be done by repo sync ?

3) I'm just following steps blindly from a guide (like you mentioned in OP :cool:). I want to learn what is actually happening in the background, which files get called. Is there any guide about the source files/folders you can point me to ?

4) Also I want to learn how my device is configured for build. I know that different .mk files are behind this but a guide would be great.

Thanks in advance :)
[XDA-University] Red Devil:
Well if you mean if ensuring the sync is done.. ? then just repo sync again.. it will check if all the stuff is downloaded and if everything is up to date.

If you have called your own repos and pushed those commits to your own repos / made changes in the stuff not declared in the manifest .. then the repo sync command shouldnt pop up an issue. Else it will ask you to commit these changes.

Do a

git reset --hard

in the local repositories where you have made the changes ...

Well, if we could, then we would write a guide on everything... But you and I know that it is not practically possible. There may be guides I may not know of but Some stuff you will have to learn on your own experimenting and going through code. Some basics of Programming languages like python , c , c++ and java are a must. So , let The Internet be your guide.
[XDA-University] AGONTUK:
What I meant is that this setup like envsetup.sh, do they actually change any files or just build the environment ? I'm more confused about make -j4 otatools cause when I run it, it shows so many verbose output that I really can't see if there is any error occurred.

So that means if I only manually edit files, then I have to do this. Those environment setup doesn't do anything to the files ?

I am trying bro. But just a info about which folders are for what purpose, which folders I will actually make changes to, which should be left untouched etc would be nice :). If you ever come across any post/thread, pls tell me.
[XDA-University] Red Devil:
envsetup just sets up which makefiles have to be called before the build starts. Ok , heres the deal. As long as the build continues.. let it keep building. If something breaks during the build.. the build will terminate.. and fix that.. After that.. see if the build is as you want it to be. If it is not then start debugging. You can use the "tee" command to capture the output of the terminal. and according to the fixes you need.. you can check the warnings and corresponding fixes.

envsetup.sh is a bash script. Open it up and see what it does. It maybe slightly different depending upon the ROM. As I said it will just call.

As far as changes are concerned. You musts be limited only to the changes only in your device tree.. i.e. android/device/oem/codename/ folder..

If you want to port a ROM to your device i.e.. If you own a legacy device , you may need to track your own device tree / and device specific repos that are needed to make the ROM boot.

If you want to make a new ROM altogether.. then yes you will have to modify a lot of stuff.
[XDA-University] Red Devil:
Le Bump :cyclops:
[XDA-University] jawz101:
no mention of using the groups switch?

http://xdaforums.com/showthread.php?t=2496326

we should ALL be using more use of group tags in the manifest. In Jean-Baptiste's (you know, Google's own lead project manager) example he trimmed down to from 8.5GB to 3.6GB with some simple group switches to tailor for his build environment and target

I wish you would make a video. I have the attention span of a gnat :P

Thank you for writing this all out, though. I'm still trying to figure out how these projects come together. Maybe the groups thing will help you as well. I'll keep this thread subscribed so I can document anything I come up with.
[XDA-University] Red Devil:
Thanks for the kind words.I know about the specific groups that can be downloaded during the sync.. I was thinking why people didn't ask about it yet !! my aim of the guide was to help noobs start off in the right direction, and learn on how the tool can be used for various projects. I thought of adding it sometime ago. But from the relatively low feedback, I guess no one really has the patience to read even this much. Adding stuff to will reduce the readers even more :p
[XDA-University] vishal_android freak:
If you add that stuff then it will be cherry on the cake bro.. :D we are here to help.. If people are not giving any response to this then they are at loss as they are missing a big deal.. Well, till u struggle you don't learn.. :p

Regards,

vishal_android freak
[XDA-University] jawz101:
I have a few questions.. do I need anything that says emulator or eclipse to actually build the rom? I'd assume these are only needed if you plan on developing (eclipse) or test from your local machine (emulators)?

Wouldn't you be able to use eclipse & android-sdk on their own if you have it on your computer independently?
[XDA-University] vishal_android freak:
Emulator is used for testing the roms you compiled.. Eclipse for apps and other debugging stuff.. Once you compiled successfully, chances are less that rom wont boot.. If you are not sure then you can you emulator to test your rom.. :)

Regards,

vishal_android freak
[XDA-University] roshan2989:
hi there i already build my homemade cm10.2, so i would like know how to change the name in about phone...?for example like as you tap the about phone you will see "cyanogenmod Version" how to change that name to my rom name...?
[XDA-University] vishal_android freak:
The name under cyanogenmod version is by default created on compile time.. Later on, after the flashable zip is created, open build.prop and there you will see "ro.display.buildid" and "ro.cmversion" or similar lines or search for "cm-build_date-UNOFFICIAL-device_name.zip" replace it with the name you want to.. :)

Regards,

vishal_android freak
[XDA-University] akash3656:
Lovely guide! I wish I found this when I started compiling android a year ago. Could have saved me tons of time and bandwidth and headache.

I found this thread from here: http://docs.omnirom.org/Setting_Up_A_Compile_Environment

[MENTION=4918174]Red Devil[/MENTION] I really appreciate your guide. I wouldn't mind if you wrote more to include this "group" feature.