yashsingh1/xdaforums huggingface dataset:
[Magisk] kantjer:
The fact I revered a few commits doesn't mean I do not trust [MENTION=4470081]topjohnwu[/MENTION] coding skills, on the contrary I have the utmost respect for his work.

Some commits can, as is to be expected during development, cause unexpected side effects, some I can live with others not so.

If it's an easy revert I will do that until the issue is fixed in the source.

I started building Magisk for personal use only, a few people following my Git picked them up and started posting links.

Everyone is free to use my builds but they were never meant as official nightlies to test upcoming changes.

Although I understand your request for unchanged builds I'm not going to do that.

Having multiple builders and then some having multiple builds, the thread would be filled with "which build should I use" questions.

EDIT: Thanks [MENTION=7187684]ianmacd[/MENTION], I think we are saying the exact same thing.
[Magisk] kantjer:
One issue fixed, tested and MM start delay is history :good:
[Magisk] edzamber:
Working great on S9 Oreo BRE5 custom Rom based :)

Do you have an idea What is breaking Titanium backup finding Su binary ?
[Magisk] martyfender:
Is that in your build today 180625?
[Magisk] abacate123:
Maybe not, this was commited 10 hours ago, and latest snapshot was built 16 hours ago (obviously, counting at the moment I posted this).

Also, nothing on the changelog (just in case you didn't know there are changelogs), unless it's not written there.
[Magisk] martyfender:
What lead me to think that was the fact that he said he tested it, more than likely not released to his build channel yet:

"One issue fixed, tested and MM start delay is history"
[Magisk] ianmacd:
Do you understand why these changes aren't automatically picked up by the app submodule in the Magisk repo?

The developer's MagiskManager repo explains "This repo is no longer an independent component. It is a submodule of the Magisk Project." But from GitHub's point of view, it still seems separate, because changes made to MagiskManager don't seem to be automatically picked up by the app submodule in the Magisk repo. Either I don't understand why these are still separate entities, or they are not, and I simply don't understand why it appears to me that they are.
[Magisk] kantjer:
I noticed after the dev started using submodules that the reference in the Magisk Project sometimes was updated days after the commit was pushed.

For example curently the MagiskManager repo is 3 commits ahead of the Magisk Project reference for the app.

To overcome this instead of the submodules I'm using a manifest that pules all repo's so I have control over everything myself.
[Magisk] ianmacd:
Interesting. Thank you for this information. This isn't the first time that git submodules have confused me.

I have since found the way to remedy this in git:

$ git submodule update --remote app

According to the man page:

What I still don't grasp is the usefulness of the distinction, or therefore the confusion that its existence can cause. Perhaps it's intended as yet another way, one that applies only to submodules, of staging commits before pushing them. In this case, the first stage push is to the submodule, and a second push action is required to actually sync the submodule with the superproject.

Even more confusing is that even after forcing an update of the submodule in this way, git log app still doesn't show the changes. The only way I've found to see them is:

$ git submodule summary app

Unfortunately, that lists only the change descriptions, not the commit hashes.
[Magisk] kantjer:
When working on the changelog stuff recently I tried doing it using submodules hoping that would make it easier but instead it made it much more complex.

I now do this to generate a raw 30 day commit list, compare with the previous commit list, write newly added commits to the changelog and strip off the hashes.

cd ${MAGISK_PATH}
repo forall -c 'git log --oneline --since=30.days' >> commit_list_raw.txt
sed -i '/[0-9]\{6\} build/d' commit_list_raw.txt
grep -F -v -x -f ${Commit_list} $Commit_list_raw > ${Changelog}
sed -i 's/^.\{,8\}//' ${Changelog}

then I use that cleaned changelog to write the changelog.md inside the MM app

cp $Changelog ${ROOT_PATH}/tmp.md
sed -i 's/^/- /' ${ROOT_PATH}/tmp.md
sed -i "1s/^/### ${buildType} (${verCode})\n\n/" ${ROOT_PATH}/tmp.md
mv ${ROOT_PATH}/tmp.md ${MAGISK_PATH}/app/src/full/res/raw/changelog.md

This is the current version of my build script
[Magisk] ianmacd:
The submodules really do make it harder.

Who'd have thought that it would be this tricky to build, whilst including all of the changes from the submodules, and a clear change log that also covers the submodules? :)

The relevant part of my build script looks like this:

# Get hash of the last build that was pushed to the world.
#
prev_build=$(git rev-parse --short origin)

changelog=app/src/full/res/raw/changelog.md
date=$(date -u +%Y%m%d)
head_commit=$(git rev-parse --short HEAD)

# Last tagged release, e.g. 16.4
tag=$(git tag --sort=v:refname | tail -n 1)

# Increment the minor release version, e.g. 16.4 > 16.5
unreleased=${tag%.*}.$((${tag#*.} + 1))

echo '###' $date $unreleased-$head_commit > $changelog

{ for i in $(git log --format='%h' | sed "/^$prev_build/Q"); do
git --no-pager log -1 --format='- %s' $i |
grep -v 'Merge remote-tracking branch' &&
git log -1 -p --submodule $i | sed -n '/^Submodule app/,$p'
done } | sed -e '/^Submodule app.*$/d' -re 's/^\s+>/-/' \
>> $changelog

Yuck!

This assembles and formats all of the commits since the last time I pushed to GitHub (i.e. all changes since the last build). Each commit is checked to see if it was a commit to the app submodule. If so, the submodule's log for that change is also fetched and formatted. Any commits arising merely from my merges of the upstream source are also filtered out.

The result is a changelog file that looks like this:

### 20180626 v16.5-fa3cf33
- Revert "Revert "Get device blocks with sysfs""
- Update MagiskManager submodule.
- More fixes for non-root devices
- Speed up startup time
- Fix non-root boot patching
- fix a bug when $ABILONG is arm64-v8a
- Update BusyBox

There's probably an easier way to do it. git is one of those programs that, after more than a decade of use, it still feels as if I'm only scratching the surface of it.
[Magisk] topjohnwu:
That is how the way git submodules works, unfortunately.

The main git repo will only record a specific commit of a submodule, think of it like a snapshot. So if you use "git submodule XXX" commands, it will fetch all submodules, and checkout to that specific commit.

In order to update a submodule in the main Magisk repo, I would have to manually add the updated submodule and commit in the main repo in order to have the new commits show up.

I develop Magisk and Magisk Manager separately, and I usually don't pull the Magisk Manager up-to-date in Magisk since the development of Magisk Manager moves much faster than the native Magisk part.

I will update it in the main repo when it is necessary (since Magisk Manager depends on scripts in Magisk), or I'm ready for a public release :)
[Magisk] ianmacd:
Thanks for the explanation. I just have to learn and remember all the ways in which submodules behave differently from their superproject, and in particular how one ensures that one always has the latest code for the superproject and the submodules alike.
[Magisk] kantjer:
The last Magisk commits pushed after my 180626 build make a simple "Remove invincible mode" git revert impossible.

Because of this I will stop reverting the commit. Continuing would mean having to change the code beyond the level I feel comfortable with.

So for those who suffered from root loss you might want to safe keep the 180626 build until we know the root loss issue if fixed.
[Magisk] martyfender:
For some reason, todays build is failing when it attempts to flash the boot.img, using the direct install method in MM. I first updated the MM before trying the direct install method. Anyone have any idea why this happened? I tried twice.

UPDATE:

I just flashed in TWRP and the flash succeeded.
[Magisk] redultra82:
Me again

Magisk-v16.5-20180626-ianmacd(20180626).zip flashed, reboot, do you want to update and repeat
[Magisk] kantjer:
Most of the time I flash in TWRP because I test before pushing a build.

Next update I will play regular users and use the app.
[Magisk] martyfender:
I usually direct install through the MM, because for me, it has usually been very reliable.

On a second device I have just flashed [MENTION=7187684]ianmacd[/MENTION] 's new build, in MM, also, to test if the newer commit in it solves the root loss issue. It flashed fine in MM.

Maybe I should always be flashing in TWRP?

I just confirmed that 180626 fails to flash after patching the boot.img, through MM direct install, on a second device of the same hardware. Again it was successful through TWRP.
[Magisk] kantjer:
What I feared happened, been running a test build with today's latest commits and thus invincible mode removed. Root loss within a few hours.

I'm testing a new app that's in development and I need root access for it to function so I'm going to skip Magisk updates for now. I will build but not test until the root loss is fixed.
[Magisk] deepu458:
I currently own a Essential Ph-1 Phone with Stock Oreo 8.1 rooted from Kantjer build for the last 3 months. Even when i saw the update today and i always use the Direct Install method because for our device there is an issue with the TWRP. And i see the Direct Install is failed and recorded the log as well, for now to overcome the installation issue i just installed the zip as a magisk module and seems like its working like that.

If the OP [MENTION=3306843]kantjer[/MENTION] is interested, i can provide the log that i have. It seems like its looking for Ramdisk.cpio.xz file when i use Direct Install Method.
[Magisk] kantjer:
The only thing I can think of are the changes I made to remove leading version "v" to accommodate alphanumeric buildType

so show " unofficial" and not "vunofficial" but if that breaks stuff I will go back to version number and leave the "v" in place.

Or as an alternative v_unofficial, need to think about it
[Magisk] Didgeridoohan:
Direct installation failing is a sepolicy thing with the Manager. It'll be fixed...
[Magisk] kantjer:
Thanks for the clarification, saves me the trouble of having to change my buildscript.

Some time ago there was a discussion about NDK r10e vs r17

At that time there were issues with r10e, any news on that?

I want to stay close as official as possible so if r10e is OK again I will switch.
[Magisk] laupuy:
Hi,

Direct install is fine here (cf signature, OP3, LineageOS...)

Thanks :good:
[Magisk] guest4711:
Same here (i9505 - jfltexx - 32bit - SeLinux enforcing). All ok, [MENTION=3306843]kantjer[/MENTION]
[Magisk] ridobe:
I'll chime in. Auto flash worked just fine as always. Taimen/DP3.
[Magisk] martyfender:
It has been over 16 hours since I installed Magisk-v16.5-20180626-ianmacd.zip and as of yet I have not lost root. Also when I installed it I used the direct install through MM with no problems. I will report if I eventually loose root with it.

On another device of the same hardware as in my signature, I will continue to test [MENTION=3306843]kantjer[/MENTION] 's latest builds.
[Magisk] kantjer:
Great.

I'm having trouble building MM with today's commits, so you will have to wait a bit longer :(
[Magisk] ianmacd:
I'm glad you wrote that. I assumed I had broken things due to my ineptitude with submodules again. :)
[Magisk] kantjer:
revert MM "New uninstallation method e86015badc0d6aa3b633bd9cb50620bbf05819b6" and it will build again
[Magisk] dingolino:
Hi,

could you provide on a regular basis also a zip file which contains both MM and Magisk for installing everything in a single step with TWRP? If installing such a zip with TWRP would than a different signature of MM matter? Or would any MM app be replaced during installing in TWRP?

Best regards

dingolino
[Magisk] kantjer:
The current zip will flash both MM and Magisk binary, but it will not install MM when installed version has a different signature

---------- Post added at 10:34 ---------- Previous post was at 10:31 ----------

@ianmacd the issue is fixed https://github.com/topjohnwu/Magisk/commit/c0ac2d540bbb13603ce9e18298fff170ee2575de

---------- Post added at 11:13 ---------- Previous post was at 10:34 ----------

Starting today I also build and upload the new Magisk-uninstaller.zip

I added the uninstaller link to the json file so the uninstaller should also work when accessed from MM (haven't tested it though)
[Magisk] dingolino:
You mean the zip file from this xda thread?

Or your zip files from your git hub in general?

https://github.com/kantjer/MagiskFiles

My thinking was to avoid trouble with the different signatures.

If I flash a "unofficial kantjer" built in TWRP it should be the "complete package" MM and Magisk. If any other MM is installed please remove it and flash the one which is included, to ensure a "clean unofficial install" (sounds a little strange - a "clean unofficial version" ... ;) )

You provide the different files anyway so anybody who only wants Magisk could do it.

ianmacd is now also providing another option (and I think it is helpfull if people are playing a little bit with different versions settings to see what is making sense on different devices, to sort out problems). (and I do not understand why there is no "official nightly built" - I think your and ianmacd approach to have your Magisk built for your personal needs and make it available for others like me is making absolutly sense)

But if you want to try the two "different unoffical Magisk flavors" you always need to make sure not to forget about the different signatures of the MM app.

Having one zip file for TWRP which ensures that the complete package is installed could/would avoid strange side effects.

Not sure whether this would be realy the case, but this is my thinking for asking whether a complete zip file (which is overwriting or deleting any exisitng MM app) would be possible.

Best regards,

dingolino
[Magisk] kantjer:
Every Magisk.zip, official or unofficial will flash both MM and Magisk.

If you want to switch between different signed build you can try to flash the Magisk-uninstaller before the Magisk.zip. I have never used the uninstaller, so no idea if that will do what you want.
[Magisk] martyfender:
[MENTION=3306843]kantjer[/MENTION] I am still loosing root with todays build 180627. I had hoped that [MENTION=4470081]topjohnwu[/MENTION] 's new commits would solve this problem. I will continue to test, but keep my primary device on the older build that works. Again, thanks for your new builds, for us to test. Has anyone else lost root with todays build? Sometimes it can take up to 24 hours to manifest itself.
[Magisk] kantjer:
John is aware of the issue. I'm trying to catch a log of the root loss.

If we think it's fixed I will post here, until then consider the issue not fixed.
[Magisk] hshah:
I've still not lost root for ages now... as mentioned previously, I was running v16.3 on DeluxeROM for the Note 8 and now flashing the latest builds by [MENTION=3306843]kantjer[/MENTION].

I'm now thinking that it must be something I've either frozen (I've not removed anything) or hidden with MagiskHide. Can't think of what else it could have been, especially when I was losing root either instantly or a few minutes after booting.

Anyways, with today's version of Magisk, I've noticed a lot of new modules available in the repo. One wasn't named right (see attached) but that one has disappeared now and links in some of the others don't seem to work. It probably isn't related to the update, and the chances are that the repo has just been updated, but thought I would ask :)
[Magisk] kantjer:
After approx 20 hours I lost root again.

This time adb logcat was running and I hope the 115000 lines of logging did catch something.

I send the log to John, hopefully it will help solving this long outstanding bug.
[Magisk] martyfender:
I hope your logs shed some light on this issue, also.

This time, I have gone ~28 hours without loosing root, since reboot, on both your latest build and with [MENTION=7187684]ianmacd[/MENTION] 's latest, on another device.
[Magisk] mLgz0rn:
Is there an issue with the current version? "180627"

It won't update to it, not through direct install, and neither through twrp.

Im currently on 180625.

Will I need to flash 26 in twrp?

Haven't had any issue so far up untill now.
[Magisk] edzamber:
No root loosed here after 24h and Titanium Backup find SU binary on fresh install :)

S9 G960f BRE5 full deodexed.

Good job [MENTION=3306843]kantjer[/MENTION] [emoji106]
[Magisk] kantjer:
That issue was fixed in the last build

---------- Post added at 10:49 ---------- Previous post was at 10:09 ----------

Without logs and more info on the device it's hard to tell
[Magisk] martyfender:
[MENTION=4470081]topjohnwu[/MENTION] As was noted previously, upon installing [MENTION=3306843]kantjer[/MENTION] 's latest build on one device and [MENTION=7187684]ianmacd[/MENTION] 's latest build, root was eventually lost. Since rebooting both devices though, root has been maintained for over 85 hours and 84, respectively.

Is it just random chance, or is it possible that the second reboot of the devices did something that the first reboot after install did not, to maintain root?

The above is also true for a third device I installed the latest kantjer build on for over 83 hours since the second reboot.

Post updated with times.
[Magisk] martyfender:
I have installed 180627 on my 4th device and instead of waiting for root to fail, I rebooted the device a second time to test my theory.

I gave it plenty of time after the first reboot, before doing the second reboot. I will report back the result in a day or two, or less if I loose root again.

As always, the hardware I speak of is listed in my signature.

I have went 36 hours, so far without loosing root on this device.

I am hoping someone such as [MENTION=4667597]Didgeridoohan[/MENTION] or [MENTION=4470081]topjohnwu[/MENTION] will reply as to why they think a second reboot could solve the root loss problem in recent builds?
[Magisk] Ferbee:
[MENTION=7187684]ianmacd[/MENTION] if it means anything to you, I've lasted 101 hours and still have root intact, on a note 8 no problems what so ever
[Magisk] oreo27:
[MENTION=3306843]kantjer[/MENTION], I'm getting a

Build release stub APK before building "magiskinit"

on my travis-ci builds. Any idea what could be causing these?
[Magisk] ianmacd:
I wish I could take credit for something here, but either the root loss bug is still there and you've just been very lucky, or it's no longer there and all the credit is due to [MENTION=4470081]topjohnwu[/MENTION].

---------- Post added at 09:36 ---------- Previous post was at 09:28 ----------

Do you have all of the latest code checked out? There hasn't been anything for a few days now, but make sure you have the latest submodule code checked out in addition to the superproject.
[Magisk] kantjer:
If you see no other errors it could be you are building things in the wrong order.

Can you show us the build script
[Magisk] oreo27:
The build script is hosted here:

https://github.com/0reo27/Travis-CI-Magisk-Builder/blob/master/.travis.yml

The relevant lines are:

- 'git clone --recurse-submodules https://github.com/topjohnwu/Magisk.git'
- cp config.prop $TRAVIS_BUILD_DIR/Magisk/
- cd $TRAVIS_BUILD_DIR/Magisk
- python3.6 build.py -h
- time python3.6 build.py all && echo "Magisk has been successfully built"

Forgive the troubleshooting commands in between.
[Magisk] ianmacd:
I looked at your full script. Are you really creating the entire build environment from scratch every time?

After the git clone command, try this:

git submodule update --remote app

I ran into a problem with this myself a few days ago. The app submodule state isn't necessarily in sync with the superproject's view of it.

If this doesn't fix it, then I'm not sure what the issue is, but I would advise configuring a permanent build environment and then just fetching the latest changes each time you build. That will reduce the capacity for errors.
[Magisk] oreo27:
Thanks. Yup. Travis spins up a new instance for every build so the environment needs to be setup each time. I've added that now and I'm still getting the error:

https://travis-ci.com/0reo27/Travis-CI-Magisk-Builder/builds/77791500
[Magisk] ianmacd:
Weird. It's asking for a release version of the stub APK, but you're performing a debug build.

I've just looked at the build.py code and, yes, the stub APK must be a release version. I have no idea why, though. You could modify that section of code to also accept a debug version. I don't see why it would matter, but there may be an unforeseen consequence. Try the attached diff, which is totally untested.
[Magisk] kantjer:
Which NDK version are you using?

I'm currently using r17b
[Magisk] oreo27:
r10e. I just tried r17b as well.

https://travis-ci.com/0reo27/Travis-CI-Magisk-Builder/builds/77792936

I think [MENTION=7187684]ianmacd[/MENTION] is right. The build.py script requires a production release for the build all command to work correctly.
[Magisk] kantjer:
Try renaming the stub.apk in the out folder to stub-release.apk and build again
[Magisk] martyfender:
[MENTION=3306843]kantjer[/MENTION] , I was wondering if you or anyone else suffering from root loss in recent builds, have tried the two reboots that I mentioned previously as solving the root loss for me?

[MENTION=4470081]topjohnwu[/MENTION] may be away from coding for a while?

Thanks
[Magisk] kantjer:
For some reason I have not had a root loss for 3 days now.

But I know a few people reported root loss the last couple off days and John is currently looking into this.
[Magisk] Didgeridoohan:
You're giving me way too much credit... I have nowhere near the knowledge of the inner workings of either Android or Magisk to be able to figure that one out.

If it really is a solution... My initial gut feeling is that it's a fluke, but since I'm only guessing I might of course be completely wrong.
[Magisk] martyfender:
Your probably right, maybe it is a fluke with my particular device and Android build. I do know that my 4 devices lost root on all recent builds with invincible mode removed, within 24 hours, until I did the second reboot. Three of those devices I did not reboot until root was lost. On a 4th device that I had an older version with invincible mode still in it, I updated to 180627, did the normal reboot, then rebooted again, with the same result. I have since rebooted all of those devices, since you released an update to your MagiskHide Props Config.
[Magisk] hshah:
I've not lost root in just over a month now on my Note 8 :highfive:

On v16.4 I used to lose it either instantly or a few minutes after booting up.

Reverted to v16.3 and all was surprisingly good for a few weeks and then I took the plunge and installed Kantjer's build. Still no issues :)

Whilst I'm happy it's sorted for me, I'm still annoyed that I don't know how/why lol.
[Magisk] kantjer:
I'm. Waiting for the source to be pushed, invincible mode will be back until the cause of the root loss is fixed.
[Magisk] martyfender:
With 180627 I have not lost root since the second reboot I discussed, previosly, on 4 devices.
[Magisk] hshah:
That won't help figure out why mine has gone from "argh, not again" to "come on you stupid thing, I dare you to lose root" :p
[Magisk] ianmacd:
Do you know that from private correspondence, or is there a public forum or channel that I should be lurking in?
[Magisk] Didgeridoohan:
Internal testing...
[Magisk] martyfender:
Apparently this root loss is a very random thing, as I have just lost root again on 180627, on 1 of my 4 devices.