yashsingh1/xdaforums huggingface dataset:
[Android Apps and Games] Max.T:
Switch to SQLCipher

I would like to suggest switch to SQLCipher.

I have seen FAQ #37 "Storing a secret together with encrypted passwords would not add anything", but there is a way around with the user of KeyStore.

The code below will work with SDK 18+, there is no need for user interaction (later with the switch to SDK 27+ you will have way more options like force to enter pin, etc.).

At the same time database will be "device locked" (many devices support hardware backed Store and keys extraction is not an easy task) with sensitive data match harder to extract from backup, etc.

import android.security.KeyPairGeneratorSpec;

import android.util.Base64;

import java.math.BigInteger;

import java.security.KeyPairGenerator;

import java.security.KeyStore;

import java.util.Calendar;

import javax.crypto.Cipher;

import javax.security.auth.x500.X500Principal;

private static char[] GetDatabasePassword(Context context)

{

try {

KeyStore keyStore = KeyStore.getInstance("AndroidKeyStore");

keyStore.load(null);

if (!keyStore.containsAlias("faircode.mail")) {

Calendar start = Calendar.getInstance();

Calendar end = Calendar.getInstance();

start.add(Calendar.YEAR, -10);

end.add(Calendar.YEAR, 10);

KeyPairGeneratorSpec spec = new KeyPairGeneratorSpec

.Builder(context)

.setAlias("faircode.mail")

.setSubject(new X500Principal("CN=Comp, O=Org C=Cnt"))

.setSerialNumber(BigInteger.ONE)

.setStartDate(start.getTime())

.setEndDate(end.getTime())

.build();

KeyPairGenerator generator = KeyPairGenerator.getInstance("RSA", "AndroidKeyStore");

generator.initialize(spec);

generator.generateKeyPair();

}

KeyStore.Entry entry = keyStore.getEntry("faircode.mail", null);

Cipher cipher = Cipher.getInstance("RSA/ECB/NoPadding");

cipher.init(Cipher.ENCRYPT_MODE, ((KeyStore.PrivateKeyEntry) entry).getPrivateKey());

char result[] = new char[17];

byte enc[] = cipher.doFinal("faircode.mail".getBytes());

Base64.encodeToString(enc, Base64.NO_WRAP).getChars(0, 16, result, 0);

return result;

}

catch (Exception e) { return "eu.faircode.mail".toCharArray(); }

}
[Android Apps and Games] M66B:
Generating a password is relatively simple, but using an encrypted database with Android ROOM is not.

There is also the matter of migrating existing not encrypted databases to encrypted databases.

Also, recent Android versions encrypt all user data by default, so I don't see much benefit in adding this complexity.
[Android Apps and Games] Max.T:
Android data encryption is kind of questionable for rooted devices, with adb access, data is extractable from backups, etc., thus extra level of encryption seems beneficial for e-mail password at least.

The idea is to create new device backed/locked RSA key, this can be used to generate master password for SQLCipher (there is a ROOM library: cwac-saferoom for example) or (much easier to implement) to encrypt-decrypt server passwords and some other sensitive details in putString("password"... getString("password"... blocks (just 6 - 7 points to change right now).

Switch from unencrypted version can be done ones on startup with the check of App RSA key existence.
[Android Apps and Games] bluegrass55:
Notification Sound

Before paying for the Pro version I have been using the free version of FairEmail for a week to test it out.

I was disappointed that you have not included LED notification. I understand your reasoning but would it be time consuming/difficult to include it?

The Notification sound doesn't work. I'm on Stock Oreo 8.1.0 with notification sounds permitted.I receive sounds on my regular email app K@Mail Pro.

Could this be because I have not allowed these permissions in XprivacyLua,

Read Identifiers

Use Tracking

I've allowed the following only,

Get Applications

Get Contacts

Public Storage

Read Network Data

Read Notifications
[Android Apps and Games] M66B:
FairEmail is not specifically designed for rooted devices and people rooting their devices should be aware there is a security risk. Rooting mostly involves unlocking the boot loader, which will result in a message about reduced security being shown at each device boot.

Also, I don't like to depend on external libraries too much because it often appears that I support my stuff longer than the library maintainers.

Nevertheless, I will do some experimentation. If you want to do a pull request for this feature, you are welcome, but first contact me.
[Android Apps and Games] M66B:
Please see this FAQ about LED notifications:

https://github.com/M66B/open-source-email/blob/master/FAQ.md#user-content-faq21
[Android Apps and Games] m0d:
I will see if I can reproduce when I receive new mail.

Another thought it might be happening in new message after clicking Quote/Image button,

But it might be only on first load, once message has been fully loaded/downloaded it probably no longer a problem.

It is not a big deal. Just that after coming across it I am trying to figure out when it happens.
[Android Apps and Games] M66B:
The next version will allow copying a single message.
[Android Apps and Games] bluegrass55:
I did read the FAQ before posting and followed this instruction "You can use the button Manage notifications in the setup to directly go to the Android notification settings"

Which lead me to the screenshot that I included in previous post.
[Android Apps and Games] M66B:
Try tapping on the text "Notifications".

Edit: see also here: https://developer.android.com/training/notify-user/channels

Android is full of wonders ...
[Android Apps and Games] beeshyams:
As earlier discussed, copying would be useful for storing "action pending" mails without searching or starring / mark as unread etc. Of course, there could be other uses but they evade me.

To see such messages one would create a folder but then the folder belongs to that mail account, so for a unified inbox with multiple accounts, we will have multiple folders. Is it possible to have one folder that is not linked to any mail account but is local, just for this purpose? Obviously, no features like sync, rules etc
[Android Apps and Games] M66B:
This is a too specific feature, sorry.

Moreover, a remote folder is easily created and results in automatic backups.
[Android Apps and Games] bluegrass55:
Thanks for the quick reply. Yes Android is full of wonders.

Unfortunately still no Notification sound. Did a reboot just in case.

New issue is trying to setup a Hotmail account. Using the default settings results in Verification error. Doing it manually gets Timed Out error when clicking in the check button.
[Android Apps and Games] m0d:
Just a quick question about stars, does it apply locally or also on server?

Also, do all providers support stars?

Checked the FAQ and the search did not come up with an answer.
[Android Apps and Games] M66B:
Android (Oreo and later) manages all notification sounds, so you'll need to figure out which setting causes it not to work. FairEmail doesn't set or clear notification sounds. For me and many others notification sounds just work by default.

Time outs are typically caused by using a wrong host name or wrong port number or by a firewall blocking connections. This is not something caused by FairEmail.
[Android Apps and Games] m0d:
For hotmail/outlook you need app passwords. You can enable it outlook settings. It is easy. Probably in Faq too mentions that.

It also helps not to be locked out when checking mails in different locations
[Android Apps and Games] M66B:
Locally and if supported by the server remote. Most providers support stars.
[Android Apps and Games] m0d:
Also, did you already check Advanced settings?
[Android Apps and Games] M66B:
These light/sound settings are only for pre Android Oreo.
[Android Apps and Games] m0d:
Great.

I am still having on-demand server connection issues, that disappear when I enable fairemail quick-tile:

When I use on-demand fairly often one or two folders hang or every once in a while I get errors,

But all that disappears when I enable global sync and refresh folders/accounts

---------- Post added at 01:29 PM ---------- Previous post was at 01:27 PM ----------

Some OEMs have custom builds using weird combinations.

I am pretty sure I saw somewhere that certain Oreo builds were fake oreo. Using most settings from earlier versions only had a Oreo/Pie overlag or something.

Anyway, worth a try. If only to test and confirm the contrary
[Android Apps and Games] M66B:
Connecting for automatic sync and manual sync is done in exactly the same way, so retrying manual sync should help too.
[Android Apps and Games] m0d:
Manual sync being syncing while global Sync is enabled, versus on-demand which is when global sync is disabled?

Automatic and Manual sync (both requiring global sync enabled) are working perfectly.

On-Demand (before with the checkbox in account setup - now just applies when global sync is disabled) gives me problems at times.
[Android Apps and Games] M66B:
The problem appears to be that a foreground sync is not discarded when there was an error and that therefore the sync cannot be repeated. This will be fixed in the next release.
[Android Apps and Games] M66B:
This is not a crash, but an error message.

"connection failure": too many connections

Which version of FairEmail were you using?
[Android Apps and Games] m0d:
fairemail: Service create version=1.352
[Android Apps and Games] bluegrass55:
After a fresh re install using the latest app v1.352 everything seems to be running as it should. Using the Quick setup method doesn't bring up the time out errors.

Have setup 7 accounts and will test for a week or so and if all goes well will contribute towards the Pro version.
[Android Apps and Games] M66B:
I have just released

beta

version 1.353

Changelog/download:

https://github.com/M66B/open-source-email/releases/

[MENTION=3138030]m0d[/MENTION] please let me know how this version behaves
[Android Apps and Games] JJ111:
Thanks a lot, great idea to toggle those stars. I don't see any real use in them, so I am happy I now can see more text/information on every line of each email.

And thanks a lot for also giving us the choice to move or to copy emails :good:

---edit---

By standard when reading an email, "delete", "move" and "reply" quick bottons are shown. Since I always copy (and not move) emails, is it possible to change this "move" button, like for instance long pressing on it to make it a "copy" button?!

---edit2---

Or make the whole line editable?! If in landscape mode if for instance would than add more quick action buttons, since there is so much more space than just those 3, like a forward button, since I use that also quite often.

But with my phone also in Portrait mode 4 quick buttons would easily fit.
[Android Apps and Games] M66B:
Unfortunately, those action bars do not support long pressing and not more than five buttons.

Copy is an exceptional operation generally speaking, which is why it is in the more menu.
[Android Apps and Games] JJ111:
OK, how to activate the 5th button? If I count the "more" as one button too, I only have 4 (more, delete, move, reply)

Than I would love the "forward this email" as 5th..
[Android Apps and Games] M66B:
If you select an archive folder there will be 'archive' too.
[Android Apps and Games] JJ111:
I found a bug:

I was surprised I didn't get an email, that you quoted me. I found out, that although manually pulling down in unified folder view, the inbox didn't get synchronized. In advanced options I activated synchronize and all emails arrived, also when manually refreshing. I than deactivated it again, and it did still work manually.

Also when closing and reopening the app.

BUT when I fully closed the app (also remove it from recent app view) while still in progress of manually syncing (with deactivated synchronizing / no constant notification), after I reopen FairEmail it didn't sync all folders anymore. Sometimes only one, the other time 4 etc...

I can than make it work again by activating and deactivating synchronization.

Is this enough information to reproduce this bug?
[Android Apps and Games] M66B:
With the latest version you can repeat a manual sync if it failed (check the folder list for any error messages).

I don't think this is a bug. Unlike automatic sync the manual sync is a one shot operation.

You can check the operations list for pending operations and errors too.
[Android Apps and Games] JJ111:
When I reproduce this 'strange anomaly', in folder list it is all the same, some hourglasses are always frozen, one crossed out cloud, but whether I pull down to sync or if I hit "sync now" just nothing happens. Everything is frozen/no new emails etc. , also no error message (at least I don't find any). You can see on the attached Screenshots, that the time moves on...

For now I just have to take care not to close FairEmail when in folder syncing process. When this anomaly is not happening, if I pull down all folders shortly have an hourglass and than it changes one by one to the two arrows and than to the cloud and everything is synchronized.
[Android Apps and Games] M66B:
What is in the operations view?

Edit: did you manual sync all the folders shown in the screenshots?
[Android Apps and Games] JJ111:
Yes, I did: long pressing on them and "synchronize now" as well as going into the folder and do a pull down. But than the blue (normally circling) button freezes and nothing happens.

I am sorry, but what is "operations view"?
[Android Apps and Games] M66B:
You can open the operations list with the second entry after the accounts in the main navigation menu.

An hourglass symbol means that synchronization was requested, but isn't being executed yet.
[Android Apps and Games] M66B:
[MENTION=7326491]JJ111[/MENTION] thanks for sending the debug info. The logcat shows the foreground service is killed after about 4 seconds. I cannot see why, but I guess your Android version is responsible for this. Can you please check if this helps? https://dontkillmyapp.com/xiaomi
[Android Apps and Games] m0d:
It seemed to work better at first. Folders were reconnecting and syncing..

Some folders sync better than others.

In one account, inbox, sent, junk synced fine but I got timeout syncing trash:

The other account, gmail, conmected fine to inbox but trash and spam are still disconnected
[Android Apps and Games] M66B:
"Timed out" is a connectivity or server problem. This cannot be fixed by FairEmail. The foreground sync makes separate connections to the server and more frequently than the background sync, which could explain the server rejecting connections. The background sync has logic to prevent too many/frequent connections, but the manual sync, which is supposed to run "now", can obviously not do this.

I am currently rewriting the foreground sync to use the work manager, hopefully this will address the problem of [MENTION=7326491]JJ111[/MENTION] too.
[Android Apps and Games] m0d:
Wow. Things have moved along so much. The 353 iterations of the app taking it from a simple app, transforming into a fine mechanism, clockwork almost.

Currently, I send all my mails through fairemail. And, transferred my main accounts.
[Android Apps and Games] JJ111:
Yes, correct, because I wanted to reproduce the problem. I fully closed FairEmail (swiped it out of recent app view - you obviously call it "killing") on purpose WHILE still syncing, cause that is when the problem happens. The next time I open FairEmail I get the problems / manually sync doesn't work anymore.

If syncing works and I fully close the app AFTER syncing finished, than there is no problem the next time.
[Android Apps and Games] M66B:
I have just released iteration 354 355 ;)

Changelog/download:

https://github.com/M66B/open-source-email/releases
[Android Apps and Games] m0d:
Congratulations.

App is almost a piece of art. Getting there, surely and steadily.

I believe the scroll thing happens indeed on first time clicking quote or image button.

Again, it only happens the first time you open the mail and click one of the two buttons.

It seems that the loading images causes a slight scroll up, sometimes to the top, after being loaded to view.

So, I found the culprit.

Great work on app.
[Android Apps and Games] M66B:
I cannot prevent this because Android will reformat the text on showing images.
[Android Apps and Games] JJ111:
You are a genius ???

1) it now syncs much faster (two folders at a time)

2) the problem is gone

Very nice, thanks a lot!
[Android Apps and Games] M66B:
Good.

This is mostly due to using the new work manager, so thank the Google engineers.
[Android Apps and Games] m0d:
I am still getting inconsistent results.

This just now with outlook inbox. Gmail connected fine, but outlook timed out.
[Android Apps and Games] M66B:
Didn't I already say that "timed out" cannot be fixed by FairEmail because it is caused by internet connectivity or server problems?

Edit: if this is caused by manual sync: try to synchronize less frequently and/or to synchronize less folders at once. The server might lock you out due to too many requests. Else switch to automatic sync and let FairEmail automatically fix this.
[Android Apps and Games] beeshyams:
[MENTION=2799345]M66B[/MENTION]

Can we have the folders shown alphabetically when one clicks "move" ?

Thanks

Edit: I know they follow the sync status but while moving it's easier to follow an order
[Android Apps and Games] M66B:
Folders are sorted in a particular order, see here, and I like to keep that consistently everywhere.
[Android Apps and Games] m0d:
Yes, this is manual/on-demand sync.

As before, after switching global sync on - folders connect well.

Just from time to time manual sync hangs.

As for number of folders, in this case I just synced inbox of 2 different accounts.

Gmail worked fine, outlook had the error.

Before manual syncing, enabling global sync probably will help with connection issues, to workaround on-demand reconnection/connection time outs.
[Android Apps and Games] M66B:
That the folders connect when doing an automatic sync is either luck or more likely because automatic sync has a retry mechanism. Manual sync is a one shot operation by nature, so just try again if it failed, but be aware that a provider might lock you out for some time when trying too many times.
[Android Apps and Games] Ænimal:
Manual sync is hanging here as well after update to 355
[Android Apps and Games] M66B:
Please send me the debug info by long pressing 'Support'.

'As well' is incorrect though because the other problems reported are different.

Edit: please describe what you exactly mean by hanging.
[Android Apps and Games] Ænimal:
Logs are sent. Hanging as in the arrow keeps spinning forever and the app is unusable

Edit: The log mail doesn't show in sent.. it's still hanging

Edit2: I managed to enable synchronization, logs now sent
[Android Apps and Games] M66B:
Thanks for the log.

The manual sync is queued for execution, but for some unknown reason the work manager is not executing it. Why does Google's stuff never works as expected?

For example Google wants developers to use loaders, which inexplicable do not run at times. StackOverflow is full of questions about this. I was so tired of it that I wrote SimpleTask, not even 200 lines of code, and it never ever failed to do what it supposed to do. It also saves so many lines of cumbersome plumbing code. I guess Google engineers just make things to complicated.

https://github.com/M66B/open-source...c/main/java/eu/faircode/email/SimpleTask.java

I will see if I can just run the refresh as a SimpleTask tomorrow.

Edit: (temporarily) switch the automatic sync to workaround this for now.
[Android Apps and Games] Ænimal:
No problem, I restored my backup of 353 for the time being. Thanx for the quick support
[Android Apps and Games] M66B:
[MENTION=5478526]Ænimal[/MENTION] are you using a VPN app?
[Android Apps and Games] Ænimal:
Nope
[Android Apps and Games] M66B:
I have just released

beta

version 1.356

Changelog/download:

https://github.com/M66B/open-source-email/releases/

[MENTION=5478526]Ænimal[/MENTION] Can you please try if this version fixes your manual sync problem?
[Android Apps and Games] Ænimal:
Yep, fixed, thanx:good:
[Android Apps and Games] M66B:
This was being caused by a bug in the work manager. According to the work manager there was no internet connection (hence the question about using a VPN app). Fortunately, my analysis was right and this was quickly fixed ;)
[Android Apps and Games] M66B:
I have just released

stable

version 1.357

Changelog/download:

https://github.com/M66B/open-source-email/releases

Feedback, thanks, donations and purchases are appreciated.
[Android Apps and Games] JJ111:
Thanks a lot, manually sync now works amazingly fast (3 folders sync at the same time)

But unfortunately the compose button is now in the middle of the screen a) there it blocks more relevant informations b) since I scroll up/down in the middle of the screen I accidentally hit the button.

So is there a possibility to move it back to the right, where it was? Or a toggle possibility?
[Android Apps and Games] M66B:
The compose and add folder action button were moved to the middle of the screen to make the info at the end of the message/folder item visible. Choices have to be made ...

Scrolling and pressing a button don't go together because a button can only be pressed by a direct tap.
[Android Apps and Games] JJ111:
In the middle the message content more to the beginning gets blocked, which is for me more relevant than the info at the end of the line.

And it is just my bad luck, that when Scrolling up when my thumb comes down (before moving up) I hit exactly the button. So I have to start swiping up a little bit below, than it works.

Thats why I in general always prefer maximum customization / toggle, than everyone has the freedom to do it like it fits for him/her best.

But it's all right for me, if it's too much / I understand you can't make it right to everyone, I will still be very happy about FairEmail ?
[Android Apps and Games] Max.T:
1) Can I suggest per Subject message sorting?

DaoMessage.java

" CASE WHEN 'subject' = :sort THEN message.subject ELSE '' END," +

FragmentMessages.java

case R.id.menu_sort_on_subject:

item.setChecked(true);

onMenuSort("subject");

return true;

2) I can see e-mails/contacts are collected from received mail. Any chance to add some global "remove all local contacts" as with messages to rebuid a way to long/broken list?

3) It seems like a have a problem with LED light with Oreo 8.1.

K-9 works fine, but unfortunately the reason for it is "targetSdkVersion 23" declaration and thus "compatibility" mode. Tried different notification channels (add, delete before add, enableLights, etc.) still no luck unfortunately. Will post the solution if any.

P.S. As to encryption proposed. I am encrypting login details straight from the tilPassword right now, (but code is SDK 26+ only):

putString("password", EnDeStr.StrToSec(tilPassword));

and decrypting in .connect(s)

.connect(host, Integer.parseInt(port), user, EnDeStr.SecToStr(password));
[Android Apps and Games] M66B:
Sorting on subject: why would that be generally useful?

Contact info: is being updated on synchronizing messages. If you are looking to clear the local (not Android) contact info, long press 'Privacy' in the setup (not in the main view) to clear all contact info and synchronize your messages again to rebuild the list.

Notification light: target SDK 23 is not going to work anymore. The notification is supposed to be managed with the Android channel settings, so this looks like an Android bug.

Password encryption: if you provide a migration (including import/export) and compatibility with all Android versions, I am willing to accept a pull request.
[Android Apps and Games] Max.T:
All sorts of notifications... same sender, different message ids (not a conversation from IMAP point of view). Just like XDA mail notifications for example.

Server 1: is up

Server2: is up

Server1: is no longer a server

Server2: is about to quite
[Android Apps and Games] M66B:
I was already thinking it is just an option, so I will add this.

I will look into encrypting the passwords too.