yashsingh1/xdaforums huggingface dataset:
[Nook Touch General] ApokrifX:
I was trying to maintain my library in Calibre for a couple weeks, than gave up...

I’m not a book worm, definitely... :o

Shelves are kept in database.

To update it, you need either:

have sqlite3 on nook and use adb shell + sql scripts

do the same via noogie script automation

via adb: pull database to PC, modify using sqlite3 on PC, push them back.

All above uses database field for auto-shelving only.

If we want to use a book metadata field not presented in database, we need to write an app or script to do it. Not a big deal, except I suck @ UI design (and implementation too).

AFAIR, the SQL script to shelve based on author was posted already.

Question to Renate:

Do you know an [easy] way to pull a book metadata field using android CLI?
[Nook Touch General] Renate:
The MediaScanner crawler on the Nook doesn't impress me.

It's never found any MP3s. Maybe other people have had other results.

If you borrow books from the library, the return date might be of interest to you.

There is no place in the database and the crawler does not read it.

The same goes for language.

I keep track of my own modest library in my own database using my own crawler,

the results being displayed on my own Library.apk replacement.

I've never actually looked at the stock bookshelves so I have no idea how they work.

Um, what was the question? Extracting metadata?
[Nook Touch General] ApokrifX:
Good for you! :)

Basically, we need unzip a file (content.opf) from .epub and do xml parsing from CLI.
[Nook Touch General] Sun_Cat:
Right. That's why I suggested a Calibre plug-in, since Calibre already has fields for lots more metadata than the Nook library.

See the Automated Shelving link in my previous post.
[Nook Touch General] Renate:
All this stuff is getting kind of complicated.

It's a question of how many data sources and how you'll resolve them.

You could check in all your books in Calibre (which I've never actually seen),

then exports its database to the Nook so that it will have all the info and shelves.

If you start checking in books directly in your Nook, how will that work?
[Nook Touch General] ApokrifX:
Might be easier to generate shelving [update] script from Calibre plug-in and execute manually than?
[Nook Touch General] Sun_Cat:
IDK. As I posted earlier, I think the trickiest part will be to match up the two libraries (the NST's and Calibre's) so Calibre knows the unique id for each book in the NST library database. I'm not sure if that's even possible. AFAIK, each time you use Calibre to copy books to the NST, there's no feedback about how the NST is storing them.

Renate, does your custom library app have additional metadata fields compared to stock? If so, then the approach in Automating shelving would work.
[Nook Touch General] Dopedangel:
Calibre does store its own database on the reader so that it know whats already on the device. So matching book etc should not be a problem as long as the book has originally been sent from calibre. If you have stored all your books in calibre and have all the correct metadata. You would be better of using folders rather than catalogs.
[Nook Touch General] Sun_Cat:
That's a good point, but I'm not sure that's the right conclusion. AFAIK Calibre can only read and write the epub files and its own metadata file. It has no way of knowing the Nook internal library unique id, which is required for the shelves database.

You mean telling Calibre how to create the folder structure on the Nook rather than trying to use shelves? That's easy to do without any hacking, but there are two deficiencies compared to shelves:

Each book can be in only one folder. (With shelves it can be on multiple shelves.)

You would have to use the "My Files / Memory Card" part of the native library interface and then navigate the folder structure, which is much clumsier than selecting a shelf.

Or did I misunderstand your suggestion?
[Nook Touch General] ApokrifX:
Just a thought:

We do have book author in nook DB, but not the series.

We can certainly pool series from book metadata, but that’s PiA.

What we can do is to create folders on nook for series and keep books there.

We can use book folder to shelve by series than.

If a book belongs to more than one series, we can encode it into folder name (horror|chldren) as well :o

Basically, if doesn’t have to be series, but any attribute you want to shelve on, not included in nook db.

Naturally, we can create shelves on more them one attribute (by author, by series. etc)

Similarly, folder name can have more than one attribute (but I’d prefer to keep it simple) :eek:
[Nook Touch General] roustabout:
Or, consider the reader broken and use a different app.

Fbreader sorts books into 36 bins (a-z + 0-9) by first letter of the title or by author name by default.

For me, that's already a very nice start on shelving.

fbreader also reads tags inserted by Calibre if you like.

In order to fully use it on a nook device, you need your BN content to live in the same place as your sideloaded content.

I set that up by linking /data/media to /media or to /sdcard.

/sdcard is handy because it doesnt' require repartitioning to contain a useful amount of space.

I use the BN reader app mostly for magazines, and fbreader for most of my books. Together with fbsync (for synicing across all devices) it works really well.
[Nook Touch General] Sun_Cat:
Sounds nice, but I would greatly prefer a solution that doesn't require rooting, or at least not rooting with gapps. I had my NST rooted for a while, and I reverted to stock after getting annoyed that most of the apps that interested me didn't work very well and that it was doing so much invisibly behind the scenes.

I don't love the native reader, but I like it well enough that I would rather see if it's possible to transcend some of its limitations.
[Nook Touch General] Dopedangel:
Well I rooted and installed coolreader on my nook in the first few days after I got it. So never used the nook reader app much. I use coolreader and it built in file explorer. I don't think you need to use google apps to use other apps just root and adb install coolreader although I persoanlly prefer to use google market as its easily allows to get the updated apps and I can read my email etc

I use calibre to send books this way I can find books by genre if I want

Genre\Sub Genre\First Letter\Author\Series\Title.epub

As coolreader scans all the books and makes a database I can also find books by series, Author or Title
[Nook Touch General] LastSilmaril:
LOL. So while I'm glad I spawned this discussion here, I fear we've become sidetracked a bit into lectury non-solutions ('just ditch the reader!' 'use a calibre hack!'). What can I say? I'm stubborn, and want to be able to fix the official, nice-looking interface, flaws and all...even if I don't ultimately end up using it. It's the principle!

In the meantime, there was one link to 'autoshelving' that I think I'll be checking out once I finally make the time to start this project.
[Nook Touch General] ApokrifX:
That's what I'd call "101% organized folder structure"! ;)

Piece of cake to parse and put Title.epub into 5 shelves:

Genre\Sub Genre\First Letter\Author\Series
[Nook Touch General] ApokrifX:
Guys,

Where does calibre keeps (Genre, Sub Genre, Series, Series index) fields?

I’ve checked few epubs content.opf

All I see (relevant):

<metadata…>
<meta name="calibre:rating" content="0.00"/>
<meta name="calibre:series_index" content="1"/>
<dc:contributor opf:role="bkp">calibre (0.6.52) [http://calibre-ebook.com]</dc:contributor>
<dc:subject>Science fiction</dc:subject>
</metadata>

series_index is 1 always for every book in series. :confused:

Is there any standard or everyone follows its own rules?

I wrote early, it’s easy to parse files path like Genre\Sub Genre\First Letter\Author\Series\Title.epub

And it’s easy to unzip and parse content.opf too, but without standard, we’d need to keep unique setting for a user, providing every user book comply with the settings...
[Nook Touch General] met67:
I can confirm this. As long as you send the book to your device (I use a folder on Dropbox :) ), Calibre updates the epub's metadata.

Alse note that Calibre holds its information in a SQLite 'metadata.db' file in the root of the library folder and in a '.metadata.calibre' file in the device folder root, which seems to be in JSON format.
[Nook Touch General] ApokrifX:
Based on

We can put book into following shelves:

<dc:creator opf:file-as="Pehov, Alexey" opf:role="aut">Alexey Pehov</dc:creator>

<meta content="Siala" name="calibre:series"/>

With index

<meta content="2" name="calibre:series_index"/>

These are tags, I assume it's Genre(s) and something else too.

<dc:subject>Fantasy</dc:subject>

<dc:subject>Something else</dc:subject>

We can add the book to all subject shelves or to the first one only.

I'm not sure if dc:subjects are sort-able in Calibre.

---------- Post added at 01:27 PM ---------- Previous post was at 01:04 PM ----------

Save to local folder does it as well.

I've tested XPath parsing with metadata.opf. :o

I guess, no big deal to parse '.metadata.calibre', except it's not necessarily contains info about all books, while metadata.opf is available for every EPUB book (I'm not talking about PDFs)

Let's get something simple working first?
[Nook Touch General] ApokrifX:
XPath is available since API 8 only. I guess, need to fallback to pull... :o
[Nook Touch General] ApokrifX:
Wrote a first draft, only to find that some books authors assign just too many "subjects".

It makes no sense to create shelves for all of them... :(

Works fine otherwise. :o

select count(*), library_item_id from shelf_item group by library_item_id having count(*) > 1;
22|content://media/internal/docs/105
2|content://media/internal/docs/106
2|content://media/internal/docs/107
17|content://media/internal/docs/110
2|content://media/internal/docs/146
18|content://media/internal/products/22

select _data from library_items where _id = 'content://media/internal/docs/105';
/sdcard/My Files/Books/Merrick - Anne Rice.epub

content.opf

<dc:creator opf:role="aut" opf:file-as="Rice, Anne">Anne Rice</dc:creator>
...
<meta name="calibre:series" content="Vampire Chronicles"/>
...
<dc:subject>Fiction</dc:subject>
<dc:subject>Large type books</dc:subject>
<dc:subject>Horror tales</dc:subject>
<dc:subject>General</dc:subject>
<dc:subject>Modern & contemporary fiction (post c 1945)</dc:subject>
<dc:subject>General & Literary Fiction</dc:subject>
<dc:subject>Vampires</dc:subject>
<dc:subject>Horror</dc:subject>
<dc:subject>Horror - General</dc:subject>
<dc:subject>Witches</dc:subject>
<dc:subject>New Orleans (La.)</dc:subject>
<dc:subject>Occult</dc:subject>
<dc:subject>Occult fiction</dc:subject>
<dc:subject>Fiction - Horror</dc:subject>
<dc:subject>working</dc:subject>
<dc:subject>Witchcraft</dc:subject>
<dc:subject>Guatemala - Fiction.</dc:subject>
<dc:subject>Guatemala</dc:subject>
<dc:subject>New Orleans (La.) - Fiction.</dc:subject>
<dc:subject>Vampires - Fiction.</dc:subject>
[Nook Touch General] ApokrifX:
If anybody wants to try it – let me know. :o

Few facts: It doesn’t need root (but you need to install it somehow)

It tracks changes by setting shelf_date_added/shelf_item_date_added to 0, so you can do auto-shelving and undo if you don’t like how it’s done. It won’t undo shelving done manually. If you put a book into an auto-created shelf, the shelf will be deleted still (as of now).

There is no any setting yet, on how to do auto-shelving.

It shelves books by:

docs.authors (from internal.db)(doesn’t split authors if more then one)

dc:subject (only if 1 or 2 of them, ignores otherwise)

calibre:series with order by calibre:series_index
[Nook Touch General] persichini:
i'd be interested in trying it out... I use calibre and i've been manually creating shelves by series.
[Nook Touch General] ApokrifX:
Just in case there might be a bug, I suggest you backup internal.db.

/data/data/com.android.providers.media/databases/internal.db
[Nook Touch General] LastSilmaril:
You know, despite what I said earlier, this seems to be the most painless way to do shelving. Also, somehow, the shelves persisted through a reboot even though I put books on an SD card, which never happened before. Does your program automatically run on boot, or have you found a way to solve this problem???

Note that ordering by series_index doesn't seem to be working.
[Nook Touch General] ApokrifX:
IMO, it’s B&N bug, I don’t know under want condition it shows up.

I didn’t do anything about it, only Renate can fix it.

Try to:

Unmount SD (from settings),
remove SD
reboot
Check shelves
insert SD
reboot
Check shelves again

Can you try basic troubleshooting with:

adb shell
sqlite3 /data/data/com.android.providers.media/databases/internal.db
select si.shelf_item_position, li._data from shelf_item si
inner join library_items li on li._id = si.library_item_id
where si.shelf_item_date_added = 0 and si.shelf_item_position is not null;
.q
^d

If you get nothing, could you upload a series (3-4 books) so I can check, please?

I don’t need whole book, only a couple files (for each book):

META-INF\container.xml

content.opf (it might be in book root or some subfolder)

Copy few books to a folder.

open each book with winzip, winrar, etc.

delete everything except two files above.

Zip all "stripped books" into one file and add as attachment to you post here.
[Nook Touch General] ApokrifX:
Got a new version ready.

A couple cosmetic changes:

Moved main routine into separate thread, which allows adding progress bar, might be helpful if you have huge book collection.

Moved setting to main menu.
[Nook Touch General] ApokrifX:
Apparently, B&N forgot to implement shelf_item_position:

I have in database:

index|path|title
1|/sdcard/My Files/Books/1.epub|Book Z 1
2|/sdcard/My Files/Books/2.epub|Book Y 2
3|/sdcard/My Files/Books/3.epub|Book X 3

But on shelf books are sorted by title

There are only two sorting option in upper right combo box: Title | Most recent

Both gave same result as I have open none of these books yet.
[Nook Touch General] ApokrifX:
Funny, I've uploaded PNG, but got low quality JPG in message... :o
[Nook Touch General] Renate:
Yes, the forums do that.

The silly thing is, the JPEGs are lower quality but larger filesize than the original PNGs.
[Nook Touch General] LastSilmaril:
I was about to say that I checked internal.db using the code you posted a few posts above and found that the shelver was sorting based on order but that this makes no difference to the front-end.

One thing that would be nice is if you would include an option to ignore the dc:subject in the content.opf of the epub itself and just use whatever is in the metadata.opf for each book. The content.opf is often larded up with nonsense that isn't editable in calibre without tweaking the epub manually afaik.
[Nook Touch General] ApokrifX:
Only Renate can fix it, but I'm sure, she is not interested.

I thought about changing book title "on the fly" in database (prepend it with series index), but I guess, it'll be overridden on media scan (SD remove/insert or nook reboot)

You can try it actually, if you want and let us know.

You do know how to write SQL, right?

Right!

Did you try tiny menu button on the top bar in the middle? ;)

Yep!

http://xdaforums.com/showpost.php?p=28392591
[Nook Touch General] Renate:
Mmm, not all that much.

In Library.apk:

The sort order arrays of strings get built in com/bn/nook/library/model/LibraryDao.smali

SORT_ORDER_ITEMS_BY_AUTHOR
SORT_ORDER_ITEMS_BY_CATEGORY
SORT_ORDER_ITEMS_BY_NEW_LAST_ACCESSED
SORT_ORDER_ITEMS_BY_PUBLISHED_LAST_ACCESSED
SORT_ORDER_ITEMS_BY_TITLE

The associated labels are in res/values/strings.xml

sort_by_most_recent
sort_by_title
sort_by_author
sort_by_date_modified
sort_by_file_name
sort_by_type
sort_by_shelf_name

It's been so long since I've seen the stock Library.apk and I've never even tried shelves.
[Nook Touch General] ApokrifX:
Renate,

Thank you.

I'll take a look.

I doubt, I can change anything, tho.

In all or shelf items view, there are 3 sorting options only:

SORT_ORDER_ITEMS_BY_NEW_LAST_ACCESSED
SORT_ORDER_ITEMS_BY_AUTHOR
SORT_ORDER_ITEMS_BY_TITLE

The others were left unimplemented, probably... :o
[Nook Touch General] LastSilmaril:
Yeah, but not well. This doesn't sound terribly complicated to do necessarily though - it's just looking at the shelf_position field you set with the series index, and prepending it to the title in the same table, right? Or did I misunderstand and the title is somewhere else? I'd have to look at it again.

That's a joke, right?

Wait so you're acknowledging the issue? What am I missing here?

I've also found that for shelving by series, your app seems to look at content.opf inside the epubs, not metadata.opf (at least that's the only conclusion I could draw, looking at the results), or at least privileges content.opf. (I have the "calibre_series" tag set inside content.opf for content I reconverted from epub to epub in order that the cover gets properly included inside the epub.)
[Nook Touch General] ApokrifX:
Try it if you can, please.

index|path|title
[B]1[/B]|/sdcard/My Files/Books/1.epub|[B]Book Z[/B]

The book should be named in database "1. Book Z"

No kidding.

You can get to setting screen (select which fields you want to create shelves from):

In 0.0.0.0b it’s from menu button.

In 0.0.0.0a it’s from main screen – preferences "stripe".

Nothing, really. I report same issue earlier, that’s all.

Not exactly.

It looks for META-INF\container.xml (same name always)

It has reference to OPF file (could be any name)

<rootfile full-path="content.opf" media-type="application/oebps-package+xml"/>

Then get the rest of the fields from it.

http://en.wikipedia.org/wiki/EPUB

.opf file

The OPF file, traditionally named content.opf houses the EPUB book's metadata, file manifest, and linear reading order. This file has a root element package and four child elements: metadata, manifest, spine, and guide.

Book cover is separate story, I’m not covering it here :o
[Nook Touch General] LastSilmaril:
I'll look into it; got new phone today so I have to deal with that atm. This is all in the shelves table in internal.db right?

Huh? I think you misunderstood what I was saying. I said

I wasn't asking for a way to disable sorting by series entirely; I was asking for a sub-option to use a different source (metadata.opf, not content.opf) to sort on.

But from what I can tell from your follow up, you don't check the calibre-created metadata.opf outside the epub at all! Just the content.opf inside the epub. I'm asking for an option to change that behavior - or if you really don't care to do that, you could just send me what you've done so to play with it (though I don't really know much about Android development) .

To be clear, the covers thing I just mentioned for why I was converting from epub to epub, not asking you to do anything about that :)
[Nook Touch General] ApokrifX:
I thought, calible keeps/accumulates changes for a book in external metadata.opf.

When calible publish the book to a device or save to a local folder, it propagates external metadata.opf content to content.opf inside the epub.

Could you post both external metadata.opf and internal content.opf for a book published to device or save to a local folder, please?
[Nook Touch General] LastSilmaril:
Oh, I left a crucial part out - I'm now syncing my entire library to an SD card using FolderSync and Dropbox, so that process doesn't happen anymore.
[Nook Touch General] Dopedangel:
You can use modify epub plugin in calibre to update content.opf inside the epub without conversion or sending to device. If you do that you wont need to use metadata.opf
[Nook Touch General] LastSilmaril:
Great tip, there we go!
[Nook Touch General] ApokrifX:
Guys,

Just want to clarify, which calibre action(s) trigger metadata update?

My understanding is:

Metadata updated on destination always (save to folder/send to device)

It's not updates in local library, need above plugin to do it.

So for dropbox sync, it depends which program(s) (and what sequence) executed to put books on destination storage.
[Nook Touch General] ApokrifX:
Version 0.0.0.0c.

Make Subject count cut off # configurable.

Removed unnecessarily dependencies – reduced apk size a bit.

Let me know if you have any bright ideas.
[Nook Touch General] LastSilmaril:
Right, so there is no action on calibre's part for that sync, and metadata isn't copied into the epub proper at all as a result. This plugin allows me to trigger such updates, which is fine, but it doesn't like it strips dc:subject for me. Still, big improvement. And the idea to configure the dc:subject limit was a good one, will try out soon
[Nook Touch General] ApokrifX:
There are workarounds, like save to local folder with metadata update, than replace/overwrite calible copy with book from this folder.

You can do it for one book only.

Still kinda annoying thing to do...

Looks like you are the only one who uses it ;)
[Nook Touch General] andi1235:
Just wanted to say that I just found your app, and it's exactly what I've been looking for. :) Wanted to say thanks!! I have to figure out what to do in Calibre to make things exactly how I want them, but this is still fantastic.
[Xperia Play Q&A, Help & Troubleshooting] ryanraven:
just rooted and installed clockwork mod on my phone but when i try and flash CM7.2 is just says aborted? any reason for this?
[Xperia Play Q&A, Help & Troubleshooting] Xcited android user:
Where du you have downloaded it, is your bootloader unlocked and which phone do you have exactly r800i or x

Sent from the best gaming phone in the world.
[Xperia Play Q&A, Help & Troubleshooting] ryanraven:
i dont understand how to unlock the bootloader and 800i
[Xperia Play Q&A, Help & Troubleshooting] Xcited android user:
You can unlock it on the sony website

Sent from the best gaming phone in the world.
[Xperia Play Q&A, Help & Troubleshooting] ryanraven:
done thanks :) what kernel is best to flash with CM7 nightly?
[Xperia Play Q&A, Help & Troubleshooting] Xcited android user:
See

http://xdaforums.com/showthread.php?t=1600833

Sent from the best gaming phone in the world.
[Samsung Galaxy Nexus] tjex:
I've been using hangout more often these days but having issues with the volume. It doesn't seem to be loud enough. Anyone else experiencing this issue?
[HTC Desire S] GrandMasterPlank:
I'm currently running the latest release of ICS -based MIUI (2.6.22) on my Desire S having previously run a Gingerbread based MIUI on a Desire HD. That's was awesome so I installed 2.6.22 on my new Desire S but I just can't get on with the ICS versions of MIUI as they're still too laggy/buggy (apologies to the Dev's who I know are workin').

I'd like to install the very last Gingerbread-based version of MIUI (which I'm assuming is mega stable) until the ICS versions are fully sorted. Can anyone remember which version of MIUI this was? I'm assuming it's still available on goo.
[HTC Desire S] dr.higgins:
MIUI 2.4.20 is the latest gb.

It's not perfect (graphics) but other than that, everything is working.
[HTC Desire S] GrandMasterPlank:
Are there other earlier releases which are better than 2.4.20 or is this the best Gingerbread Miui rom? I just assumed that the last release would be the best. I'm just looking for the best Gingerbread Miui to run until ICS rom's are up to speed.
[HTC Desire S] kyeeliu:
http://bigota.d.miui.com/2.4.13/miui_DesireS_2.4.13_s9ol1csxlx_2.3.zip

This download address is the Chinese version of the MIUI 2.4.13, based on android 2.3.7.

Only Chinese and English.
[HTC Desire S] dr.higgins:
I'd say you assumed right.

Although I did run 2.4.13 for quite some time.
[Sprint HTC EVO 4G LTE] fifasarajevo:
so i was wondering where the wallpapers for the stock wallpaper picker are located on this phone...i was thinking of putting my own wallpapers in that section because theyre quite bad when cropped by the gallery

please and thanks
[Sprint HTC EVO 4G LTE] evojag:
I am looking for the same thing, any luck friend?
[Sprint HTC EVO 4G LTE] fachadick:
As long as the wallpaper sizes are the correct size (1440x1280), it doesn't matter where you put them. Just expand the box in gallery to cover the whole image, and even though it will say cropping, it won't crop.