Repository: AntennaPod/AntennaPod (language: java)

Issue:
Support for podcast:transcript tag
# Checklist
<!-- Place an x in the boxes to tick them: [x] -->

- [x] I have used the search function to see if someone else has already submitted the same feature request.
- [x] I will only create one feature request per issue.
- [x] I will describe the problem with as much detail as possible

# Feature description
Some podcasts include text transcript using `podcast:transcipt`

**Suggested solution**:
Display transcript and sync it with the audio.

Meeting notes from Nov 2nd, 2023 https://github.com/AntennaPod/AntennaPod/issues/4935#issuecomment-1791614891



Discussion hints:
@keunes or @ByteHamster please assign to me, since I have the podcastindex namespace parsing done in #4933, I could tackle `podcast:transcript`
I have a a few suggestions for transcript related features I think  would be great to include in AntennaPod (originally mentioned in https://github.com/AntennaPod/AntennaPod/issues/5002):

1. Optionally including timestamps when copying from the trancsript. This would be helpful when using a noteapp to write down interesting segments from the podcast as it makes it easier to go back to the podcast an relisten to the particular segment.
2.  Having the transcript autoscroll as the episode is playing, so when switching to the transcript tab, the text is in the same position as the audio. This would facilitate notetaking while listening.
3. Searching the transcript. I am referring to searching within a single transcript , which would make it easier to find interesting segments (which in turn could be navigated to if combined with the below item).
4. Using the transcript for seeking in the audio (clicking sentences to jump to that position).

There are currently at least two podcast apps on iOS that provide some of these features, but none on Android as far as I know. Those apps are https://podcast9.com/ and https://www.airr.io/. They are labeling themselves as more educational / productive alternatives to regular podcast apps and their main value proposition is the ability to interact with and remember what you listen to as well as easily share your favorite bits with others.

A future item to consider which might be out of scope currently is to automatically create transcripts for episodes that don't provide it, e.g. using the live transcribe feature on phones where it is available.
Nice ideas 👍 My guess would be that # 4 wouldn't be too difficult. But for the others I guess we (the app) would need to receive the timestamps with the transcript I think - as I reckon it's not feasible for AP to automatically match the text with the audio. Do you know if transcripts typically have such timestamps?
Thanks @keunes ! I am not very experienced with podcast data and metadata, but I looked around a bit and it seems like at least some transcripts provide this. The [example used on the podcastindex API docs](https://podcastindex-org.github.io/docs-api/#get-/episodes/byid) page includes this segment:

```json
transcriptUrl: "https://mp3s.nashownotes.com/NA-1322-Captions.srt"
```

That srt file includes timestamps which look like this:

```srt
1
00:00:00,000 --> 00:00:05,340
Pull my mask over my nose. Adam
Curry John C Dvorak, Thursday,

2
00:00:05,340 --> 00:00:09,030
February 18 2021. This is your
award winning media

3
00:00:09,030 --> 00:00:13,380
assassination Episode 13. This
is no agenda.

...
```

I am not sure how common it is, but at least it exists.

A complimentary approach that is less robust but does not rely on external data could be to match the relative position of sentences in the transcript with the relative time of the audio file (when no timestamps are available). Even if this was only able to generate precision to the minute, I think it would be helpful for most items I mentioned above. For some, like the scrolling text, it would probably perform quite well since so much of the transcript would be shown on the screen that the matched position does not have to be exact. It would probably benefit from calculating the relative audio position without silences (I noted that there is already an option in AP to skip these when listening).

I like to add an idea mentioned in the issue above: **hide empty tabs** - that could be empty transcripts and especially nonexistent chapters, if only one tab remains the tablayout itself should probably be hidden.

I think thats a sensible solution, to not crowd the AudioPlayer with empty tabs and require to user to swipe a thousand times to get to the desired tab, makes the UX clearer :)
@tonytamsf Any idea how to implement the data store? Downloading all transcripts of all episodes could result in a huge amount of text to be stored. We already have the problem that the database is pretty big. I think it would be better to only store the URLs and then load the transcript when actually viewing the transcript page in AntennaPod. We could enable caching with okhttp to reduce the number of web requests. 
> @tonytamsf Any idea how to implement the data store? Downloading all transcripts of all episodes could result in a huge amount of text to be stored. We already have the problem that the database is pretty big. I think it would be better to only store the URLs and then load the transcript when actually viewing the transcript page in AntennaPod. We could enable caching with okhttp to reduce the number of web requests.

I like your idea, there is no need to store transcription.  Depending on the user, they may not want to be downloading transcript using mobile data, so we might have to get user consent to use mobile to download transcript.
Why not storing transcript on file text on device? Same way episodes are stored. They would be removed along with episodes and it would allow to make them available like episodes are.

Edit :
And it won't clutter database except maybe to put a flag to know there is transcript downloaded. I am not sure it's even needed as all logics would be parallel to what is done for downloaded episodes.
Sounds like a good idea. When streaming, users need a data connection anyways. When downloading, we can download the transcripts, too. :+1:
> Why not storing transcript on file text on device?

I'm just wondering: this wouldn't hinder 'processing' the info (esp. timestamps, for # 4 in OP).

> When downloading, we can download the transcripts, too

Just to throw an idea out: couldn't this be done, but then in the db? As in, download & store in db when episode is downloaded; remove from dB when file is deleted? (Don't know if this would have any actual benefit, see above).
I don't think storing and using text files would cause any problem : it is what video player does with srt files for subtitles.
Storing it in the database but only for downloaded episodes should work :) Then we don't need to store the file path, we can just store the transcript directly.
Don't know if you saw how Podverse implemented transcript with search. For reference there is a video of what they did : https://twitter.com/Podverse/status/1413642917760184325?s=19

IMHO about their screen : 
- I would prefer timecode on left side and with a smaller font and using grey as text color. So time code would be there but won't distract your eyes when reading transcription
- I wouldn't separate block like they did. It's like there is an empty line beetween each segment which is annoying when reading
Thanks for sharing that @Matth7878!

> I would prefer timecode on left side and with a smaller font and using grey as text color

Fully agree

> I wouldn't separate block like they did

I understand your point, but we can't really merge sentences that are separated into two blocks I think:
* we don't know where we can stitch parts together, and where a hard enter should be kept (in the interface). If there's no whitespace, it might still be complicated to read:
![Schermafdruk van 2021-07-11 10-45-14](https://user-images.githubusercontent.com/11229646/125188759-38432280-e235-11eb-89f0-e0ce24a2ffff.png)
* if we do stitch together the sentences, there might be a situation where one line on the screen covers three lines in the transcript file. Then we can only display the first timestamp, which can be problematic in some cases (e.g. when there is a line with `[silence]` that takes a minute - you wouldn't be able to jump to the bit after the silence).

Let me take a stab at getting the podcast:transcript tag to download and stored.  Then we can work on the UI on the next step
> Let me take a stab at getting the podcast:transcript tag to download and stored. Then we can work on the UI on the next step

Sorry I have been taking a break from AntennaPod.  I'm un-assigning myself until I get back

> Sorry I have been taking a break from AntennaPod. I'm un-assigning myself until I get back

I'm back with my beloved AP app. I'll start working on downloading the transcript
Any thoughts about the support of all 4 content types in the `<podcast:transcript>` tag?

Spec: https://github.com/Podcastindex-org/podcast-namespace/blob/main/transcripts/transcripts.md

* HTML - this format has no or limited timecode - I recommend that we ignore this
* JSON - Much more flexible - we should fully support this, I'm searching for sample podcast that use this format (not seeing it yet)
* SRT - great for video or audio, max number of lines 2, max chars 32 - I recommend we support this fully
* WebVTT - for HTML on the web, I recommend we don't support this

Example: https://podnews.net/rss

```
<podcast:transcript url="https://podnews.net/audio/podnews230310.mp3.vtt" type="text/vtt" />
<podcast:transcript url="https://podnews.net/audio/podnews230310.mp3.srt" type="application/srt" />
<podcast:transcript url="https://podnews.net/audio/podnews230310.mp3.txt" type="text/plain" />
<podcast:transcript url="https://podnews.net/audio/podnews230310.mp3.srt" type="application/x-subrip" rel="captions" />
```

It looks like Podnews also provides `text/pain` which is not in the Podcast 2.0 spec

@jamescridland I am a loyal listener of Podnews, is there a reason you don't have JSON version of `podcast:transcript`?  

Example: Podcasting 2.0 https://feeds.podcastindex.org/pc20.xml

It uses the SRT format 
`<podcast:transcript url="https://mp3s.nashownotes.com/PC20-125-Captions.srt" type="application/srt" rel="captions"/>`
@ByteHamster - do you have objections if I pull in this code to parse SRT files? https://github.com/gusthavosouza/SRTParser.

I'm contacting the author about the missing license file his source code.

If you prefer I write my parser, I could do that.  It's not that hard.  Before I embark, I wanted to hear your opinion.
> HTML - this format has no or limited timecode - I recommend that we ignore this

Agreed.

Supporting JSON & SRT is quite nice already.
@tonytamsf I can't generate JSON transcripts - whisper.cpp is what I use, and it doesn't make them.

I agree that supporting all four is redundant, and would agree with your suggestion to cease support for VTT. VTT seems to have been chosen because of support from libraries; but it seems to me that we are offering too many formats here for sensible support. It's not complicated to make an SRT using the same tools as that might make a VTT.

(Edited) I was under the impression that support for HTML was as a simple link to a full web page with a transcript. This doesn't appear to be the case, and I believe that we should also remove this. So, we end up with (in preference order):

1. JSON files for word-by-word closed-captioning, editing, etc.
2. SRT files for closed-captioning (max 32 char length)

(I shall also go and remove the TXT format from the Podnews feed; but will keep the VTT file there).
> Any thoughts about the support of all 4 content types in the `<podcast:transcript>` tag?
> 
> Spec: https://github.com/Podcastindex-org/podcast-namespace/blob/main/transcripts/transcripts.md
> 
> * HTML - this format has no or limited timecode - I recommend that we ignore this
> * JSON - Much more flexible - we should fully support this, I'm searching for sample podcast that use this format (not seeing it yet)
> * SRT - great for video or audio, max number of lines 2, max chars 32 - I recommend we support this fully
> * WebVTT - for HTML on the web, I recommend we don't support this
> 
> Example: https://podnews.net/rss
> 
> ```
> <podcast:transcript url="https://podnews.net/audio/podnews230310.mp3.vtt" type="text/vtt" />
> <podcast:transcript url="https://podnews.net/audio/podnews230310.mp3.srt" type="application/srt" />
> <podcast:transcript url="https://podnews.net/audio/podnews230310.mp3.txt" type="text/plain" />
> <podcast:transcript url="https://podnews.net/audio/podnews230310.mp3.srt" type="application/x-subrip" rel="captions" />
> ```
> 
> It looks like Podnews also provides `text/pain` which is not in the Podcast 2.0 spec
> 
> @jamescridland I am a loyal listener of Podnews, is there a reason you don't have JSON version of `podcast:transcript`?
> 
> Example: Podcasting 2.0 https://feeds.podcastindex.org/pc20.xml
> 
> It uses the SRT format `<podcast:transcript url="https://mp3s.nashownotes.com/PC20-125-Captions.srt" type="application/srt" rel="captions"/>`

@tonytamsf, @jamescridland, `text/plain` is mentioned [here](https://github.com/Podcastindex-org/podcast-namespace/blob/main/docs/1.0.md#transcript) but not [here](https://github.com/Podcastindex-org/podcast-namespace/blob/main/transcripts/transcripts.md).
I don't get this removing supported formats business.  VTT is the only format supported natively by browsers, and like SRT+ in many ways but a proper w3c standard.  If some of the first apps to take a stab at implementing don't support all formats, that's on them (and will improve with time) - no need to penalize everyone else by removing valuable stuff from the spec.  We should encourage the server side to push as much signal out to the world as they can.
> I don't get this removing supported formats business. VTT is the only format supported natively by browsers

Note that this issue is about support for the transcript tag in AntennaPod. First I thought that HTML was pointless, but I see now that it has a (different) use case - outside of the scope of this issue.

Maybe let's create a separate issue in the PC namespace repo about this wider (PC namespace) topic.
Ah, sorry - followed a clickbaity link from [podnews](https://podnews.net/update/size-matters), and thought this was referring to the spec itself!

> [Should we drop support for VTT and even HTML file support](https://github.com/AntennaPod/AntennaPod/issues/4935#issuecomment-1465273529?utm_source=podnews.net&utm_medium=web&utm_campaign=podnews.net:2023-03-13) for the podcast:transcript tag?

> do you have objections if I pull in this code to parse SRT files? https://github.com/gusthavosouza/SRTParser.

If the license allows that, it should be okay. Probably needs some changes, though, because we only want English comments and maybe don't need all the features.
> because we only want English comments

Do we? With 'comments' do you refer to the actual transcripts (subtitles)? If a user has their device in French and there are transcripts/subtitles available in French, why not use that instead.

Side note on implementation: I would be in favour of storing the transcript 'on drive' for downloaded episodes. As we've discussed for chapters, it's probably too big for in the database, and retrieving it only on playtime causes issues like #6062.
This issue has been mentioned on **AntennaPod Forum**. There might be relevant details there:

https://forum.antennapod.org/t/podcast-subtitles/3365/4

@keunes - I'm done with the data parsing for JSON and working on the SRT format now for `<podcast:transcript>`.  After I'm done with the data storage, I would like some mockups on how transcription would work in the AP app.

1. How does transcription display while an episode is being played?  Does it by default hover as an overlay and users can click on the words to jump into a full timecode view?
2. In the timecode full transcription view, I assume the user can scroll back and forward to view the transcription and tap on a timecode to jump into the audio?  Would each word highlight to follow along the audio?
3. The user can copy and paste any portion transcript?
4. Without the overlay of the words, can users jump to view the transcript before listening, how would that work?
5. Propose Phase 2 : Can users search the transcripts in one episode, across all episodes and subscriptions?
Yeey. That's great news @tonytamsf!

Let me try to reply to some of your questions:

> How does transcription display while an episode is being played? Does it by default hover as an overlay and users can click on the words to jump into a full timecode view?

By default this 'live subtitling' should be enabled/visible in the full player, for discoverability (there are not yet many feeds with this tag, so we want to show it when it's available). It should be possible via a setting in the UI section to disable this. (I expect that someone at some point will ask to make this a setting at podcast level, e.g. to only turn it on for their language learning, but let's see.)

We'll need a mock-up for this. We can probably take inspiration from some of the other apps who have enabled this already. To make it work nicely, we might have to make some design/UI changes.

> In the timecode full transcription view, I assume the user can scroll back and forward to view the transcription and tap on a timecode to jump into the audio?

Yes. We'll need a mock-up for this.

> Would each word highlight to follow along the audio?

Depends. It's not always possible at word-level. E.g. SRT/VTT does not – here I think we should highlight each fragment.

In any case, I think the highlighting should be quite subtle (and I'm wondering if it's acceptable if e.g. highlight colour is not visible to visually impaired/does not comply with accessibility standards). We'll need a mock-up for this.

https://github.com/Podcastindex-org/podcast-namespace/blob/main/transcripts/transcripts.md

(~~Which transcript file types do you support at thi point?~~ I see it's SRT and JSON.)

>  The user can copy and paste any portion transcript?

Yes.

> Without the overlay of the words, can users jump to view the transcript before listening, how would that work?

Yes, this should be possible. I made a proposal a long time ago here, following user feedback on a beta version: https://forum.antennapod.org/t/user-feedback-swipe-navigation-from-player-screen-2-3-beta/1119

It's probably a lot of work to implement, bit I still stand behind the UX concept.

(As a back-up, there should probably be an entry in the overflow menu.)

> Propose Phase 2 : Can users search the transcripts in one episode, across all episodes and subscriptions?

That would be cool indeed. Certainly *within* the current episode (e.g. to find the section where they talk about X). Global search is an entirely separate thing.
> I made a proposal a long time ago here... It's probably a lot of work to implement, bit I still stand behind the UX concept. (As a back-up, there should probably be an entry in the overflow menu.)

That's outside the scope of this PR (and would need some discussion). For now, I would do the overflow button. There is enough to discuss about the subtitles dialog/page itself
I have a working PR that can parse, scroll through the transcript while following the audio.   Feel free to download a test version and give some feedback. 
 I use `Podnews Daily` to do my test https://github.com/AntennaPod/AntennaPod/pull/6710

Unfortunately, after sampling podcasts that I am subscribed to, the state of the transcript format is still not super well supported.  Here is a sample of of 104 feeds

# Podcasts sampling
- Roblox Tech Talk - `text/plain` https://share.transistor.fm/s/39ce510f/transcript.txt
   -  parsable with timecoees
-  Dev Interrupted - `text/html` https://feeds.buzzsprout.com/1422892/10190284/transcript
   - not parsable
- The Method - json & 'text/html' https://feeds.buzzsprout.com/1859927/9638130/transcript
   - html not parsable
   - json is fine https://feeds.buzzsprout.com/1859927/9638130/transcript.json
   - (only 2 episode)
- Radical Candor - `application/x-subrip` https://freepodcasttranscription.com/transcription/5e6070b67882e784d6b5baf4b19976bb717f1115.srt
   - parsable
- Indie Bites - `text/plain` https://share.transistor.fm/s/6cdce2f8/transcript.txt (no timecodes)
  - not parsable
- The Golden Hour - `text/html` https://www.podcastone.com/episode/Someones-Getting-Tased--The-Golden-Hour-51-w/-Brendan-Schaub-Erik-Griffin--Chris-DElia?view=transcript
   - not parsable
Hi, really exciting times for AntennaPod and Podcsting in general. I've been doing some research about the use of transcript in the Podcasting 2.0 namespace.
- I've found that even these mime types being equivalent `text/srt` `application/srt` `application/x-subrip` most player don't support all 3, so in in a "test" feed I declare all for the same file. 
- SRT is the most used format, then JSON, VTT and TXT (but these are usually full transcripts, not subtitle with parsable timecodes)

In case is useful, I have a feed with different options of redirect/hosting with different formats and mime-types if you want to test different edge cases, as discussed here : https://github.com/podverse/podverse-web/issues/1168#issuecomment-1695231077
Probably the redirect / hosting part is no problem here as that part was for testing CORS in different web players. (Gitlab Pages works best with web players than Archive if something is interested)

```
Episode 16
Op3+ archive.org + vtt only

Episode 15
Op3+ gitlab + vtt only

Episode 09
Op3 + gitlab + vtt
Op3 + archive + srt

Episode 08 (without OP3)
Gitlab + vtt
Archive + srt

Episode Episode 00
Op3 + gitlab + vtt/srt

All others
OP3 + archive + vtt/srt
```
A problem that I've found with some player, in case you want to take it into account, is that if there is first a transcript tag with a file that doesn't work, some player won't try the next tag that would work.

Feed:
https://kdeexpress.gitlab.io/feed

**TL;NR**

I would prioritize SRT parsing, then JSON and if there is no much trouble VTT given the similarities with SRT. 
For `text/html` and `text/plain` I would either show a new screen with all the text just assuming that is the full transcript to read or just offer a link to be open in the web browser.
I'd jump in here and say two things:

The accurate stat is [contained here](https://stats.podcastindex.org/hourly_counts.json) - which at the time of writing is: 

>    "feedsWithTranscripts": 38,255
>    "episodesWithTranscripts": 1,379,606

There's obviously a chicken and egg scenario with support. If AntennaPod supports transcripts, it would be a significant benefit to podcasting as a whole, but also to the uptake of this feature. But you'll get considerable promotion when this is launched. (I will, as one guarantee, give you a headline story in Podnews.)

Secondly; might I suggest you ideally support these in this format:

1. JSON - this is accurate to the word level, and allows [this kind of highlighted transcript](https://www.steno.fm/show/9b024349-ccf0-5f69-a609-6b82873eab3c/episode/aHR0cHM6Ly9wb2RuZXdzLm5ldC91cGRhdGUvZ2Vuei1mYXN0ZXN0LWdyb3dpbmc=), where the individual word being said is highlighted on screen. That also allows you to lay out closed-captions in a very bespoke way - allowing you to properly wordwrap them. It also allows more features in future like clipping, where you could highlight a word or sentence, and have the app specifically clip that accurately.

2. SRT - these are accurate to about 32 characters, and therefore the timing isn't as perfect as it could be. [It works like this](https://www.steno.fm/show/917393e3-1b1e-5cef-ace4-edaa54e1f810/episode/YTQ2OWE5ZmQtMzVlOC00MDNhLWIzMDUtZGEyNmU3MDE1ZjBh). 

I'd, frankly, not support TXT or HTML unless you really want to - my suspicion is that these are relatively poorly supported anyway. But it doesn't seem like a heavy lift to simply open these in a webview. (I'm also very unclear if anyone is using VTT.)

I hope that helps a bit!
> If AntennaPod supports transcripts

Thanks for chipping in @jamescridland - one of our contributors is already working on this functionality. Hope we can coordinate re your coverage when it gets released. :-)

> JSON … That also allows you to lay out closed-captions in a very bespoke way - allowing you to properly wordwrap them

See, that's what I was thinking. But I'm also wondering: how. Let's say it is at word level, how do you wrap that into the traditional two lines neatly? Starting from the first word (with its time stamp) and fill up the two lines until the screen is full, and take the end-time of the last word as the end-time for these two lines? Do you 'hard break' when there's a comma, or a period? What about speaker changes?

I feel that SRT - given that it's the provider's responsibility to make it work with the character limit - often will provide a more logical view (even if less accurate).

Either way - I'm very happy that both SRT & JSON already can be fetched and parsed 😀
Thank you @jamescridland for the details, it is helpful.

What would be helpful for me when I am implementing the transcript feature is to look at other Android apps that are supporting the transcript tag.  Do you recommend an app that has implemented this feature well and taken advantage of the JSON word level time code? 
[Podverse](https://github.com/podverse) could be a reference. [The web displays](https://podverse.fm/es/episode/hazS_hcJz) 1 word per line :-1: , but the mobile (-rn repo) app shows the full sentence :+1: . So I assume it's using a word level JSON.
In the Android app each line on sentences is a placeholder for the audio timestamps, not exactly a word by word. [Steno.fm](https://www.steno.fm/show/396d9ae0-da7e-5557-b894-b606231fa3ea/episode/QnV6enNwcm91dC0xMzgwNjg5Nw==) is 1 by 1, but I don't think it's open source :-1: 

![imagen](https://github.com/AntennaPod/AntennaPod/assets/2069735/ae51a17b-13b0-4928-934c-9c49b2156130)
![imagen](https://github.com/AntennaPod/AntennaPod/assets/2069735/2d31a517-72df-4b48-9da9-00c957dc6f04)

> What would be helpful for me when I am implementing the transcript feature is to look at other Android apps that are supporting the transcript tag.

@tonytamsf Sorry for the silence on my end. I've been doing a comparative analysis of all the We/Android apps that support the transcript tag and will at some point this weekend share a couple of nice examples.
Hello,

I've thought and designed a bit more over the weekend. Please find below some recordings of (web) apps that I particularly liked, and a link to some mock-ups I made.

Inspiration
---
I tested all Android & web-[apps listed on the Podcast Index website](https://podcastindex.org/apps?appTypes=podcast+player&elements=Transcript) that support the Transcript tag and are accessible without log-in. I always checked  episode [1601 - "Unkool & The Gang"](https://podcastindex.org/podcast/41504?episode=15754728488) of No agenda. It has [this SRT captions file](https://mp3s.nashownotes.com/NA-1601-Captions.srt). Here is [the podcast's RSS feed](https://feeds.noagendaassets.com/noagenda.xml).

### Podfriend **
Cool because: has nicely designed subtitles, with different colours, and speaker images.

https://github.com/AntennaPod/AntennaPod/assets/11229646/aa6d70d0-77d6-42dd-9d5a-ec7557145dd6

### Steno **
cool because: grouping of source content into blocks per speaker, highlighting background of actively playing blocks as well as selected block (when paused or searching)
[Try for yourself](https://www.steno.fm/show/856cd618-7f34-57ea-9b84-3600f1f65e7f/episode/aHR0cDovLzE2MDEubm9hZ2VuZGFub3Rlcy5jb20=) - my recording is too big for GitHub.

### Podfans *
cool because: has up/down arrows in search bar, nice design of timestamps (thin + lighter colour - should try that in my mock-ups too, althgouh they only seem to use mm:ss, not hh)
![Screenshot from 2023-10-29 23-16-03 podfans](https://github.com/AntennaPod/AntennaPod/assets/11229646/e94b019e-d7ba-4824-8c67-de9c5a9459a0)

### Fountain *
cool because: has a little (i) icon with info about the transcript timings - noting they "might not be accurate if the podcaster has dynamic ads." --> not in mock-ups yet
![Screenshot from 2023-10-29 23-09-47 Fountain](https://github.com/AntennaPod/AntennaPod/assets/11229646/007f0bdd-4fe6-40af-8100-a4cac7b4c1af)

### Anytime Player *
cool because: has 'follow transcript' toggle, the way scrolling position follows active block
![Screenshot from 2023-10-29 23-13-14 anytime player](https://github.com/AntennaPod/AntennaPod/assets/11229646/350b356a-f936-4b2f-8a55-20c0aae2038d)

### Podcast Addict
cool because: has option in the player settings to (de)activate subtitles

Mock-up & general considerations
---

Please find here the mock-ups I have so far: https://design.penpot.app/#/view/e357a320-7535-11eb-8a88-eb0147e048f5?page-id=eae2ba44-fe23-80be-8003-5736fde8c987&section=interactions&index=0&share-id=b509ea44-7222-80be-8003-5db588cbed16

Some notes:
* Several apps leverage speaker indication (either name + colon in SRT, or as metadata in JSON). It would be really neat if we can do that too. 
  * Steno 'collates' blocks - [Nathan explained how](https://mastodon.social/@keunes/111305523569815424).
  * Podfriend & Anytime Player somehow connect speaker info from the subtitles with info from the `person` tags within the item (episode)
```
<podcast:person href="http://curry.com/" img="https://upload.wikimedia.org/wikipedia/commons/thumb/d/d5/Adam_Curry_2016.jpg/440px-Adam_Curry_2016.jpg" group="cast" role="host">Adam Curry</podcast:person>
<podcast:person href="http://dvorak.org/blog" img="https://speaking.com/wp-content/uploads/2018/09/John-Dvorak.jpg" group="cast" role="host">John C Dvorak</podcast:person>
```
* I think we'll have to first identify if we have or can extract speakers, and adjust the design of the player & transcript screens for it. Yet maybe speaker indication (especially on the player screen) might be out of scope for now.
* I kept the playback controls visible on purpose - I think it's handy to be able to pause and rewind when viewing the transcript.

_The first set of screens you'll see are  in case there are speakers extracted/known, which can be displayed on the player screen. Followed by some screens where this is not the case. There's also some other details. Maybe it's best if we have a call @tonytamsf & @ByteHamster to see what's feasible for now._
@keunes Thanks for the great research and screen casts of other players.   I can try to join the next community call about this topic if finding time is difficult 
We'll be meeting to discuss the mock-ups Thursday 2 November at 21:00 CET, over at https://meet.antennapod.org/transcripts. Anyone interested in this UX review is welcome to join.
As discussed and agreed during the meeting:
* implement as full-screen dialog (https://m3.material.io/components/dialogs/overview)
* what to do with landscape player view? For now, easy implementation on the right-hand side.
* store transcript SRT/JSON in filesystem
* if there's word-level JSON without speaker:
  * collate text blocks
    * Sentence-based (using periods & question marks)
    * If long pause, start new block
    * If long single block without period, then break it up after x (30?) seconds
  * combine highlighting in blocks of min. 1 second (e.g. 2 words) to avoid words being skipped due to 1-second refresh rate.

Design change to do:
* tap on CC in player, open transcript dialog
* setting to show/hide closed caption (TBC if in player settings or only in global>UI settings)

In separate PRs:
* playback controls in transcript view
* option to reach transcript from ep. detail page; show 'can't jump to postion' warning
* Search in transcript
* Speaker tag (parse & store; display in ep. details; display speaker photo in player screen)

Pull request: Parse podcast:transcript url and store in SQLite
Fix #4935 

@keunes , @ByteHamster mentioned that for the big feature for transcript, we will use the `transcript` branch and target many small PR's

This PR parses the `<podcast:transcript>` tag from a feed and store the URL of most preferred format (JSON and then SRT)

It's ready for review

Validation using the `Podnews Daily` podcast and `No Agenda`
```
sqlite> .schema

CREATE TABLE FeedItems (id INTEGER PRIMARY KEY AUTOINCREMENT ,title TEXT,pubDate INTEGER,read INTEGER,link TEXT,description TEXT,payment_link TEXT,media INTEGER,feed INTEGER,has_simple_chapters INTEGER,item_identifier TEXT,image_url TEXT,auto_download INTEGER,podcastindex_chapter_url TEXT,podcastindex_transcript_type TEXT,podcastindex_transcript_url TEXT);

sqlite> select podcastindex_transcript_type, podcastindex_transcript_url from FeedItems;
application/json|https://podnews.net/audio/podnews231102.mp3.json
application/json|https://podnews.net/audio/podnews231101.mp3.json
application/json|https://podnews.net/audio/podnews231031.mp3.json
...
application/srt|https://mp3s.nashownotes.com/NA-1604-Captions.srt
application/srt|https://mp3s.nashownotes.com/NA-1603-Captions.srt
application/srt|https://mp3s.nashownotes.com/NA-1602-Captions.srt
```

Test command: ./gradlew :parser:feed:testDebugUnitTest --tests "de.danoeh.antennapod.parser.feed.element.namespace.RssParserTest"

Fix patch:
diff --git a/model/src/main/java/de/danoeh/antennapod/model/feed/FeedItem.java b/model/src/main/java/de/danoeh/antennapod/model/feed/FeedItem.java
index a8570ea4e6f..a6142048073 100644
--- a/model/src/main/java/de/danoeh/antennapod/model/feed/FeedItem.java
+++ b/model/src/main/java/de/danoeh/antennapod/model/feed/FeedItem.java
@@ -1,8 +1,10 @@
 package de.danoeh.antennapod.model.feed;
 
+import android.util.Log;
 import androidx.annotation.NonNull;
 import androidx.annotation.Nullable;
 
+import org.apache.commons.lang3.StringUtils;
 import org.apache.commons.lang3.builder.ToStringBuilder;
 import org.apache.commons.lang3.builder.ToStringStyle;
 
@@ -20,6 +22,8 @@
  */
 public class FeedItem extends FeedComponent implements Serializable {
 
+    public static String TAG = "FeedItem";
+
     /** tag that indicates this item is in the queue */
     public static final String TAG_QUEUE = "Queue";
     /** tag that indicates this item is in favorites */
@@ -42,6 +46,8 @@ public class FeedItem extends FeedComponent implements Serializable {
     private transient Feed feed;
     private long feedId;
     private String podcastIndexChapterUrl;
+    private String podcastIndexTranscriptUrl;
+    private String podcastIndexTranscriptType;
 
     private int state;
     public static final int NEW = -1;
@@ -82,7 +88,8 @@ public FeedItem() {
      * */
     public FeedItem(long id, String title, String link, Date pubDate, String paymentLink, long feedId,
                     boolean hasChapters, String imageUrl, int state,
-                    String itemIdentifier, long autoDownload, String podcastIndexChapterUrl) {
+                    String itemIdentifier, long autoDownload, String podcastIndexChapterUrl,
+                    String transcriptType, String transcriptUrl) {
         this.id = id;
         this.title = title;
         this.link = link;
@@ -95,6 +102,10 @@ public FeedItem(long id, String title, String link, Date pubDate, String payment
         this.itemIdentifier = itemIdentifier;
         this.autoDownload = autoDownload;
         this.podcastIndexChapterUrl = podcastIndexChapterUrl;
+        if (transcriptUrl != null) {
+            this.podcastIndexTranscriptUrl = transcriptUrl;
+            this.podcastIndexTranscriptType = transcriptType;
+        }
     }
 
     /**
@@ -162,6 +173,9 @@ public void updateFromOther(FeedItem other) {
         if (other.podcastIndexChapterUrl != null) {
             podcastIndexChapterUrl = other.podcastIndexChapterUrl;
         }
+        if (other.getPodcastIndexTranscriptUrl() != null) {
+            podcastIndexTranscriptUrl = other.podcastIndexTranscriptUrl;
+        }
     }
 
     /**
@@ -440,6 +454,45 @@ public void setPodcastIndexChapterUrl(String url) {
         podcastIndexChapterUrl = url;
     }
 
+    public void setPodcastIndexTranscriptUrl(String type, String url) {
+        updateTranscriptPreferredFormat(type, url);
+    }
+
+    public String getPodcastIndexTranscriptUrl() {
+        return podcastIndexTranscriptUrl;
+    }
+
+    public String getPodcastIndexTranscriptType() {
+        return podcastIndexTranscriptType;
+    }
+
+    public void updateTranscriptPreferredFormat(String type, String url) {
+        if (StringUtils.isEmpty(type) || StringUtils.isEmpty(url)) {
+            return;
+        }
+
+        String canonicalSrr = "application/srr";
+        String jsonType = "application/json";
+
+        switch (type) {
+            case "application/json":
+                podcastIndexTranscriptUrl = url;
+                podcastIndexTranscriptType = type;
+                break;
+            case "application/srr":
+            case "application/srt":
+            case "application/x-subrip":
+                if (podcastIndexTranscriptUrl == null || !podcastIndexTranscriptType.equals(jsonType)) {
+                    podcastIndexTranscriptUrl = url;
+                    podcastIndexTranscriptType = canonicalSrr;
+                }
+                break;
+            default:
+                Log.d(TAG, "Invalid format for transcript " + type);
+                break;
+        }
+    }
+
     @NonNull
     @Override
     public String toString() {
diff --git a/parser/feed/src/main/java/de/danoeh/antennapod/parser/feed/namespace/PodcastIndex.java b/parser/feed/src/main/java/de/danoeh/antennapod/parser/feed/namespace/PodcastIndex.java
index 1f543a5ae53..dd11fb4e7b4 100644
--- a/parser/feed/src/main/java/de/danoeh/antennapod/parser/feed/namespace/PodcastIndex.java
+++ b/parser/feed/src/main/java/de/danoeh/antennapod/parser/feed/namespace/PodcastIndex.java
@@ -14,6 +14,8 @@ public class PodcastIndex extends Namespace {
     private static final String URL = "url";
     private static final String FUNDING = "funding";
     private static final String CHAPTERS = "chapters";
+    private static final String TRANSCRIPT = "transcript";
+    private static final String TYPE = "type";
 
     @Override
     public SyndElement handleElementStart(String localName, HandlerState state,
@@ -28,6 +30,12 @@ public SyndElement handleElementStart(String localName, HandlerState state,
             if (!TextUtils.isEmpty(href)) {
                 state.getCurrentItem().setPodcastIndexChapterUrl(href);
             }
+        } else if (TRANSCRIPT.equals(localName)) {
+            String href = attributes.getValue(URL);
+            String type = attributes.getValue(TYPE);
+            if (!TextUtils.isEmpty(href) && !TextUtils.isEmpty(type)) {
+                state.getCurrentItem().setPodcastIndexTranscriptUrl(type, href);
+            }
         }
         return new SyndElement(localName, this);
     }
diff --git a/storage/database/src/main/java/de/danoeh/antennapod/storage/database/DBUpgrader.java b/storage/database/src/main/java/de/danoeh/antennapod/storage/database/DBUpgrader.java
index e4a1f5a3dd8..bde74948f5d 100644
--- a/storage/database/src/main/java/de/danoeh/antennapod/storage/database/DBUpgrader.java
+++ b/storage/database/src/main/java/de/danoeh/antennapod/storage/database/DBUpgrader.java
@@ -334,6 +334,12 @@ static void upgrade(final SQLiteDatabase db, final int oldVersion, final int new
             db.execSQL("ALTER TABLE " + PodDBAdapter.TABLE_NAME_FEEDS
                     + " ADD COLUMN " + PodDBAdapter.KEY_NEW_EPISODES_ACTION + " INTEGER DEFAULT 0");
         }
+        if (oldVersion < 3030000) {
+            db.execSQL("ALTER TABLE " + PodDBAdapter.TABLE_NAME_FEED_ITEMS
+                    + " ADD COLUMN " + PodDBAdapter.KEY_PODCASTINDEX_TRANSCRIPT_URL + " TEXT");
+            db.execSQL("ALTER TABLE " + PodDBAdapter.TABLE_NAME_FEED_ITEMS
+                    + " ADD COLUMN " + PodDBAdapter.KEY_PODCASTINDEX_TRANSCRIPT_TYPE + " TEXT");
+        }
     }
 
 }
diff --git a/storage/database/src/main/java/de/danoeh/antennapod/storage/database/PodDBAdapter.java b/storage/database/src/main/java/de/danoeh/antennapod/storage/database/PodDBAdapter.java
index 21f12e2230c..3140fdd0ea5 100644
--- a/storage/database/src/main/java/de/danoeh/antennapod/storage/database/PodDBAdapter.java
+++ b/storage/database/src/main/java/de/danoeh/antennapod/storage/database/PodDBAdapter.java
@@ -52,7 +52,7 @@ public class PodDBAdapter {
 
     private static final String TAG = "PodDBAdapter";
     public static final String DATABASE_NAME = "Antennapod.db";
-    public static final int VERSION = 3010000;
+    public static final int VERSION = 3030000;
 
     /**
      * Maximum number of arguments for IN-operator.
@@ -120,6 +120,8 @@ public class PodDBAdapter {
     public static final String KEY_EPISODE_NOTIFICATION = "episode_notification";
     public static final String KEY_NEW_EPISODES_ACTION = "new_episodes_action";
     public static final String KEY_PODCASTINDEX_CHAPTER_URL = "podcastindex_chapter_url";
+    public static final String KEY_PODCASTINDEX_TRANSCRIPT_URL = "podcastindex_transcript_url";
+    public static final String KEY_PODCASTINDEX_TRANSCRIPT_TYPE = "podcastindex_transcript_type";
 
     // Table names
     public static final String TABLE_NAME_FEEDS = "Feeds";
@@ -172,7 +174,9 @@ public class PodDBAdapter {
             + KEY_HAS_CHAPTERS + " INTEGER," + KEY_ITEM_IDENTIFIER + " TEXT,"
             + KEY_IMAGE_URL + " TEXT,"
             + KEY_AUTO_DOWNLOAD_ATTEMPTS + " INTEGER,"
-            + KEY_PODCASTINDEX_CHAPTER_URL + " TEXT)";
+            + KEY_PODCASTINDEX_CHAPTER_URL + " TEXT,"
+            + KEY_PODCASTINDEX_TRANSCRIPT_TYPE + " TEXT,"
+            + KEY_PODCASTINDEX_TRANSCRIPT_URL + " TEXT" + ")";
 
     private static final String CREATE_TABLE_FEED_MEDIA = "CREATE TABLE "
             + TABLE_NAME_FEED_MEDIA + " (" + TABLE_PRIMARY_KEY + KEY_DURATION
@@ -260,7 +264,9 @@ public class PodDBAdapter {
             + TABLE_NAME_FEED_ITEMS + "." + KEY_ITEM_IDENTIFIER + ", "
             + TABLE_NAME_FEED_ITEMS + "." + KEY_IMAGE_URL + ", "
             + TABLE_NAME_FEED_ITEMS + "." + KEY_AUTO_DOWNLOAD_ATTEMPTS + ", "
-            + TABLE_NAME_FEED_ITEMS + "." + KEY_PODCASTINDEX_CHAPTER_URL;
+            + TABLE_NAME_FEED_ITEMS + "." + KEY_PODCASTINDEX_CHAPTER_URL + ", "
+            + TABLE_NAME_FEED_ITEMS + "." + KEY_PODCASTINDEX_TRANSCRIPT_TYPE + ", "
+            + TABLE_NAME_FEED_ITEMS + "." + KEY_PODCASTINDEX_TRANSCRIPT_URL;
 
     private static final String KEYS_FEED_MEDIA =
             TABLE_NAME_FEED_MEDIA + "." + KEY_ID + " AS " + SELECT_KEY_MEDIA_ID + ", "
@@ -656,6 +662,14 @@ private long updateOrInsertFeedItem(FeedItem item, boolean saveFeed) {
         values.put(KEY_IMAGE_URL, item.getImageUrl());
         values.put(KEY_PODCASTINDEX_CHAPTER_URL, item.getPodcastIndexChapterUrl());
 
+        // We only store one transcript url, we prefer JSON if it exists
+        String type = item.getPodcastIndexTranscriptType();
+        String url = item.getPodcastIndexTranscriptUrl();
+        if (url != null) {
+            values.put(KEY_PODCASTINDEX_TRANSCRIPT_TYPE, type);
+            values.put(KEY_PODCASTINDEX_TRANSCRIPT_URL, url);
+        }
+
         if (item.getId() == 0) {
             item.setId(db.insert(TABLE_NAME_FEED_ITEMS, null, values));
         } else {
diff --git a/storage/database/src/main/java/de/danoeh/antennapod/storage/database/mapper/FeedItemCursorMapper.java b/storage/database/src/main/java/de/danoeh/antennapod/storage/database/mapper/FeedItemCursorMapper.java
index fcf51e31ec3..c4e271eb23e 100644
--- a/storage/database/src/main/java/de/danoeh/antennapod/storage/database/mapper/FeedItemCursorMapper.java
+++ b/storage/database/src/main/java/de/danoeh/antennapod/storage/database/mapper/FeedItemCursorMapper.java
@@ -28,6 +28,9 @@ public static FeedItem convert(@NonNull Cursor cursor) {
         int indexAutoDownload = cursor.getColumnIndexOrThrow(PodDBAdapter.KEY_AUTO_DOWNLOAD_ATTEMPTS);
         int indexImageUrl = cursor.getColumnIndexOrThrow(PodDBAdapter.KEY_IMAGE_URL);
         int indexPodcastIndexChapterUrl = cursor.getColumnIndexOrThrow(PodDBAdapter.KEY_PODCASTINDEX_CHAPTER_URL);
+        int indexPodcastIndexTranscriptUrl = cursor.getColumnIndexOrThrow(PodDBAdapter.KEY_PODCASTINDEX_TRANSCRIPT_URL);
+        int indexPodcastIndexTranscriptType = cursor.getColumnIndexOrThrow(
+                PodDBAdapter.KEY_PODCASTINDEX_TRANSCRIPT_TYPE);
 
         long id = cursor.getInt(indexId);
         String title = cursor.getString(indexTitle);
@@ -41,8 +44,11 @@ public static FeedItem convert(@NonNull Cursor cursor) {
         long autoDownload = cursor.getLong(indexAutoDownload);
         String imageUrl = cursor.getString(indexImageUrl);
         String podcastIndexChapterUrl = cursor.getString(indexPodcastIndexChapterUrl);
+        String podcastIndexTranscriptUrl = cursor.getString(indexPodcastIndexTranscriptUrl);
+        String podcastIndexTranscriptType = cursor.getString(indexPodcastIndexTranscriptType);
 
         return new FeedItem(id, title, link, pubDate, paymentLink, feedId,
-                hasChapters, imageUrl, state, itemIdentifier, autoDownload, podcastIndexChapterUrl);
+                hasChapters, imageUrl, state, itemIdentifier, autoDownload, podcastIndexChapterUrl,
+                podcastIndexTranscriptType, podcastIndexTranscriptUrl);
     }
 }


Test patch:
diff --git a/parser/feed/src/test/java/de/danoeh/antennapod/parser/feed/element/namespace/RssParserTest.java b/parser/feed/src/test/java/de/danoeh/antennapod/parser/feed/element/namespace/RssParserTest.java
index 88ac5c73146..4ce8911e921 100644
--- a/parser/feed/src/test/java/de/danoeh/antennapod/parser/feed/element/namespace/RssParserTest.java
+++ b/parser/feed/src/test/java/de/danoeh/antennapod/parser/feed/element/namespace/RssParserTest.java
@@ -97,6 +97,14 @@ public void testMultipleFundingTags() throws Exception {
         assertEquals("https://example.com/funding3", feed.getPaymentLinks().get(2).url);
     }
 
+    @Test
+    public void testPodcastIndexTranscript() throws Exception {
+        File feedFile = FeedParserTestHelper.getFeedFile("feed-rss-testPodcastIndexTranscript.xml");
+        Feed feed = FeedParserTestHelper.runFeedParser(feedFile);
+        assertEquals("https://podnews.net/audio/podnews231011.mp3.json", feed.getItems().get(0).getPodcastIndexTranscriptUrl());
+        assertEquals("application/json", feed.getItems().get(0).getPodcastIndexTranscriptType());
+    }
+
     @Test
     public void testUnsupportedElements() throws Exception {
         File feedFile = FeedParserTestHelper.getFeedFile("feed-rss-testUnsupportedElements.xml");
diff --git a/parser/feed/src/test/resources/feed-rss-testPodcastIndexTranscript.xml b/parser/feed/src/test/resources/feed-rss-testPodcastIndexTranscript.xml
new file mode 100644
index 00000000000..f972ee2076e
--- /dev/null
+++ b/parser/feed/src/test/resources/feed-rss-testPodcastIndexTranscript.xml
@@ -0,0 +1,13 @@
+<?xml version='1.0' encoding='UTF-8' ?>
+<rss version="2.0" xmlns:podcast="https://podcastindex.org/namespace/1.0">
+    <channel>
+        <title>title</title>
+        <item>
+            <title>Podcasts in YouTube make it to the UK</title>
+            <link>https://podnews.net/update/youtube-podcasts-uk</link>
+            <pubDate>Tue, 10 Oct 2023 08:46:31 +0000</pubDate>
+            <podcast:transcript url="https://podnews.net/audio/podnews231011.mp3.json" type="application/json" />
+            <podcast:transcript url="https://podnews.net/audio/podnews231011.mp3.srt" type="application/srt" />
+        </item>
+    </channel>
+</rss>