Repository: yt-dlp
Issue: [SenateGov] Add metadata fields to extractor
Labels: site-enhancement,triage

### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [x] I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field

### Checklist

- [x] I'm requesting a site-specific feature
- [x] I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels))
- [x] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [x] I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [x] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [x] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

US, probably no geo restrictions

### Example URLs

https://www.help.senate.gov/hearings/vaccines-saving-lives-ensuring-confidence-and-protecting-public-health

### Provide a description that is worded well enough to be understood

Add additional metadata fields to the `SenateGov` extractor.

`upload_date`, `timestamp`
The `Date` and `Time` fields should be present on most hearings pages. This can be used to calculate the `upload_date` and `timestamp`.
Ex.
```
Date: Wednesday, September 9th, 2020
Time: 10:00am
Location: 430 Dirksen Senate Office Building
```
Since the hearings take place in Washington, DC, I believe these should all be eastern US time. In addition to the metadata fields for `upload_date` and `timestamp`, it would be nice if the `Date` and `Time` could be preserved without time zone conversion.

`live_status`
I'm not particularly sure how to extract `live_status` from the `SenateGov` pages. For my use case, I just want to avoid downloading the livestreams, so only videos with a `live_status` of `not_live` or `was_live`.

`hearing_type`
I would like a custom metadata field for `hearing_type`. For this example, it would be `Full Committee Hearing` above the title in `<div class="Heading--overline">
          Full Committee Hearing
      </div>`.

`witnesses`
I would also like to add a custom metadata field for `witnesses` to get the witnesses present in the hearing.
For the included URL, that would be `Francis Collins, MD, PhD` and `VADM Jerome M. Adams, MD, MPH`.

The additional witness information could be included in a dictionary:
```
witnesses = {'Francis Collins, MD, PhD': 'Director\nNational Institutes of Health\nBethesda, MD', 'VADM Jerome M. Adams, MD, MPH': 'Surgeon\nGeneral of the United States\nUnited States Department of Health and Human Services\nWashington, DC'
```
For my use case, I really only need the witness names, so the additional witness information is not a priority.

`description`
I would like to change the content of  `description` to not be a generic descriptor like `The U.S. Senate Committee on Health, Education, Labor & Pensions`, but rather include the specific hearing information available on the webpage.
Maybe something like `innerText` of `<div class="Hearing container pt-0 PageContent">`

```
Date: Wednesday, September 9th, 2020
Time: 10:00am
Location: 430 Dirksen Senate Office Building
OPEN IN NEW WINDOW 
Witnesses
Francis Collins, MD, PhD
Director
National Institutes of Health
Bethesda, MD
DOWNLOAD TESTIMONY
VADM Jerome M. Adams, MD, MPH
Surgeon General of the United States
United States Department of Health and Human Services
Washington, DC
DOWNLOAD TESTIMONY
Related Transcripts
 Click here to view the final hearing transcript as printed.
Related Photos
View null Photo 1
View null Photo 2
View null Photo 3
View null Photo 4
View null Photo 5
View null Photo 6
```
For my use case, I would be using description to manually check the other above metadata fields if there is some problem with the extractor. There might be some formatting differences on some of the senate websites, so I'm not sure if it will be easy to parse all the information correctly.

### Provide verbose output that clearly demonstrates the problem

- [x] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [x] If using API, add `'verbose': True` to `YoutubeDL` params instead
- [x] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
PyDev console: using IPython 8.18.1
Python 3.11.11 (main, Dec  4 2024, 08:55:07) [GCC 11.4.0] on linux
import yt_dlp
ydl_opts = {
    'verbose': True
}
gov_url = 'https://www.help.senate.gov/hearings/vaccines-saving-lives-ensuring-confidence-and-protecting-public-health'
with yt_dlp.YoutubeDL(ydl_opts) as ydl:
    gov_info = ydl.sanitize_info(ydl.extract_info(url=gov_url, download=False))
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8 (No ANSI), error utf-8 (No ANSI), screen utf-8 (No ANSI)
[debug] yt-dlp version nightly@2025.01.19.232735 from yt-dlp/yt-dlp-nightly-builds [68221ecc8] (pip) API
[debug] params: {'verbose': True, 'compat_opts': set(), 'http_headers': {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Accept-Language': 'en-us,en;q=0.5', 'Sec-Fetch-Mode': 'navigate'}}
[debug] Python 3.11.11 (CPython x86_64 64bit) - Linux-6.8.0-51-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg N-71064-gd5e603ddc0-static (setts), ffprobe 4.4.2
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.11.17, mutagen-1.47.0, requests-2.32.3, secretstorage-3.3.3, sqlite3-3.37.2, urllib3-2.2.3, websockets-13.1
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1837 extractors
[SenateGov] Extracting URL: https://www.help.senate.gov/hearings/vaccines-saving-lives-ensuring-confidence-and-protecting-public-health
[SenateGov] vaccines-saving-lives-ensuring-confidence-and-protecting-public-health: Downloading webpage
[SenateISVP] Extracting URL: https://www.senate.gov/isvp/?auto_play=false&comm=help&filename=help090920&poster=https://www.help.senate.gov/assets/images/video-poster.png&stt=950
[SenateISVP] help090920: Downloading webpage
[SenateISVP] help090920: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec, channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
gov_info
Out[3]: 
{'id': 'help090920',
 'title': 'Vaccines: Saving Lives, Ensuring Confidence, and Protecting Public Health',
 'formats': [{'format_id': '395',
   'format_index': None,
   'url': 'https://www-senate-gov-msl3archive.akamaized.net/help/help090920_1/index_396_av-p.m3u8?sd=10&rebase=on',
   'manifest_url': 'https://www-senate-gov-msl3archive.akamaized.net/help/help090920_1/master.m3u8',
   'tbr': 395.0,
   'ext': 'mp4',
   'fps': None,
   'protocol': 'm3u8_native',
   'preference': None,
   'quality': None,
   'has_drm': False,
   'width': 640,
   'height': 360,
   'vcodec': 'avc1.77.30',
   'acodec': 'mp4a.40.2',
   'dynamic_range': 'SDR',
   'video_ext': 'mp4',
   'audio_ext': 'none',
   'vbr': None,
   'abr': None,
   'resolution': '640x360',
   'aspect_ratio': 1.78,
   'http_headers': {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36',
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    'Accept-Language': 'en-us,en;q=0.5',
    'Sec-Fetch-Mode': 'navigate'},
   'format': '395 - 640x360'}],
 'subtitles': {},
 'thumbnail': 'https://www.help.senate.gov/assets/images/sharelogo.jpg',
 '_old_archive_ids': ['senategov help090920'],
 'original_url': 'https://www.help.senate.gov/hearings/vaccines-saving-lives-ensuring-confidence-and-protecting-public-health',
 'webpage_url': 'https://www.help.senate.gov/hearings/vaccines-saving-lives-ensuring-confidence-and-protecting-public-health',
 'webpage_url_basename': 'vaccines-saving-lives-ensuring-confidence-and-protecting-public-health',
 'webpage_url_domain': 'help.senate.gov',
 'extractor': 'SenateISVP',
 'extractor_key': 'SenateISVP',
 'display_id': 'vaccines-saving-lives-ensuring-confidence-and-protecting-public-health',
 'description': 'The U.S. Senate Committee on Health, Education, Labor & Pensions',
 'age_limit': 0,
 'playlist': None,
 'playlist_index': None,
 'thumbnails': [{'url': 'https://www.help.senate.gov/assets/images/sharelogo.jpg',
   'id': '0'}],
 'fulltitle': 'Vaccines: Saving Lives, Ensuring Confidence, and Protecting Public Health',
 'release_year': None,
 'requested_subtitles': None,
 '_has_drm': None,
 'epoch': 1737391137,
 'format_id': '395',
 'format_index': None,
 'url': 'https://www-senate-gov-msl3archive.akamaized.net/help/help090920_1/index_396_av-p.m3u8?sd=10&rebase=on',
 'manifest_url': 'https://www-senate-gov-msl3archive.akamaized.net/help/help090920_1/master.m3u8',
 'tbr': 395.0,
 'ext': 'mp4',
 'fps': None,
 'protocol': 'm3u8_native',
 'preference': None,
 'quality': None,
 'has_drm': False,
 'width': 640,
 'height': 360,
 'vcodec': 'avc1.77.30',
 'acodec': 'mp4a.40.2',
 'dynamic_range': 'SDR',
 'video_ext': 'mp4',
 'audio_ext': 'none',
 'vbr': None,
 'abr': None,
 'resolution': '640x360',
 'aspect_ratio': 1.78,
 'http_headers': {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36',
  'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
  'Accept-Language': 'en-us,en;q=0.5',
  'Sec-Fetch-Mode': 'navigate'},
 'format': '395 - 640x360',
 '_type': 'video',
 '_version': {'version': '2025.01.19.232735',
  'current_git_head': None,
  'release_git_head': '68221ecc87c6a3f3515757bac2a0f9674a38e3f2',
  'repository': 'yt-dlp/yt-dlp-nightly-builds'}}
```