Package: beeb
beebA modern interface to the BBC Sounds radio catalogue.Motivationbeebis a light alternative toget_iplayer(10,000 lines of Perl)
andyoutube-dl(750+ compatible sites).These libraries aim to be deep and broad,beebjust aims to be specific and quick,
handling common use cases (scheduling and download links) clearly, in nicely
object-oriented Python.Requirements❧ Click here to show requirementsBeautifulSoupHTTPXArequests-like API with async and HTTP/2 supportaiostream, aiofilesAsynchronous requests to speed up channel listings retrieval and stream downloadstqdmGives the option to show progress when multiprocessing channel listingsmore_itertoolsdateutilSchedule handlingAChannelSchedulestores a single day's listings, for a single channel.❧ Click here for basic info on ChannelScheduleNational, local, regional channels can be selected by ID or short nameThe schedule with today's date is loaded by defaultTo load today's schedule for BBC R4:frombeeb.navimportChannelScheduleChannelSchedule.from_channel_name("r4")⇣ChannelSchedule for BBC Radio 4 on 2021-03-17These ChannelSchedule objects can be used to find programmes:>>>frombeeb.navimportChannelSchedule>>>s=ChannelSchedule.from_channel_name("r4")>>>s.get_broadcast_by_title("Today",pid_only=True)'m000t476'>>>s.get_broadcast_by_title("Midnight News")00:00on17/03/2021—MidnightNews>>>forbins.get_broadcast_by_title("Shipping Forecast",multi=True):b00:48on17/03/2021—ShippingForecast05:20on17/03/2021—ShippingForecast12:03on17/03/2021—ShippingForecast❧ Click here for more complex ChannelSchedule examples>>>forbins.get_broadcast_by_title(r".*\bNews\b",regex=True,multi=True):b...00:00on17/03/2021—MidnightNews05:30on17/03/2021—NewsBriefing12:00on17/03/2021—NewsSummary18:00on17/03/2021—SixO'Clock News>>>forbins.get_broadcast_by_title(r".*\bnews\b",multi=True,...case_insensitive=True,regex=True,synopsis=True):print(b)...00:00on17/03/2021—MidnightNews05:30on17/03/2021—NewsBriefing06:00on17/03/2021—Today12:00on17/03/2021—NewsSummary13:00on17/03/2021—WorldatOne17:00on17/03/2021—PM18:00on17/03/2021—SixO'Clock News20:00on17/03/2021—MoralMaze22:00on17/03/2021—TheWorldTonight23:30on17/03/2021—TodayinParliament>>>forbins.get_broadcast_by_title(...r".*\b(pandemic|virus|coronavirus|Covid|vaccines?|vaccinations?|health|healthcare|NHS)\b",...multi=True,case_insensitive=True,regex=True,synopsis=True):print(b)...10:00on17/03/2021—Woman's Hour15:00on17/03/2021—MoneyBox15:30on17/03/2021—InsideHealthChannelListingsare a collection ofChannelScheduleobjects over a
given time period (from up to 30 days ago).❧ Click here for basic info on ChannelListingsThe schedules are loaded asynchronously and then their HTML is parsed on all CPU cores (fast!)No interface is currently implemented for multiple channels (as I don't particularly need it), butbeeb.nav.ChannelPickergives all available channels if you wanted to iterate over them.>>>frombeeb.navimportChannelListings>>>ChannelListings.from_channel_name("r4")ChannelListingsforBBCRadio4from2021-02-17to2021-03-18(30days)The schedules are stored as a chronological list in theChannelListings.schedulesattribute>>>frombeeb.navimportChannelListings>>>l=ChannelListings.from_channel_name("r4")ChannelListingsforBBCRadio4from2021-02-17to2021-03-18(30days)>>>l.schedules[0]ChannelScheduleforBBCRadio4on2021-02-17ChannelListings follows the same interface as ChannelSchedule (they share a commonly bound method,
to avoid too much parameter passing).❧ Click here for examples of ChannelListings queriesThere were 26 'Today' episodes aired on BBC R4 in the last 30 days (not aired on Sundays):>>>fori,binenumerate(l.get_broadcast_by_title("Today",multi=True)):...print(f"{i:2}){b}")...0)06:00onWed17/02/2021—Today1)06:00onThu18/02/2021—Today2)06:00onFri19/02/2021—Today3)07:00onSat20/02/2021—Today4)06:00onMon22/02/2021—Today5)06:00onTue23/02/2021—Today6)06:00onWed24/02/2021—Today7)06:00onThu25/02/2021—Today8)06:00onFri26/02/2021—Today9)07:00onSat27/02/2021—Today10)06:00onMon01/03/2021—Today11)06:00onTue02/03/2021—Today12)06:00onWed03/03/2021—Today13)06:00onThu04/03/2021—Today14)06:00onFri05/03/2021—Today15)07:00onSat06/03/2021—Today16)06:00onMon08/03/2021—Today17)06:00onTue09/03/2021—Today18)06:00onWed10/03/2021—Today19)06:00onThu11/03/2021—Today20)06:00onFri12/03/2021—Today21)07:00onSat13/03/2021—Today22)06:00onMon15/03/2021—Today23)06:00onTue16/03/2021—Today24)06:00onWed17/03/2021—Today25)06:00onThu18/03/2021—TodayHere's a query of all programmes which mention vaccin(es,ations,inologists) in their
title/subtitle/synopsis:>>>forbinl.get_broadcast_by_title(r".*\b(vaccin.+?)\b",multi=True,case_insensitive=True,...regex=True,synopsis=True):print(b)...15:30onWed17/02/2021—InsideHealth18:00onWed17/02/2021—SixO'Clock News11:30onMon22/02/2021—HowtoVaccinatetheWorld14:00onSat27/02/2021—AnyAnswers?07:10onSun28/02/2021—Sunday11:30onMon01/03/2021—HowtoVaccinatetheWorld20:00onWed03/03/2021—MoralMaze22:15onSat06/03/2021—MoralMaze11:30onMon08/03/2021—HowtoVaccinatetheWorld11:30onMon15/03/2021—HowtoVaccinatetheWorld18:00onMon15/03/2021—SixO'Clock News22:00onMon15/03/2021—TheWorldTonight21:00onTue16/03/2021—InsideHealth15:30onWed17/03/2021—InsideHealth18:00onWed17/03/2021—SixO'Clock NewsFrom the ChannelListings you can buildProgrammeCataloguedictionaries,
by inspecting the parent programme IDs of each of the broadcasts in the schedules
within the listings. Catalogues for multiple stations can be organised in aProgrammeGuide.❧ Click here for ProgrammeCatalogue examplesTo obtain an up-to-dateProgrammeCataloguewith just programme PIDs and titles:beeb.api.get_programme_dict("r4",n_days=1)⇣{'b00cs19l':'Midnight News','b006qfvv':'Shipping Forecast','b006s54y':'Selection of BBC World Service Programmes','b007rhyn':'News Briefing','b006qmpj':'Prayer for the Day','b006qj8q':'Farming Today','b01s6xyk':'Tweet of the Day','b006qj9z':'Today','b09zgd6y':'Chinese Characters','b007qlvb':"Woman's Hour",'m0002rjm':"Alexei Sayle's The Absence of Normal",'b04fc120':'News Summary','b006qps9':'You and Yours','b007rn05':'Weather','b006qptc':'World at One','b006qpgr':'The Archers','b04xxp0g':'Drama','b006qjnv':'Money Box','b019dl1b':'Inside Health','m000s2kt':'Sideways','b00dv9hq':'The Media Show','b006qskw':'PM','b006qjxt':"Six O'Clock News",'b006qsq5':'Front Row','b006qk11':'Moral Maze','b006xp1x':'Lent Talks','b006r4wn':'Costing the Earth','b006qtl3':'The World Tonight','m000czyb':'The Skewer','b006qtqd':'Today in Parliament'}To obtain aProgrammeCataloguewith genres (be warned — for the last 30 days this takes 60 seconds):beeb.api.get_genre_programme_dict("r4",n_days=30)⇣{'Arts':[('b006v8jn','A Good Read')],'Arts, Culture & the Media':[('b01875r3','One to One'),('b006qsq5','Front Row'),('b00dv9hq','The Media Show'),('b006qp6p','Open Book'),('b006r5jt','The Film Programme'),('b006slnx','Feedback'),('m00055q2','Rewinder'),('b006qjym','Loose Ends'),('b06p0p0g','The Why Factor'),('b006qpdd','Pick of the Week'),('b006r9xr','Start the Week'),('b006s5sf','Bookclub'),('b09w07c4','Art of Now')],'Biographical':[('b0721qqk','Riot Girls')],'Chat':[('p04x5pd7','Fortunately... with Fi and Jane'),('m000s9s1','Between Ourselves with Marian Keyes'),('b00snr0w','The Infinite Monkey Cage')],'Classic & Period':[('m000j0t9','Electric Decade')],'Comedy':[('b00x8dq1','My Teenage Diary'),('b08mj1wj','Reluctant Persuaders'),('m0002rjm',"Alexei Sayle's The Absence of Normal")],'Consumer':[('b006qps9','You and Yours')],'Crime & Justice':[('b006tgy1','Law in Action'),('m0000nfh','Intrigue')],'Disability':[('b006qxww','In Touch')],'Drama':[('b04xxp0g','Drama'),('b08lw2hh','Short Works'),('m000s855',"Hardy's Women")],'Entertainment':[('b060cdyj','Bunk Bed')],'Factual':[('b006s54y','Selection of BBC World Service Programmes'),('b007qlvb',"Woman's Hour"),('m0001kbd','Born in Bradford'),('b006th08','File on 4'),('m000s2kt','Sideways'),('b006qk11','Moral Maze'),('b006qjlq','From Our Own Correspondent'),('b006qnc7','Radio 4 Appeal'),('b07cblx9','The Briefing Room'),('b006qng8','A Point of View'),('b006qnj3','Broadcasting House'),('m0003r3t','My Name Is...'),('m00019hp','Archive on 4'),('b007qxpr','Round Britain Quiz'),('b03w7bwg','Out of the Ordinary'),('b09zgd6y','Chinese Characters')],'Families & Relationships':[('b006qgj4','Saturday Live')],'Food & Drink':[('b006qnx3','The Food Programme')],'Gardens':[('b006qp2f',"Gardeners' Question Time")],'Health & Wellbeing':[('b019dl1b','Inside Health')],'Historical':[('m000sqkk','Gudrun')],'History':[('b006qykl','In Our Time'),('b00nrtd2','A History of the World in 100 Objects')],'Life Stories':[('b01mk3f8','Short Cuts'),('b006qnmr','Desert Island Discs'),('b006qpmv','Last Word'),('b006qjz5','Profile'),('b03cdpww','Meeting Myself Coming Back'),('b01cqx3b','The Listening Project')],'Money':[('b006qjnv','Money Box'),('b006sz6t','The Bottom Line')],'Music':[('b00704s1','Counterpoint')],'Nature & Environment':[('b006qj8q','Farming Today'),('b006xrr2','Ramblings'),('b05w99gb','Natural Histories'),('b006r4wn','Costing the Earth')],'News':[('b00cs19l','Midnight News'),('b007rhyn','News Briefing'),('b006qj9z','Today'),('b04fc120','News Summary'),('b006qptc','World at One'),('b006qskw','PM'),('b006qjxt',"Six O'Clock News"),('b006qtl3','The World Tonight'),('b007rhyy','News and Papers'),('b00g3j4x','News'),('b006qnz4','The World This Weekend')],'Panel Shows':[('b006s5dp','Just a Minute')],'Politics':[('b006qtqd','Today in Parliament'),('b006qgvj','Any Questions?'),('b006qjfq','The Week in Westminster'),('b006qmmy','Any Answers?'),('m0001xq1','The Battles That Won Our Freedoms'),('b006r4vz','Analysis'),('b006s624','Westminster Hour')],'Religion & Ethics':[('b006qmpj','Prayer for the Day'),('b006xp1x','Lent Talks'),('b006sgsh','Bells on Sunday'),('b006qn7f','Something Understood'),('b006qnbd','Sunday'),('b006qnds','Sunday Worship')],'Satire':[('b09rwvwt','Henry Normal: A Normal...'),('m000czyb','The Skewer'),('b006qgt7','The Now Show')],'Science & Nature':[('b01s6xyk','Tweet of the Day'),('b07dx75g','The Curious Cases of Rutherford & Fry'),('b036f7w2','BBC Inside Science'),('b06vy2jd','Science Stories')],'Science & Technology':[('b01n7094','The Digital Human')],'Sitcoms':[('b0b2nh1n','Ability')],'Sketch':[('b00kvs8r','Newsjack')],'Soaps':[('b006qpgr','The Archers'),('b006qnkc','The Archers Omnibus')],'Standup':[('b0b22qhr','Stand-Up Specials'),('b011tzjy','Meet David Sedaris')],'Weather':[('b006qfvv','Shipping Forecast'),('b007rn05','Weather')]}This is equivalent to constructing the class directly and accessing its.keyed_by_genreproperty.beeb.nav.ProgrammeCatalogue("r4",n_days=30,with_genre=True)This takes about 7 or 8 seconds (fast given the number of requests it's making!)Note that these requests occasionally fail (async sessions are prone to rare connection errors),
and are silently retried up to 3 times (one seems to be enough in my experience).The programme catalogues can be stored in a database and then restored from there:>>>pc=beeb.nav.ProgrammeCatalogue("r4",n_days=1,with_genre=True,async_pull=True)'>>>pc.store_db()>>>rc=beeb.nav.ProgrammeCatalogue.regenerate_from_db("r4")>>>pc==rc,pc.genred==rc.genred,pc.db.path==rc.db.path,type(rc)istype(pc)(True,True,True,True)Note that regenerated catalogues have no associated day range>>>rc.n_days0The SQLite3 databaseprogramme_catalogue.dbrecords a simple four field table,programmes, inbeeb.data.store:SELECT*FROMprogrammes;⇣m000tcdg|Midnight News|News|r4
m000tcdj|Shipping Forecast|Weather|r4
m000tcdl|Selection of BBC World Service Programmes|Factual|r4
...If your programme(s) of interest are regular, long-running ones, they should be
included in the database of programmes which ships with beeb.❧ Click here for ProgrammeCatalogue database support and multiple channel catalogues in ProgrammeGuideMultipleProgrammeCatalogueobjects can also be produced in aProgrammeGuide.
This is a glorified dictionary, keyed by the station name and with the catalogues
as its values.To create aProgrammeGuidefor an entire categories of stations (available
categories: "national", "regional", "local"), and automatically store them in
a database, callProgrammeGuide.generate_by_category(c),
wherecis either one of the options, or a list of them.The package ships with a catalogue database populated with the national channels
(excluding variants: only FM 'outlets' for those with multiple which share a station title).Populating the programme catalogues database for all national channels
takes approximately 60 to 90 seconds, and then can be reloaded instantly:beeb.nav.ProgrammeGuide.generate_by_category("national")'Unlike the episodes inChannelScheduleandChannelListings, this is
(potentially) available at runtime, instantly, making it preferable for
metadata lookup in some cases.>>>guide=beeb.nav.ProgrammeGuide.generate_by_category("national")>>>guide.get_programme_by_title("Today")('b006qj9z',('Today','News'))>>>guide.get_programme_by_title("Today",pid_only=True)'b006qj9z'The following SQL query shows duplicates in the resulting database
(via). Note that these are
permitted as the primary key consists of both the programme ('brand') PID
and the station short name (corresponding to the channel's entry inbeeb.nav.channel_ids).This is not an error: these shows are simply syndicated across channels.SELECT*FROMprogrammesWHEREpidIN(SELECTpidFROMprogrammesGROUPBYpidHAVINGCOUNT(pid)>1);⇣b006wkp7|Annie Nightingale presents...|Dance & Electronica|r1
b006wkp7|Annie Nightingale presents...|Dance & Electronica|r1x
b006wkry|Newsbeat|News|an
b006wkry|Newsbeat|News|r1
b006wkry|Newsbeat|News|r1x
...Seebeeb.nav.cat.catalogueandbeeb.share.db_utilsfor more details.Stream handling❧ Click here for the basics of stream handlingEpisodes are downloaded from BBC Sounds as M4S (MPEG-DASH streams). There is a header.dashfile
and then multiple.dashfiles, and if you have all of these you can build a MP4 audio file.As far as I know the access to these is geo-fenced, i.e. you must
be in the UK to download but this may vary between programmes/stations.To download an episode, initialise aStreamobject, which will pull the component MPEG-DASH stream
parts, merge them, and transcode from MP4 to WAV at 16k, as is desirable for most audio handling
(but note this transcoding does increase the filesize). By default, the individual parts are deleted
after download.The default download directory is a package-internal path beneathbeeb.data.store, followed by
a subpath denoting: station » programme (by PID) » year » month » day. To change the directory,
subclass or otherwise set_root_store_dir.Another option is to initialise withdefer_pull=False, set the attribute,
then call theStream'spull()andpreprocess()methods.To download an episode by name and date, use the helper class methodStream.from_name:beeb.stream.Stream.from_name("r4","Today",ymd=(2021,3,30))The station ID (in this example, "r4") can be looked up inchannel_idsor with theChannelPicker.by_namehelper class method.The most directly useful functions (which I've needed when interacting with BBC Sounds API) are
those to obtain the M4S links: you only need the final one to construct the full set of URLs
to obtain a complete MP4.The following functions handle this inbeeb.api:get_episode_dicta trivial wrapper to access theepisodes_dictattribute ofEpisodeListingsHtmlfinal_m4s_link_from_programme_pida wrapper to access thelast_m4s_linkattribute of theMpdXmlclass constructed with thefrom_episode_pidclass methodfinal_m4s_link_from_episode_pida wrapper to access thelast_m4s_linkattribute of theMpdXmlclass constructed with thefrom_episode_pidclass method
after obtaining the episode PID from the episode dictget_programme_pid_by_namea wrapper to access thefilteredattribute of aEpisodeMetadataPidJsonobject
constructed with theget_programme_pidclass method.technically it's "by programme title and station name" (the arguments are in this order)You may very well prefer to construct the objects and handle the attributes involved yourself,
these are given as 'recipes' to make it clear how to use beeb's functionality.