Package: meaningless
Meaningless is a Python library used to retrieve, process and download Bible passages from Bible Gateway.Features include:Passage retrieval from theBible Gatewaysite or from a local YAML/JSON/XML/CSV file.Different output formats for different purposes:Multi-line strings for printing Bible passages.Python list of strings (or in-memory data structure) for passing Bible passages to other Python logic.YAML/JSON/XML/CSV files for persistent storage of Bible passages or as input for other applications and scripts.Handling of edge case passages, such as those with tabular data and omitted passages in certain translations.Flags to enable particular content modifications, such as ignoring passage numbers.Filtering on Bible passages from a local file based on a given text input or regular expression.Now accepting feature requests!If you want to see a certain feature included, please create an issue describing all the necessary details.Installationpip install meaninglessAPI DocumentationDocumentation is generated using Sphinx.OnlineThe API documentation is hosted through GitHub Pages at the following link:https://daniel-tran.github.io/meaningless/OfflineYou can view the API documentation as static HTML documents fromdocs\index.html. After cloning this repo, you can load the HTML files in a web browser, which allows you to navigate to other sections.Supported translationsEnglishASVAKJVBRGEHVESVESVUKGNVGWISVJUBKJVKJ21LEBMEVNASBNASB1995NETNIVNIVUKNKJVNLTNLVNMB (New Testament only)NOGNRSVNRSVUEWEBYLTEspañolRVAUsageWeb ExtractorThe Web Extractor is used to obtain passage information directly fromBible Gateway.frommeaninglessimportWebExtractorif__name__=='__main__':bible=WebExtractor()passage=bible.get_passage('Ecclesiastes',1,2)print(passage)Output:² “Meaningless! Meaningless!”
    says the Teacher.
“Utterly meaningless!
    Everything is meaningless.”YAML DownloaderThe YAML Downloader is which formats passages obtained from the Bible Gateway website (using the Web Extractor) into a YAML structure and writes it out to a file:frommeaninglessimportYAMLDownloaderif__name__=='__main__':downloader=YAMLDownloader()downloader.download_passage('Ecclesiastes',1,2)Output:Running the above code would produce a file calledEcclesiastes.yamlin the current working directory with the approximate contents:Ecclesiastes:1:2:"²“Meaningless!Meaningless!”\nsaystheTeacher.\n“Utterlymeaningless!\n\\Everythingismeaningless.”"Info:Language:EnglishTranslation:NIVTimestamp:'0000-00-00T00:00:00.000000+00:00'Meaningless:0.0.0YAML ExtractorThe YAML Extractor uses the generated files from the YAML Downloader to find passages. This is faster than the Web Extractor, since it is not retrieving information from the Internet and is also unaffected by bandwidth limitations.frommeaninglessimportYAMLExtractorif__name__=='__main__':bible=YAMLExtractor()passage=bible.get_passage('Ecclesiastes',1,2)print(passage)Output:Assuming the YAML downloader has already generated a YAML file in the current directory calledEcclesiastes.yamlwhich contains the book of Ecclesiastes in YAML format:² “Meaningless! Meaningless!”
    says the Teacher.
“Utterly meaningless!
    Everything is meaningless.”YAML File InterfaceThe YAML File Interface is a set of helper methods used to read and write YAML files. This can be useful if you need to do some customised processing on a downloaded YAML file.frommeaninglessimportYAMLDownloader,yaml_file_interfaceif__name__=='__main__':downloader=YAMLDownloader()downloader.download_passage('Ecclesiastes',1,2)bible=yaml_file_interface.read('./Ecclesiastes.yaml')bible['Info']['Customised?']=Trueyaml_file_interface.write('./Ecclesiastes.yaml',bible)Output:Running the above code would produce a file calledEcclesiastes.yamlin the current working directory with the approximate contents:Ecclesiastes:1:2:"²“Meaningless!Meaningless!”\nsaystheTeacher.\n“Utterlymeaningless!\n\\Everythingismeaningless.”"Info:Language:EnglishTranslation:NIVTimestamp:'0000-00-00T00:00:00.000000+00:00'Meaningless:0.0.0Customised?:trueJSON DownloaderThe JSON Downloader is effectively the same as the YAML Downloader except the resulting file is in JSON format and has a different file extension.frommeaninglessimportJSONDownloaderif__name__=='__main__':downloader=JSONDownloader()downloader.download_passage('Ecclesiastes',1,2)Output:Running the above code would produce a file calledEcclesiastes.jsonin the current working directory with the approximate contents:{"Ecclesiastes":{"1":{"2":"² “Meaningless! Meaningless!”\n    says the Teacher.\n“Utterly meaningless!\n    Everything is meaningless.”"}},"Info":{"Language":"English","Meaningless":"0.0.0","Timestamp":"0000-00-00T00:00:00.000000+00:00","Translation":"NIV"}}JSON ExtractorMuch like the YAML Extractor, the JSON Extractor uses the generated files from the JSON Downloader to find passages.frommeaninglessimportJSONExtractorif__name__=='__main__':bible=JSONExtractor()passage=bible.get_passage('Ecclesiastes',1,2)print(passage)Output:Assuming the JSON downloader has already generated a JSON file in the current directory calledEcclesiastes.jsonwhich contains the book of Ecclesiastes in JSON format:² “Meaningless! Meaningless!”
    says the Teacher.
“Utterly meaningless!
    Everything is meaningless.”JSON File InterfaceThe JSON File Interface is a set of helper methods used to read and write JSON files. Similar to the YAML File Interface, it can be used to do customised processing on a JSON file or its contents.frommeaninglessimportJSONDownloader,json_file_interfaceif__name__=='__main__':downloader=JSONDownloader()downloader.download_passage('Ecclesiastes',1,2)bible=json_file_interface.read('./Ecclesiastes.json')bible['Info']['Customised?']=Truejson_file_interface.write('./Ecclesiastes.json',bible)Output:Running the above code would produce a file calledEcclesiastes.jsonin the current working directory with the approximate contents:{"Ecclesiastes":{"1":{"2":"² “Meaningless! Meaningless!”\n    says the Teacher.\n“Utterly meaningless!\n    Everything is meaningless.”"}},"Info":{"Customised?":true,"Language":"English","Meaningless":"0.0.0","Timestamp":"0000-00-00T00:00:00.000000+00:00","Translation":"NIV"}}XML DownloaderThe XML Downloader is effectively the same as the YAML Downloader except the resulting file is in a specific XML format and has a different file extension.frommeaninglessimportXMLDownloaderif__name__=='__main__':downloader=XMLDownloader()downloader.download_passage('Ecclesiastes',1,2)Output:Running the above code would produce a file calledEcclesiastes.xmlin the current working directory with the approximate contents:<?xml version="1.0" encoding="utf-8"?><root><info><language>English</language><translation>NIV</translation><timestamp>0000-00-00T00:00:00.000000+00:00</timestamp><meaningless>0.0.0</meaningless></info><bookname="Ecclesiastes"tag="_Ecclesiastes"><chapternumber="1"tag="_1"><passagenumber="2"tag="_2">²“Meaningless!Meaningless!”saystheTeacher.
“Utterlymeaningless!Everythingismeaningless.”</passage></chapter></book></root>XML ExtractorMuch like the YAML Extractor, the XML Extractor uses the generated files from the XML Downloader to find passages.frommeaninglessimportXMLExtractorif__name__=='__main__':bible=XMLExtractor()passage=bible.get_passage('Ecclesiastes',1,2)print(passage)Output:Assuming the XML downloader has already generated an XML file in the current directory calledEcclesiastes.xmlwhich contains the book of Ecclesiastes in XML format:² “Meaningless! Meaningless!”
    says the Teacher.
“Utterly meaningless!
    Everything is meaningless.”XML File InterfaceThe XML File Interface is a set of helper methods used to read and write XML files. Unlike the other file interfaces, this is more geared towards the specific document format used by the XML Downloader and Extractor, so you may observe some strange behaviour if you try using this for general purpose XML file interactions.frommeaninglessimportXMLDownloader,xml_file_interfaceif__name__=='__main__':downloader=XMLDownloader()downloader.download_passage('Ecclesiastes',1,2)bible=xml_file_interface.read('./Ecclesiastes.xml')bible['Info']['Customised']=Truexml_file_interface.write('./Ecclesiastes.xml',bible)Output:Running the above code would produce a file calledEcclesiastes.xmlin the current working directory with the approximate contents:<?xml version="1.0" encoding="utf-8"?><root><info><language>English</language><translation>NIV</translation><timestamp>0000-00-00T00:00:00.000000+00:00</timestamp><meaningless>0.0.0</meaningless><customised>true</customised></info><bookname="Ecclesiastes"tag="_Ecclesiastes"><chapternumber="1"tag="_1"><passagenumber="2"tag="_2">²“Meaningless!Meaningless!”saystheTeacher.
“Utterlymeaningless!Everythingismeaningless.”</passage></chapter></book></root>Note that you are allowed to write badly formed XML documents using this file interface, but they will cause runtime errors in your code upon trying to read and process them.CSV DownloaderThe CSV Downloader is effectively the same as the YAML Downloader except the resulting file is in CSV format and has a different file extension.frommeaninglessimportCSVDownloaderif__name__=='__main__':downloader=CSVDownloader()downloader.download_passage('Ecclesiastes',1,2)Output:Running the above code would produce a file calledEcclesiastes.csvin the current working directory with the approximate contents:Book,Chapter,Passage,Text,Language,Translation,Timestamp,Meaningless
Ecclesiastes,1,2,"² “Meaningless! Meaningless!”
    says the Teacher.
“Utterly meaningless!
    Everything is meaningless.”",English,NIV,0000-00-00T00:00:00.000000+00:00,0.0.0CSV ExtractorMuch like the YAML Extractor, the CSV Extractor uses the generated files from the CSV Downloader to find passages.frommeaninglessimportCSVExtractorif__name__=='__main__':bible=CSVExtractor()passage=bible.get_passage('Ecclesiastes',1,2)print(passage)Output:Assuming the CSV downloader has already generated a CSV file in the current directory calledEcclesiastes.csvwhich contains the book of Ecclesiastes in CSV format:² “Meaningless! Meaningless!”
    says the Teacher.
“Utterly meaningless!
    Everything is meaningless.”CSV File InterfaceThe CSV File Interface is a set of helper methods used to read and write CSV files. This is geared towards the CSV document format used by the CSV Downloader and Extractor and cannot be used to add custom attributes to the output file when writing CSV data.frommeaninglessimportCSVDownloader,csv_file_interfaceif__name__=='__main__':downloader=CSVDownloader()downloader.download_passage('Ecclesiastes',1,2)bible=csv_file_interface.read('./Ecclesiastes.csv')bible['Info']['Language']='English (EN)'csv_file_interface.write('./Ecclesiastes.csv',bible)Output:Running the above code would produce a file calledEcclesiastes.csvin the current working directory with the approximate contents:Book,Chapter,Passage,Text,Language,Translation,Timestamp,Meaningless
Ecclesiastes,1,2,"² “Meaningless! Meaningless!”
    says the Teacher.
“Utterly meaningless!
    Everything is meaningless.”",English (EN),NIV,0000-00-00T00:00:00.000000+00:00,0.0.0Text searching within filesAll file-based extractors support passage filtering by search text or by regular expression.The section below is a simple example that prints passages containing "meaningless" from Ecclesiastes 1:frommeaninglessimportYAMLDownloader,YAMLExtractorif__name__=='__main__':downloader=YAMLDownloader()downloader.download_chapter('Ecclesiastes',1)bible=YAMLExtractor()print(bible.find_text_in_chapter('meaningless','Ecclesiastes',1))Output:² “Meaningless! Meaningless!”
    says the Teacher.
“Utterly meaningless!
    Everything is meaningless.”
¹⁴ I have seen all the things that are done under the sun; all of them are meaningless, a chasing after the wind.The section below is another simple example that prints passages containing "meaningless" or "wisdom" from Ecclesiastes 1:frommeaninglessimportYAMLDownloader,YAMLExtractorif__name__=='__main__':downloader=YAMLDownloader()downloader.download_chapter('Ecclesiastes',1)bible=YAMLExtractor()print(bible.find_text_in_chapter('meaningless|wisdom','Ecclesiastes',1,is_regex=True))Output:² “Meaningless! Meaningless!”
    says the Teacher.
“Utterly meaningless!
    Everything is meaningless.”
¹³ I applied my mind to study and to explore by wisdom all that is done under the heavens. What a heavy burden God has laid on mankind!
¹⁴ I have seen all the things that are done under the sun; all of them are meaningless, a chasing after the wind.
¹⁶ I said to myself, “Look, I have increased in wisdom more than anyone who has ruled over Jerusalem before me; I have experienced much of wisdom and knowledge.”
¹⁷ Then I applied myself to the understanding of wisdom, and also of madness and folly, but I learned that this, too, is a chasing after the wind.
¹⁸ For with much wisdom comes much sorrow;
    the more knowledge, the more grief.Q&AHow to report potential bugs and other feedback?To report bugs and other problems, create an issue in this repo that details:A brief description of the problem encounteredSteps to recreate the problem (or a code sample that demonstrates the problem)Expected resultThe version of this library being usedIf you have any questions, complaints, compliments or even ideas to improve this library, you can also leave them as a GitHub issue with the appropriate label.
Or you can also send an email todantran.au@gmail.com, although a response will likely take longer than replying to a GitHub issue.Should I manually edit the downloaded file?This is NOT recommendedunder normal circumstances, as it may cause problems with the library API when using the modified file.If multiple translations are supported, why aren't there more unit tests for these?The Base Extractor and Downloader all use the same overall structure to represent passage contents for all translations.For the Web Extractor, the page structure of theBible Gatewaysite is mostly the same across different translations, so as long as the translation-specific differences are handled correctly, the same set of unit tests should suffice.Will this library ever drop support for certain translations?Short answer: It depends.The primary case for dropping support for a certain Bible translation is if there is an observed problem in the extracted Bible contents which is considerably difficult to address due to how the Biblical contents is structured (in terms of HTML) on theBible Gatewaysite. With that in mind, reinstating support for Bible translations can be reconsidered when such problems subside.How are omitted passages determined for each translation?Without having to go through every single passage and check if it is omitted, a set of common omitted passages are found here:https://en.wikipedia.org/wiki/List_of_New_Testament_verses_not_included_in_modern_English_translationshttp://textus-receptus.com/wiki/List_of_Omitted_Bible_Verses#List_of_Bible_verses_totally_omittedThese passages are checked on theBible Gatewaysite, and then added to the Base Downloader's internal list of omitted passages for the relevant translation.If you notice any problems such as unhandled omitted passages or incorrect tagging of an omitted passage in the Base Downloader, please create an issue to report it.Does this library provide support for the Apocrypha books?At the moment, you can use the Web Extractor'ssearch(),search_multiple()and other related functions to obtain passages from the Apocrypha books.
There is currently no official support for the Apocrypha books in the other downloaders and extractors, at least until the issues identifiedherebecome easier to work around.Contributorsdaniel-tran(Creator & current maintainer)To make a contribution to this library, refer toCONTRIBUTING.md.DisclaimerThis library is not endorsed by nor affiliated with Bible Gateway or any of its partners.Change Log1.0.0Removed legacy_xml_file_interface module and theuse_legacy_modeflag in the XML Downloader and ExtractorAdded experimental Apocrypha Web Extractor for the NRSVUE translationAdded translation support for: NMB0.7.0Refined xml_file_interface to use a more standard XML document structure with easier integration with XSLTAddeduse_legacy_modeflag to XML Downloader and Extractor to continue using the original behaviour and assist with transitioning to the updated XML file interfaceAdded legacy_xml_file_interface module for backward compatibility with XML files using the previous (deprecated) document structureAdded download timestamp and library version information to output filesDrop library support for Python 3.70.6.1Fixed an issue where the JSON file interface was writing Unicode characters incorrectly to output files0.6.0Added text searching functionality into Base ExtractorAdded a dedicated area for experimental functionality and helper scriptsWith enough refinement, some of these might become new features in future releasesAdded translation support for: NRSVUERemoved translation support for: CJB0.5.0Added CSV Extractor and CSV DownloaderAdded csv_file_interface module to assist with CSV file accessAdded translation support for: NIVUKDrop library support for Python 3.60.4.0Documentation is now publicly available through GitHub Pages. Seehttps://daniel-tran.github.io/meaningless/This was previously an undocumented change as part of the 0.3.0 release.Added XML Extractor and XML DownloaderAdded xml_file_interface module to assist with XML file accessAdded translation support for: RVA0.3.0Added initial continuous integration workflow with GitHub ActionsAdded translation support for: BRG, CJB, EHV, ESVUK, GNV, GW, ISV, JUB, NASB1995, NOGDrop library support for Python 3.50.2.0Added Base Extractor, Base Downloader, JSON Extractor and JSON DownloaderBase Extractor contains shared logic for both the YAML and JSON ExtractorsBase Downloader contains shared logic for both the YAML and JSON DownloadersAdded json_file_interface module to assist with generic JSON file accessAdded translation support for: ASV, AKJV, KJ21, LEB, MEV, NET, NLV, YLT0.1.0Initial release!Added Web Extractor, YAML Extractor and YAML DownloaderAdded yaml_file_interface module to assist with generic YAML file accessAdded translation support for: ESV, KJV, NASB, NIV, NKJV, NLT, NRSV, WEB