Package: npttf2utf
npTTF2UTFPython module/script to map Nepali ASCII font faces like Preeti, Sagarmatha, and more to devanagari unicodeThis is mainly a python module to help in mapping of various nepali ASCII font faces to its unicode counterpart and unicode or any unicode mappable font back to Preeti font face. It currently supports directly mapping passed strings (fontmapper.py) as well font auto detection and selecting components to map for mapping for docx files (docxhandler.py) and plain text files (txthandler.py).Requirementspython3Installation1) Using pip - Pre packaged$ pip install npttf2utf
        or
$ pip3 install npttf2utfOr you can package it locally and install. Useful if you want to change something before installing2) Using pip - Packaged manually$ pip install build
    or
$ pip3 install build
$ git clone https://github.com/trippygeese/npttf2utf.git
$ cd npttf2utf
$ python3 -m buildThis will create a .whl file for installation in 'dist' directory. To install$ cd dist
$ pip install *.whl
    or
$ pip3 install *.whlUsage1) As CLI application$ npttf2utf -m [mode] -if [origin font] -of [output font] -dc [docx components] -kf [extra known unicode fonts] -i [input file/string] -o [output file] -mf [mapping defination]Parameters (Same for both) :ParameterHelp/Usage-h*Shows help and information about the program-v*Shows version information-mUsage mode. "string" to pass input string and output on console window, "docx" for working on docx files and "plain" for working with plaintext files-ifThe font face which  was used for the string or creating the file. In "docx" mode you can use "auto" to autodetect used fonts and map them-of*The font to which the string or file will be mapped to. Currently, supports 'unicode' and 'Preeti' . Defaults to 'Preeti' if unspecified-dc*The components of docx which will be processed during mapping. Components are separated by a comma ',' (Defaults to all supported components 'body_paragraph,table,shape')-kf*While converting docx to Preeti known nepali unicode will be mapped to preeti and other fonts will be ignored. Specify fonts seperated by comma to add more fonts to default list "Kalimati,Mangal,Noto Sans Devanagari"'-iInput string or path to input file-o*Path to output file. Not required for "string" mode-mf*Path to mapping definition file. If not passed it will look for "map.json" in current script directoryNote: The parameters marked with * are optionalExample usage:To pass string in Preeti in terminal and map it to unicode ("string" mode)$ npttf2utf -m string -if Preeti -of unicode -i "asdfghjk"It will map "asdfghjk" to unicode following mapping for Preeti and output "बकमानजवप".To pass string in unicode in terminal and map it to preeti ("string" mode)$ npttf2utf -m string -if unicode -of Preeti -i "सबिन आचार्य"It will map "सबिन आचार्य" to Preeti and output ";lag cfrf/\o" (You can also convert other non unicode fonts like Kantipur, Sagarmatha etc to Preeti).To convert docx or txt file ("plain"/"docx" mode) to unicode$ npttf2utf -m docx -if auto -of unicode -i "document_with_ASCII_font_faces.docx" -o "document_mapped_to_unicode.docx"It will map the content of document to unicode  and save it as "document_mapped_to_unicode.docx" ("auto" as font is available for "docx" mode only)To convert docx or txt file ("plain"/"docx" mode) to Preeti$ npttf2utf -m docx -if auto -of Preeti -i "document_with_ASCII_font_faces_or_unicode.docx" -o "document_mapped_to_preeti.docx" -dc "body_paragraph,table,shape" -kf "some,extra,unicode,fonts"It will map the content of document to unicode  and save it as "document_mapped_to_unicode.docx" ("auto" as font is available for "docx" mode only)2) As python module$ python
>> import npttf2utfClass: npttf2utf.FontMapper"npttf2utf.FontMapper" class can be used to map the fonts to their unicode counterpart. It is also the base for other document convertersMethod: __init __This method initializes the FontMapper classdef __init__(self, map_json):Returns: NoneArgumentDescriptionOptionalmap_jsonPath to mapping definition file (Must be readable by current user)FalseMethod: map_to_unicodeThis method maps the passed string to of defined origin font to unicode using the mapping definitiondef map_to_unicode(self, string, from_font="Preeti", unescape_html=False):Returns: StringArgumentDescriptionOptionalstringString to mapFalsefrom_fontThe origin font in which string was written. Defaults to "Preeti" if not passedTrueunescape_html_inputUnescape HTML entities in input string before mapping them (Bool) (Defaults to False)Trueescape_html_outputEscape HTML entities in mapped string before returning them (Bool) (Defaults to False)TrueExample usage:>> import npttf2utf
>> mapper = npttf2utf.FontMapper("npttf2utf/map.json")
>> mapper.map_to_unicode("asdfghjk", from_font="Preeti", unescape_html_input=False, escape_html_output=False)
बकमानजवप
>>Method: map_to_preetiThis method maps the passed Devanagari unicode string to preeti using the preetimapper.pydef map_to_preeti(self, string, from_font="Preeti", unescape_html=False):Returns: StringArgumentDescriptionOptionalstringString to mapFalsefrom_fontThe origin font in which string was written. Defaults to "Preeti" if not passedTrueunescape_html_inputUnescape HTML entities in input string before mapping them (Bool) (Defaults to False)Trueescape_html_outputEscape HTML entities in mapped string before returning them (Bool) (Defaults to False)TrueExample usage:>> import npttf2utf
>> mapper = npttf2utf.FontMapper("npttf2utf/map.json")
>> mapper.map_to_preeti("सबिन आचार्य", from_font="unicode", unescape_html_input=False, escape_html_output=False)
;lag cfrf/\o
>>Class: npttf2utf.DocxHandler"npttf2utf.DocxHandler" class can be used to map docx files to unicode and save themMethod: __init __This method initializes the DocxHandler class which can be used to map docx filesdef __init__(self, rules_file, default_unicode_font_name="Kalimati"):Returns: NoneArgumentDescriptionOptionalrules_filePath to mapping definition file (Must be readable by current user)Falsedefault_unicode_font_nameThe name of font which will be set for a converted segment of docx files. (Defaults to "Kalimati")TrueMethod: detect_used_fontsThis method returns list of fonts supported by mapping definition which are used in the docx filedef detect_used_fonts(self, docx_file_path):Returns: ListArgumentDescriptionOptionaldocx_file_pathPath to docx file whose fonts are to be detectedFalseMethod: map_fontsThis method maps the font in docx file and creates new docx file with mapping applieddef map_fonts(self, orginal_file_path, output_file_path="mapped.docx", from_font="auto", to_font="unicode", components=["body_paragraph", "table", "shape"], known_unicode_fonts=[]):Returns: NoneArgumentDescriptionOptionaloriginal_file_pathPath to docx file whose fonts are to be mappedFalseoutput_file_pathPath where the mapped docx file is to saved (Defaults to "mapped.docx")Truefrom_fontThe origin font in which string was written. (Defaults to "auto"). "auto" can be passed to detect used font automatically and map them accordingly and leave english characters untouchedTrueto_fontTarget for font conversion. (Defaults to "unicode"). Only "unicode" is supported as of nowTruecomponents[List] List of components of docx file which will be looked up for text contents. (Defaults to: ["body_paragraph", "table", "shape"]). "body_paragraph", "table" and "shape" are supported as of nowTrueknown_unicode_fonts[List] List of extra nepali unicode font that when detected will be mapped (Only used while mapping to Preeti)TrueExample usage:>> import npttf2utf
>> converter = npttf2utf.DocxHandler("npttf2utf/map.json", default_unicode_font_name="Kalimati")
>> converter.detect_used_fonts("document_with_ASCII_font_faces.docx")
["Preeti", "Sagarmaths"]
>> converter.map_fonts("document_with_ASCII_font_faces.docx", output_file_path="mapped_document.docx", from_font="auto", to_font="unicode", components=["body_paragraph", "table"], known_unicode_fonts=["some", "extra", "nepali", "font"])
>>Class: npttf2utf.TxtHandler"npttf2utf.TxtHandler" class can be used to map plain text files to unicode and save themMethod: __init __This method initializes the TxtHandler class which can be used to map txt filesdef __init__(self, rules_file):Returns: NoneArgumentDescriptionOptionalrules_filePath to mapping definition file (Must be readable by current user)FalseMethod: map_fontsThis method maps the font in txt file and creates new txt file with mapping applieddef map_fonts(self, orginal_file_path, output_file_path="mapped.txt", from_font="Preeti", to_font="unicode", components=[], known_unicode_fonts=[]):Returns: NoneArgumentDescriptionOptionaloriginal_file_pathPath to txt file whose fonts are to be mappedFalseoutput_file_pathPath where the mapped txt file is to saved (Defaults to "mapped.txt")Truefrom_fontThe origin font in which string was written. (Defaults to "Preeti").Trueto_fontTarget for font conversion. (Defaults to "unicode"). Only "unicode" is supported as of nowTruecomponentsServes no purpose, just there to match the method call of DocxHandlerTrueknown_unicode_fontsServes no purpose, just there to match the method call of DocxHandlerTrue>> import npttf2utf
>> converter = npttf2utf.TxTHandler("npttf2utf/map.json")
>> converter.map_fonts("txt_with_ASCII_font_faces.txt", output_file_path="mapped_txt.txt", from_font="Preeti", to_font="unicode", components=[], known_unicode_fonts=[])
>>Supported docx componentsText content in Text boxes/ShapesGeneral paragraphsText content in tableSupported ASCII font facesPreetiSagarmathaKantipurFONTASY_HIMALI_TTPCS NEPALISupported Output fontsDevanagari UnicodeTodosAdd support for headers/footersOptimize the codeAbility to unify fonts to Preeti as wellAdding support for new file typeCreate a new file to handle the file type. (You can use docxhandler.py and modify it as needed). The class constructor should take map definition as the first parameter and the file handler class should contain "map_fonts" methods that take original user file, the path for converted file, from the font, to font, and a list of components as arguments. You can map a string to Unicode by using FontMapper.map_to_unicode if the mapping for origin font exists in the definition. (map_to_unicode takes unescape_html argument to that can be used to unescape HTML string before processing and escape it before returning)Adding mapping for a new fontOpen "src/npttf2utf/map.json" and add a JSON key with this structure"font_name":{
  "version": "v1",
  "rules": {
    "character-map": {
      "character-in-origin-font": "unicode-equivalent-character"
    },
    "pre-rules": [
      ["regex-string-search", "regex-string-replace"]
    ],
    "post-rules": [
      ["regex-string-search", "regex-string-replace"]
    ]
  }
}pre-rules     - Regex find and replace to apply before mapping characters to Unicode
character map - Directly mappable character from source font to Unicode. For Preeti: a <-> ब
post-rules    - The words may not be as expected directly after mapping. So this contains regexes to find them and replace them with corrections (Regex find and replace to apply before mapping characters to Unicode)Feel free to use this project for any purpose and long as you comply with the license. Any contribution to the project is highly appreciated. If you find any bugs please report it