Package: unicodetext
unicodetextProcessing Unicode Text🌍中文How To UseUnicode's blocks (unicodetext.UnicodeBlocks), which can be quickly accessed through the IDE's auto-completion feature to obtain all the characters within the block.importunicodetextprint(unicodetext.UnicodeBlocks.Emoticons)Unicode's categories (unicodetext.UnicodeCategories), which can be quickly accessed through the IDE's auto-completion feature to obtain all the characters within the category.importunicodetextprint(unicodetext.UnicodeCategories.Punctuation)Character extraction.importunicodetexttext="i am here 237 .! 3 *。、！ +-23689068 发斯蒂芬改 23579 😄"print(unicodetext.extract_emoticon(text))Custom extraction can also be performed by providing specific Unicode blocks or categories:importunicodetexttext="i am here 237 .! 3 *。、！ +-23689068 发斯蒂芬改 23579 😄"print(unicodetext.extract_chr(text,chrs=unicodetext.UnicodeCategories.Symbol))Character removalimportunicodetexttext="i am here 237 .! 3 *。、！ +-23689068 发斯蒂芬改 23579 😄"print(unicodetext.remove_punctuation(text))The removed characters can be replaced with specified characters byreplace_str:importunicodetexttext="i am here 237 .! 3 *。、！ +-23689068 发斯蒂芬改 23579 😄"print(unicodetext.remove_punctuation(text,replace_str='[del]'))Custom removal can also be performed by providing specific Unicode blocks or categories:importunicodetexttext="i am here 237 .! 3 *。、！ +-23689068 发斯蒂芬改 23579 😄"print(unicodetext.remove_chr(text,chrs=unicodetext.UnicodeCategories.Symbol))Unicode character setunicodetext.UnicodeBlocksUnicode 15.1 defines328 blocks. To enable automatic code completion for block names as variables, we remove all spaces and hyphens from their names.unicodetext.UnicodeCategoriesLetter = Lu | Ll | Lt | Lm | LoMark = Mn | Mc | MeNumber = Nd | Nl | NoPunctuation = Pc | Pd | Ps | Pe | Pi | Pf | PoSymbol = Sm | Sc | Sk | SoSeparator = Zs | Zl | ZpOther = Cc | Cf | Cs | Co | CnCased_Letter = Lu | Ll | LtAbbrLongDescriptionLuUppercase_Letteran uppercase letterLlLowercase_Lettera lowercase letterLtTitlecase_Lettera digraph encoded as a single character, with first part uppercaseLmModifier_Lettera modifier letterLoOther_Letterother letters, including syllables and ideographsMnNonspacing_Marka nonspacing combining mark (zero advance width)McSpacing_Marka spacing combining mark (positive advance width)MeEnclosing_Markan enclosing combining markNdDecimal_Numbera decimal digitNlLetter_Numbera letterlike numeric characterNoOther_Numbera numeric character of other typePcConnector_Punctuationa connecting punctuation mark, like a tiePdDash_Punctuationa dash or hyphen punctuation markPsOpen_Punctuationan opening punctuation mark (of a pair)PeClose_Punctuationa closing punctuation mark (of a pair)PiInitial_Punctuationan initial quotation markPfFinal_Punctuationa final quotation markPoOther_Punctuationa punctuation mark of other typeSmMath_Symbola symbol of mathematical useScCurrency_Symbola currency signSkModifier_Symbola non-letterlike modifier symbolSoOther_Symbola symbol of other typeZsSpace_Separatora space character (of various non-zero widths)ZlLine_SeparatorU+2028 LINE SEPARATOR onlyZpParagraph_SeparatorU+2029 PARAGRAPH SEPARATOR onlyCcControla C0 or C1 control codeCfFormata format control characterCsSurrogatea surrogate code pointCoPrivate_Usea private-use characterCnUnassigneda reserved unassigned code point or a noncharacterInstallInstall the library with:pipinstall-UunicodetextYou can also clone this repository and install:gitclonehttps://github.com/huang22/unicodetext.gitcdunicodetext
pipinstall.