Package: dframcy
DframCyDframCy is a light-weight utility module to integrate Pandas Dataframe to spaCy's linguistic annotation and training tasks. DframCy provides clean APIs to convert spaCy's linguistic annotations, Matcher and PhraseMatcher information to Pandas dataframe, also supports training and evaluation of NLP pipeline from CSV/XLXS/XLS without any changes to spaCy's underlying APIs.Getting StartedDframCy can be easily installed. Just need to the following:RequirementsPython 3.6 or laterPandasspaCy >= 3.0.0Also need to download spaCy's language model:python-mspacydownloaden_core_web_smFor more information refer to:Models & LanguagesInstallation:This package can be installed fromPyPiby running:pipinstalldframcyTo build from source:gitclonehttps://github.com/yash1994/dframcy.gitcddframcy
pythonsetup.pyinstallUsageLinguistic AnnotationsGet linguistic annotation in the dataframe. For linguistic annotations (dataframe column names) refer tospaCy's Token APIdocument.importspacyfromdframcyimportDframCynlp=spacy.load("en_core_web_sm")dframcy=DframCy(nlp)doc=dframcy.nlp(u"Apple is looking at buying U.K. startup for $1 billion")# default columns: ["id", "text", "start", "end", "pos_", "tag_", "dep_", "head", "ent_type_"]annotation_dataframe=dframcy.to_dataframe(doc)# can also pass columns names (spaCy's linguistic annotation attributes)annotation_dataframe=dframcy.to_dataframe(doc,columns=["text","lemma_","lower_","is_punct"])# for separate entity dataframetoken_annotation_dataframe,entity_dataframe=dframcy.to_dataframe(doc,separate_entity_dframe=True)# custom attributes can also be includedfromspacy.tokensimportTokenfruit_getter=lambdatoken:token.textin("apple","pear","banana")Token.set_extension("is_fruit",getter=fruit_getter)doc=dframcy.nlp(u"I have an apple")annotation_dataframe=dframcy.to_dataframe(doc,custom_attributes=["is_fruit"])Rule-Based Matching# Token-based Matchingimportspacynlp=spacy.load("en_core_web_sm")fromdframcy.matcherimportDframCyMatcher,DframCyPhraseMatcher,DframCyDependencyMatcherdframcy_matcher=DframCyMatcher(nlp)pattern=[{"LOWER":"hello"},{"IS_PUNCT":True},{"LOWER":"world"}]dframcy_matcher.add("HelloWorld",None,pattern)doc=dframcy_matcher.nlp("Hello, world! Hello world!")matches_dataframe=dframcy_matcher(doc)# Phrase Matchingdframcy_phrase_matcher=DframCyPhraseMatcher(nlp)terms=[u"Barack Obama",u"Angela Merkel",u"Washington, D.C."]patterns=[dframcy_phrase_matcher.get_nlp().make_doc(text)fortextinterms]dframcy_phrase_matcher.add("TerminologyList",None,*patterns)doc=dframcy_phrase_matcher.nlp(u"German Chancellor Angela Merkel and US President Barack Obama "u"converse in the Oval Office inside the White House in Washington, D.C.")phrase_matches_dataframe=dframcy_phrase_matcher(doc)# Dependency Matchingdframcy_dependency_matcher=DframCyDependencyMatcher(nlp)pattern=[{"RIGHT_ID":"founded_id","RIGHT_ATTRS":{"ORTH":"founded"}}]doc=dframcy_dependency_matcher.nlp(u"Bill Gates founded Microsoft. And Elon Musk founded SpaceX")dependency_matches_dataframe=dframcy_dependency_matcher(doc)Command Line InterfaceDframcy supports command-line arguments for the conversion of a plain text file to linguistically annotated text in CSV/JSON format.
Previous versions of Dframcy were used to support CLI utilities for training and evaluation of spaCy models from CSV/XLS files.
After thev3release, spaCy's training pipeline has become much more flexible and robust so didn't want to introduce additional
step using Dframcy for just format conversion (CSV/XLS tospaCy’s binary format).# convertdframcydframe-iplain_text.txt-oannotations.csv-fcsv