Package: argilla-plugins
Argilla Plugins🔌 Open-source plugins for extra features and workflowsWhy?The design of Argilla is intentionally programmable (i.e., developers can build complex workflows for reading and updating datasets). However, there are certain workflows and features which are shared across different use cases and could be simplified from a developer experience perspective. In order to facilitate the reuse of key workflows and empower the community, Argilla Plugins provides a collection of extensions to super power your Argilla use cases.
Some of this pluggable method could be eventually integrated into thecore of Argilla.Quickstartpipinstallargilla-pluginsfromargilla_plugins.datasetsimportend_of_lifeplugin=end_of_life(name="plugin-test",end_of_life_in_seconds=100,execution_interval_in_seconds=5,discard_only=False)plugin.start()How to develop a pluginPick a cool plugin from the list of topics or our issue overview.Think about an abstraction for the plugin as shown below.Refer to the solution in the issue.fork the repo.commit your codeopen a PR.Keep it simple.Have fun.Development requirementsFunctionWe want to to keep the plugins as abstract as possible, hence they have to be able to be used within 3 lines of code.fromargilla_plugins.topicimportpluginplugin(name="dataset_name",ws="workspace"query="query",interval=1.0)plugin.start()Variablesvariablesname,ws, andqueryare supposed to be re-used as much as possible throughout all plugins. Similarly, some functions might contain adaptations likename_fromorquery_from. Whenever possible re-use variables as much as possible.Ohh, and don`t forget to have fun! 🤓TopicsReportingWhat is it?Create interactive reports about dataset activity, dataset features, annotation tasks, model predictions, and more.Plugins:automated reporting pluging usingdatapane.issueautomated reporting pluging forgreat-expectations.issueDatasetsWhat is it?Everything that involves operations on adataset level, like dividing work, syncing datasets, and deduplicating records.Plugins:sync data between datasets.directional A->B.issuebi-directional A <-> B.issueremove duplicate records.issuecreate train test splits.issueset limits to records in datasetsend of life time.issuemax # of records.issueEnd of LifeAutomatically delete or discard records afterxseconds.fromargilla_plugins.datasetsimportend_of_lifeplugin=end_of_life(name="plugin-test",end_of_life_in_seconds=100,execution_interval_in_seconds=5,discard_only=False)plugin.start()Programmatic LabellingWhat is it?Automatically updateannotationsandpredictionslabels and predictions ofrecordsbased on heuristics.Plugins:annotated spans as gazzetteer for labelling.issuevector search queries and similarity threshold.issueuse gazzetteer for labelling.issuematerialize annotations/predictions from rules using Snorkel or a MajorityVoterissueToken CopycatIf we annotate spans for texts like NER, we are relatively certain that these spans should be annotated the same throughout the entire dataset. We could use this assumption to already start annotating or predicting previously unseen data.fromargilla_pluginsimporttoken_copycatplugin=token_copycat(name="plugin-test",query=None,copy_predictions=True,word_dict_kb_predictions={"key":{"label":"label","score":0}},copy_annotations=True,word_dict_kb_annotations={"key":{"label":"label","score":0}},included_labels=["label"],case_sensitive=True,execution_interval_in_seconds=1,)plugin.start()Active learningWhat is it?A process during which a learning algorithm can interactively query a user (or some other information source) to label new data points.Plugins:active learning forTextClassification.classy-classification.issuesmall-text.issueactive learning forTokenClassification.issuefromargilla_pluginsimportclassy_learnerplugin=classy_learner(name="plugin-test",query=None,model="all-MiniLM-L6-v2",classy_config=None,certainty_threshold=0,overwrite_predictions=True,sample_strategy="fifo",min_n_samples=6,max_n_samples=20,batch_size=1000,execution_interval_in_seconds=5,)plugin.start()Inference endpointsWhat is it?Automatically add predictions to records as they are logged into Argilla. This can be used for making it really easy to pre-annotated a dataset with an existing model or service.inference with un-authenticated endpoint.issueembed incoming records in the background.issueTraining endpointsWhat is it?Automatically train a model based on dataset annotations.TBDSuggestionsDo you have any suggestions? Pleaseopen an issue🤓