Package: execdmscript
execdmscriptA Python module for executing DM-Script from Python inGatan Microscopy Suite (GMS) (Digital Micrograph) Version 3.4 or later.Table of ContentsForewordUsageStep by step examplesHello worldGetting startedOuthouse codeTagGroups and TagListsDebug your codeMultiple scriptsMore examplesOne glance exampleHelper functionsType mappingEscaping codeConverting TagGroupsGet global tagRemove global tagsInstallationLicense and PublicationsForewordThis module is created because I needed to use DM-Scripts Dialogs in my project. But this
project was written in Python. Because there is no Python implementation for dialogs I
decided to execute DM-Script that creates the dialogs. But then getting data from one to
the other programming language was more difficult than I thought. This module tries to
solve those problems and hide verything away.UsageImportant:This module is for running DM-Script scriptswithin and only withinthe Python
interpreter of a running GMS (DigitalMicrograph). So you must only use this when you
are in a Python script window in GMS (DigitalMicrograph). Thisdoes not work in the
command line!After theinstallationyou can execute the following examples. Check out
theone glance exampleif you used this module already and just
forgot how it worked. If you have never dealt with Python in GMS or you are new toexecdmscript, check out thestep by step examplesbelow.Step by step examplesThe following examples are taken from theexamplesdirectory which can also be found on
thegithub page.Example 0: Hello WorldBecause of tradition, let's do the hello world first. Obviously this is a lot more
complicated than actually needed, but it teaches the basics.importexecdmscript# set the text in pythonworld_text="Hello World!"# create an executable dm-script codedmscript="""OKDialog(text);result(text);"""# save which variables should be passed from python to dm-script and how they should be# calledsetvars={"text":world_text}# execute the scriptwithexecdmscript.exec_dmscript(dmscript,setvars=setvars):passExample 1: Getting startedAfter theinstallationopen GMS (DigitalMicrograph). Create a new script
window (Ctrl+K) and make sure it is set toPythonas the
scripting language. For this example it does not matter if the script is executed in the
background or not. Now type in (or copy) the following code:importexecdmscript# `a` and `b` are given in pythona=10b=20# This is the dm-script to executedmscript="number c = a + b;"# This are the variables the upper dm-script will knowsetvars={"a":a,"b":b}# This are the variables this python script will know after the executionreadvars={"c":int}withexecdmscript.exec_dmscript(dmscript,setvars=setvars,readvars=readvars)asscript:# now we can access `c` because it is set in the `readvars`print(script["c"])This script calculatesa+b. Butaandbare Python variables whilecis a
variable created in dm-script.Thesetvarsis adictthat takes the variable name as the key and the variable value
as its value. These values with their names will be accessable in the executed dm-script.
Values of the typesint,float,bool,str,listanddictare
supported. They will be converted to their dm-script equivalent. This meansint,floatandboolwill be of the typenumber.strwill bestring. Andlistanddictwill both beTagGroups but the former will be aTagListand the latter will be a
"true"TagGroup.Thereadvarsis adictthat takes the dm-script variable name as the key and the type
to expect as the value. This means that all the variables that have a key in thereadvarsand are defined in the executed dm-script will later be accessable in Python.Example 2: Outhouse codeImportant:When you are trying the example code, make sure you always know the complete path of
the dm-script *.s file in your Python script. It is not enough to have the file name
only!Now let's go a little bit more complicated. There is no need to execute a sum operation in
dm-script.This time we will create two files. The first one will be a dm-script file, the second one
will be a Python file again.Open GMS and create a new dm-script window. Enter the following code and save it to
somewhere where it is easy to find, sayC:\testdmscript2.s.TagGroupdlg,dlg_items,field;dlg=DLGCreateDialog("Headlines of website",dlg_items);// Note that the `text` is neither initialized nor declared, that is important as it will// be done by the python script!field=DLGCreateLabel(text);field.DLGWidth(100);field.DLGHeight(4);dlg.DLGAddElement(field);alloc(UIFrame).init(dlg).pose();As said, save the file somewhere, sayC:\testdmscript2.s.Now open a new Python script window in GMS. Create the following code:importexecdmscriptimporturllib.requestimporthtml.parserimportre# get the text of example.comcontent=str(urllib.request.urlopen("https://www.example.com/").read())# get all headlinesmatches=re.findall(r"<h([\d])>([^<]+)</h\1>",content)ifmatchesisnotNone:headlines=[x[1]forxinmatches]text="Headlines:\n- "+"\n- ".join(headlines)else:headlines=[]text="*No headlines found*"# Tell the dm-script the variables it should knowsetvars={"text":text}# set your filepath, needs to be the complete path, not just the name!path=r"C:\testdmscript2.s"withexecdmscript.exec_dmscript(path,setvars=setvars):passThis will create a dm-script dialog that shows all headlines of the website stated in the
code, in this caseexample.com.This time the dm-script is in a separate file which is strongly recommended. This way the
dm-script file can be debugged a lot easier. Also the code is better to read and
structured in a better way.Also as one can see, this are just a few Python lines but will be very hard to get with
pure dm-script. On the other hand there no GMS-dialogs in Python.Example 3: TagGroups and TagListsImportant:When you are trying the example code, make sure you always know the complete path of
the dm-script *.s file in your Python script. It is not enough to have the file name
only!While Python haslists anddicts (and a lot more of course) for dealing with multiple
values, dm-script does that withTagLists andTagGroups. Both are more less equivalent
even though the handling is very different. Thereforeexecdmscriptconverts both types
automatically.Open GMS again and create a new Python script window. Create the following code:TagGroupdlg,dlg_items,wrapper,inputs;dlg=DLGCreateDialog("Add text to the headlines",dlg_items);wrapper=DLGCreateGroup();wrapper.DLGTableLayout(2,headlines.TagGroupCountTags(),0);inputs=NewTagList();for(numberi=0;i<headlines.TagGroupCountTags();i++){stringtext;if(headlines.TagGroupGetIndexedTagAsString(i,text)){TagGrouplabel=DLGCreateLabel("Text for "+text,25);wrapper.DLGAddElement(label);TagGroupinput=DLGCreateStringField("");input.DLGIdentifier("input-"+i);wrapper.DLGAddElement(input);inputs.TagGroupInsertTagAsTagGroup(infinity(),input);}}dlg.DLGAddElement(wrapper);objectdialog=alloc(UIFrame).init(dlg);// make sure the variable always exists, it may be empty but// it has to be declared!TagGroupheadline_texts=NewTagList();if(dialog.pose()){for(numberi=0;i<headlines.TagGroupCountTags();i++){stringtext;if(headlines.TagGroupGetIndexedTagAsString(i,text)){TagGroupinput;inputs.TagGroupGetIndexedTagAsTagGroup(i,input);TagGroupvals=NewTagGroup();vals.TagGroupCreateNewLabeledTag("headline");vals.TagGroupSetTagAsString("headline",text);vals.TagGroupCreateNewLabeledTag("text");vals.TagGroupSetTagAsString("text",input.DLGGetStringValue());headline_texts.TagGroupInsertTagAsTagGroup(infinity(),vals);}}}This creates a dialog that shows all headlines in theheadlinesTagListand allows to
add texts to them. Save this file with any name, sayC:\testdmscript3.s.Now open a new Python script window in GMS. Create the following code:importexecdmscriptheadlines=["Section 1","Section 2","Section 3"]# Tell the dm-script the variables it should knowsetvars={"headlines":headlines}# Get the list of headlinesreadvars={"headline_texts":list}# set your filepath, needs to be the complete path, not just the name!path=r"C:\testdmscript3.s"text=""withexecdmscript.exec_dmscript(path,setvars=setvars,readvars=readvars)asscript:forsectioninscript["headline_texts"]:text+="**{}**\n{}\n\n".format(section["headline"],section["text"])iftext!="":print(text)else:print("Could not find any sections.")Thescript["headline_texts"]is a list containingdicts becauseTagLists are
converted tolists automatically andTagGroups todicts.Example 4: Debug your codeThe downside of using combined scripts is that one cannot really see what the dm-script
interpreter is actually doing. For this problem there is a debug mode. Simply useexecdmscript.exec_dmscript(code, debug=True, debug_file=file_path)to get the dm-script
file that normally would be executed.Let's assume that the fileC:\testdmscript4.sexists and contains any code (if you want
you can create this file and see what happens). To debug the complete file check out the
following example Python code:importexecdmscript# Tell the dm-script the variables it should knowsetvars={"variable1":1,"variable2":"B","variable3":False}# Get the list of headlinesreadvars={"variable4":str,"variable5":int}# set your filepath, needs to be the complete path, not just the name!path=r"C:\testdmscript4.s"withexecdmscript.exec_dmscript(path,setvars=setvars,readvars=readvars,debug=True,debug_file=r"C:\debugfile.s")asscript:passThis code will not execute dm-script code! But it will instead create the file atC:\debugfile.sthat includes all the synchronization mechanisms that thesetvarsandreadvarsneed to work. This way you may find and fix bugs. Note that thedebug_fileis ignored whendebugis notTrue.To be able to run the code with and without python, one can use the start and end markers
to tellexecdmscriptto ignore the current part. In fact, this part will simply be
commented out before execution.Create the following code and save it asC:\testdmscript4.2.s.// @execdmscript.ignore.start// all the code here will never be executed except this// dm-script code is executed manuallystringvariable1="Test";numbervariable2=1;result("This will not be printed when executed via python.\n")// @execdmscript.ignore.endresult(variable1+"\n");result(variable2+"\n");// @execdmscript.ignore.startresult("Ignored again")// @execdmscript.ignore.endThis is a valid dm-script file. It can be executed as it is. But as the comment already
sais, the python execution viaexecdmscriptwill not see the lines between@execdmscript.ignore.startand@execdmscript.ignore.end.One can see this by executing the above file in GMS. Then create the following python file
and execute it in GMS.importexecdmscript# Tell the dm-script the variables it should knowsetvars={"variable1":"Executed from python","variable2":99999}# set your filepath, needs to be the complete path, not just the name!path=r"C:\testdmscript4.2.s"withexecdmscript.exec_dmscript(path,setvars=setvars):passThe variables have the values as defined in python. But there is no error, even though
the variables were defined already. In addition, theresult()function calls between
the ignore markers are not executed.Example 5: Multiple scriptsSometimes it is useful to structure your dm-script code too, especially if you have bigger
files to include. You can add as many dm-scripts directly or as files as you want. Note
that thereadvarsand thesetvarswill be defined before and after all scripts. So you
must not use variable names in one of the scripts that are in thesetvarsand all
variables in thereadvarshave to be delcared in exactly one file (not allreadvarshave to be in the same file.Create the dm-script fileC:\testdmscript5.swith the following content:numbere=d+b;Now create a new Python script with this code:importexecdmscriptdmscript1="number c = a + b;"dmscript2="number d = c + a;"dmscript3=r"C:\testdmscript5.s"setvars={"a":1,"b":2}readvars={"c":int,"d":int,"e":int}withexecdmscript.exec_dmscript(dmscript1,dmscript2,dmscript3,setvars=setvars,readvars=readvars)asscript:print("c:",script["c"])print("d:",script["d"])print("e:",script["e"])More ExamplesFor more examples check out the github page. There you can find the following example
scripts with more complicated behaviour:example_separate_thread.py: Show theseparate_threadparameter ofexecdmscript.exec_dmscript()by creating a progress dialog with dm-script that shows
the progress a python thread doesexample_combined.py: Show multiple scripts with a more complicated situation, this
basically takes all above scenes in one exampleOne glance exampleThe following example shows the basic usage:try:fromexecdmscriptimportexec_dmscript# some script to executescript="OKDialog(start_message)"script_file="path/to/script.s"# variables that will be defined for the scripts (and readable later on in Python)sv={"start_message":"Starting now!"}# variables the dm-script defines and that should be readable in the Python filerv={"selected_images":list,"options":"TagGroup","show_message":"nUmBeR"}withexec_dmscript(script,script_file,readvars=rv,setvars=sv)asscript:# all variables can be accessed via indexing `script` or by using# `script.synchronized_vars`, note that `script` is also iterable like a dictprint(script["start_message"])print(script["selected_images"])print(script["options"])print(script["show_message"])exceptExceptionase:# dm-script error messages only show the error type but not the messageprint("Exception: ",e)importtracebacktraceback.print_exc()Helper functionsexecdmscriptdefines some helper functions. Those provide often used and needed
functions related to executing dm-script code from Python.Type mappingTo find the Python type for a dm-script type or the other way around,execdmscriptprovides theget_dm_type()and theget_python_type()functions. Both functions take
a "type expression" and return the corresponding type in the specified language. Theget_dm_type()takes a python type and returns the type for dm-script,get_python_type()takes the dm-script type and returns the python type.get_python_type()will return a pythontypeobject and take strings as arguments.get_dm_type()takes both, strings and pythontypeobjects and returns strings.The following example shows the usage:importexecdmscriptexecdmscript.get_dm_type(int)# returns "number"execdmscript.get_dm_type(str)# returns "string"execdmscript.get_dm_type("text")# returns "string"execdmscript.get_dm_type(list)# returns "TagGroup"execdmscript.get_dm_type(dict)# returns "TagGroup"execdmscript.get_dm_type(bool,for_taggroup=False)# returns "number"execdmscript.get_dm_type(bool,for_taggroup=True)# returns "Boolean"execdmscript.get_python_type("int")# returns <class 'int'>execdmscript.get_python_type("Integer")# returns <class 'int'>execdmscript.get_python_type("TagGroup")# returns <class 'dict'>execdmscript.get_python_type("TagList")# returns <class 'list'>Theget_dm_type()supports thefor_taggroupparameter. This toggles whether the
returned string can be used inTagGroupSetTagAs...()or if it is the type definition to
create a new variable in dm-script.Escaping codeWhen creating dm-script code dynamically, which is often the case when using dm-script
code form Python, there can be problems with escaping. For thisexecdmscriptcontains
theescape_dm_string()function. This function escapes strings that should be included
in a string in dm-script. Theescape_dm_variable()function escapes variable names in
a way that they are allowed in dm-script code.importexecdmscriptvar="invalid variable name"text="""This is "a"veryproblematic text"""var=execdmscript.escape_dm_variable(var)text=execdmscript.escape_dm_string(text)dm_code="string{}=\"{}\"".format(var,text)print(dm_code)Using those two functions changes the dm-script code form (without escaping)stringinvalidvariablename="This is "a"veryproblematictext"to the valid dm-script codestringinvalid_variable_name="This is\"a\"\nvery\nproblematic text"Converting TagGroupsTo convert dm-scriptTagGroups from and todicts orlists,execdmscriptoffers theconvert_to_taggroup()and theconvert_from_taggroup()functions. Note that those
functions only work withTagGroups created in Python. Persistent tags are not supported.
Copying persitent tags to a local variable is not possible (GMS 3.4).convert_from_taggroup()takes aPy_TagGroupas a parameter and returns a dict or list.
NestedTagGroups are supported. The following example shows the usage:importpprintimportDigitalMicrographasDMimportexecdmscript# create a test TagGrouptaggroup=DM.NewTagGroup()taggroup.SetTagAsString("key1","First text value")taggroup.SetTagAsFloat("key2",-823.83)taggroup2=DM.NewTagGroup()taggroup2.SetTagAsString("inner-key1","Another text")taggroup2.SetTagAsBoolean("inner-key2",False)taggroup.SetTagAsTagGroup("key3",taggroup2)taggroup3=DM.NewTagList()taggroup3.InsertTagAsText(0,"Value in list")taggroup3.InsertTagAsText(1,"Next value in list")taggroup3.InsertTagAsLong(2,1234)taggroup.SetTagAsTagGroup("key4",taggroup3)# convert to a dictdict_data=execdmscript.convert_from_taggroup(taggroup)# show the converted dictpprint.pprint(dict_data)# convert to a listlist_data=execdmscript.convert_from_taggroup(taggroup3)pprint.pprint(list_data)convert_to_taggroup()is the opposite ofconvert_from_taggroup(). It takes a python
(nested)dictorlistand creates aPy_TagGroup.importDigitalMicrographasDMimportexecdmscriptdata={'key1':'First text value','key2':-823.8300170898438,'key3':{'inner-key1':'Another text','inner-key2':False},'key4':['Value in list','Next value in list',1234]}taggroup=execdmscript.convert_to_taggroup(data)taggroup.OpenBrowserWindow(False)The following code shows an example application. It allows to modify image tags by the
user. SinceTagGroups cannot be travelled through in Python code, this is the only
possibility to show the current tag structure (which is unknown because it comes from the
camera) to the user. Adding tags shows another example for the use ofexecdmscript.exec_dm_script().try:importtimeimportpprintimportnumpyasnpimportDigitalMicrographasDMimportexecdmscriptdefrecordImage()->DM.Py_Image:"""Record an image with the attatched camera.This is a dummy implementation and creates a randomimage with random tags."""# create random data within [0..255]img_data=np.random.rand(64,64)img_data=(img_data*255).astype(np.uint8)# create Py_Imageimg=DM.CreateImage(img_data)# set some tagsimg.GetTagGroup().SetTagAsFloat("Acquire time",time.time())returnimg# record the imageimg=recordImage()# the tags to set to the imagetags={}# prepare a dialogdm_code="number add_tag = TwoButtonDialog(\"Do you want to add more tags?\\nCurrent Tags:\\n{}\",\"Add Tag\",\"Done\");"# whether to add another tagadd_tag=Truewhileadd_tag:# ask for the tag name and valuetag_name=input("Please enter a tag name to add to the image")tag_value=input("Please enter the value for the tag '{}'".format(tag_name))tags[tag_name]=tag_value# format the current tags, escape quotes to prevent destroying the string in# the dm-script codetag_str=execdmscript.escape_dm_string(pprint.pformat(tags))# ask whether to add another tagadd_tag=Falsewithexecdmscript.exec_dmscript(dm_code.format(tag_str),readvars={"add_tag":int})asscript:add_tag=script["add_tag"]# convert the tags to a tag group objecttags=execdmscript.convert_to_taggroup(tags)# apply the tag group object to the imageimg.GetTagGroup().DeleteAllTags()img.GetTagGroup().CopyTagsFrom(tags)img.ShowImage()exceptExceptionase:# dm-script error messages only show the error type but not the messageprint("Exception: ",e)importtracebacktraceback.print_exc()Get global tagFor accessing the persistent global tagsexecdmscriptprovides theget_persistent_tag()function. This function allows to directly access the persistent tags and receive the
value in the Python environment already converted to the python variable type. This
includes the base types plusTagGroups andTagLists which are automatically converted
todicts andlists.Thepathparameter of theget_persistent_tag()function specifies which value to get.
If nopathis given, the current global tags will completely be returned as a Pythondict. Note that updating this dict does not change the persistent tags!The following example shows how this works:importexecdmscript# get a tag value by the path with : as a separatorprogram_name=execdmscript.get_persistent_tag("Private:Configuration:ApplicationName")# get a tag value by using a tuple with the path componentsprogram_version=execdmscript.get_persistent_tag(("Private","Configuration","ApplicationVersion_2"))print("This is{}with version{}.".format(program_name,program_version))# get another valueprint("The current save path is{}.".format(execdmscript.get_persistent_tag("Private:Current Directory")))# get a TagGroup which is automatically converted to a dictsettings_dict=execdmscript.get_persistent_tag("Private:CreateNewDialog")print("When creating a new image, the following settings apply:")forname,valinsettings_dict.items():print("{}:{}".format(name,val))# when the tag does not exist, a KeyError is raisedtry:execdmscript.get_persistent_tag("This:persistent:tag:does:not:exist")exceptKeyErrorase:print("This tag does not exist:{}".format(e))Remove global TagsSometimes it is necessary or convenient to set values to global tags.execdmscriptitself makes heavy use of this. To remove the remaining global tag after performing the
synchronization,execdmscriptoffers theremove_global_tag()function sice this does
not work with plain python.This function can be used like the following:importDigitalMicrographasDMimportexecdmscripttagname="new_global_tag"DM.GetPersistentTagGroup().SetTagAsString(tagname,"test value")# shows that the tag with the tagname existsDM.GetPersistentTagGroup().OpenBrowserWindow(False)execdmscript.remove_global_tag(tagname)#  the global tag with the tagname is removed againDM.GetPersistentTagGroup().OpenBrowserWindow(False)Example execution without installationIf you want to try out the module or if you don't want to install it, make sure to add the
import path tosys.path. You can add the path manually:importosimportsys# add the path to the execdmscript directory (so in execdmscript-dir there is the file# __init__.py and the file execdmscript.py)sys.path.insert(0,"path/to/execdmscript-dir/")If you only know the path relatively to your executing file, you can find the__file__(which does not exist in GMS) like the following code:try:importDigitalMicrographasDMin_digital_micrograph=TrueexceptImportError:in_digital_micrograph=Falsefile_is_missing=Falsetry:if__file__==""or__file__==None:file_is_missing=TrueexceptNameError:file_is_missing=Trueifin_digital_micrographandfile_is_missing:# the name of the tag is used, this is deleted so it shouldn't matter anywayfile_tag_name="__python__file__"# the dm-script to execute, double curly brackets are used because of the# python format functionscript=("\n".join(("DocumentWindow win = GetDocumentWindow(0);","if(win.WindowIsvalid()){{","if(win.WindowIsLinkedToFile()){{","TagGroup tg = GetPersistentTagGroup();","if(!tg.TagGroupDoesTagExist(\"{tag_name}\")){{","number index = tg.TagGroupCreateNewLabeledTag(\"{tag_name}\");","tg.TagGroupSetIndexedTagAsString(index, win.WindowGetCurrentFile());","}}","else{{","tg.TagGroupSetTagAsString(\"{tag_name}\", win.WindowGetCurrentFile());","}}","}}","}}"))).format(tag_name=file_tag_name)# execute the dm scriptDM.ExecuteScriptString(script)# read from the global tags to get the value to the python scriptglobal_tags=DM.GetPersistentTagGroup()ifglobal_tags.IsValid():s,__file__=global_tags.GetTagAsString(file_tag_name);ifs:# delete the created tag againDM.ExecuteScriptString("GetPersistentTagGroup()."+"TagGroupDeleteTagWithLabel(\"{}\");".format(file_tag_name))else:del__file__try:__file__exceptNameError:# set a default if the __file__ could not be received__file__=""importosimportsysif__file__!="":# add the parent directory to the system path so the execdmscript file# can be importedbase_path=str(os.path.dirname(os.path.dirname(__file__)))ifbase_pathnotinsys.path:sys.path.insert(0,base_path)importexecdmscriptThe upper code works for file structures like+ base
|   + execdmscript
|   |   - __init__.py
|   |   - execdmscript.py
|   + code
|   |   - your-file-with-the-upper-code.pyInstallationVia PIP (Recommended)You can installexecdmscriptviaPyPI. To
install it into the GMS virtual Python environment execute the following commands:conda activate GMS_VENV_Python
Python -m pip install execdmscriptManuallyTo manually installexecdmscriptdownload theexecdmscriptdirectory (the one that
contains the__init__.pyand theexecdmscript.py) and install themRecommended if manuallyin one of the GMS plugin directories byusing theFileMenu and then click onInstall Script File(Check out GMS Help in the chapterScripting>Installing Scripts and Plugins)orplacing theexecdmscriptdirectory in the plugin directory manually. Plugin
directories areC:\Users\UserName\AppData\Local\Gatan\Pluginsor inC:\InstallationDir\Gatan\Plugins. To find all plugin directories execute the
following code:stringdirs="Plugin Directories:\n\n";stringdir;for(numberi=1011;i>=1008;i--){try{dir=GetApplicationDirectory(i,0);dirs+=dir+"\n"}catch{break;}}result(dirs);OKDialog(dirs);orin the miniconda plugin directory (On windows normally in%ProgramData%\Miniconda3\envs\GMS_VENV_PYTHON\Lib\site-packages, then place theexecdmscriptdirectory here)License and PublicationsThis module is licensed underMozilla Public License.This means you can use the code for whatever you want. But please do not publish my code
as your work.If you want to publish papers, do so. There is no need to cite me. If you still want to
cite me (for any reason), please contact me via Github. For any questions please also
contact me on Github.