Package: cdo
Cdo.{rb,py} - Use Ruby/Python to access the power of CDOWelcome to the scripting interfaces ofCDO!
This repository contains interfaces forRubyandPython. If you are not sure, wether this is useful or not, please have a look at:Why the .... should I use this???What's going onCurrently this package is in a re-design phase. The target is a 2.0 release that willnot be compatiblewith the exising release 1.5.x:Write operator chains like methods chains with.as much as possiblehopefully reduce the number ofkwargskeyskeep the Ruby and Python interface similarpossibly drop python-2.x support ... I am not sure when to do this bestInstallationReleases are distributed viapypiandrubygems:Rubygem install cdo (--user-install)Pythonpip install cdo (--user)
    conda -c conda-forge install python-cdoRequirementsCdo.{rb,py} requires a working CDO binary and Ruby 2.x or Python 2.7/3.xPLEASE NOTE: python-2.7 is unmaintained since January 2021Many dependencies dropped support for 2.7 so I do manual testing with it,only.Multi-dimensional arrays (numpy for python, narray for ruby) require addtional
netcdf-io modules. These arescipyorpython-netcdf4for python andruby-netcdffor ruby. Because scipy has some difficulties with netcdf, I dropped the support of it with release 1.5.0.Thx to Alexander Winkler there is also an IO option for XArray.UsageYou can find a lot of examples in the unit tests for both languages. Here are the direct links to theruby testsand thepython tests.The following describes the basic features for both languagesRun operatorsBefor calling operators, you have to create an object first:cdo=Cdo.new#rubycdo=Cdo()#pythonPlease check the documentation for constructor paramaters. I try to have equal interfaces in both languages for all public methods.Choose CDO binaryBy default the cdo-bindings use the 'cdo' binary found in your $PATH variable. To change that, you canload a module before calling your script(module commandor another package manager likecondaorspack)use the CDO environment variable to set the path to be useduse the python/ruby methodcdo.setCdo('/path/to/the/CDO/executable/you/want'). By this technique you can create different objects for different CDO versions.DebuggingFor debugging purpose, both interfaces provide a "debug" attribute. If it is set to a boolian true, the complete commands and the return values will be printed during executioncdo.debug=true#rubycdo.debug=True#pythonThe default is false of cause.File informationcdo.infov(input:ifile)#rubycdo.showlevels(input:ifile)cdo.infov(input=ifile)#pythoncdo.showlevels(input=ifile)Operators with user defined regular output filescdo.timmin(input:ifile,output:ofile)#rubycdo.timmin(input=ifile,output=ofile)#pythonBy default the return value of each call is the name of the output files (no matter if its a temporary file or not)Use temporary output filesIf the output key is left out, one or more (depending on the operator) temporary files are generated and used as return value(s). In a regular script or a regularly closed interactive session, these files are removed at the end automatically.tminFile=cdo.timmin(input:ifile)#rubytminFile=cdo.timmin(input=ifile)#pythonHowever these tempfiles remain if the session/script is killed with SIGKILL or if the bindings are used via Jupyter notebooks. Those session are usually long lasting and the heavy usage of tempfiles can easily fill the system tempdir - your system will become unusable then.
The bindings offer two ways to cope with thatSet another directory for storing tempfiles with a constructor option and remove anything left in there when you experienced a crash or something like thiscdo=Cdo(tempdir=tempPath)#pythoncdo=Cdo.new(tempdir:tempPath)#rubyremove all tempfiles created by this or former usage of the cdo-bindings belonging to your current Unix-user with (taking into account user-definedtempdirfrom abovecdo.cleanTempDir() #python
   cdo.cleanTempDir   #rubyAlternatively you can use environment variables to set this. Python's and Ruby'stempfilelibraries support the variables 'TMPDIR', 'TEMP' and 'TMP' in their current versions (python-3.8.2, ruby-2.7.0). This feature might be used by administrators to keep users from filling up system directories.Operators with parametercdo.remap([gridfile,weightfile],input:ifile,output:ofile)#rubycdo.remap([gridfile,weightfile],input=>ifile,output=>ofile)#pythonloggingcdo=Cdo.new(logging:true,logFile:'cdo_commands.log')#rubycdo=Cdo(logging=True,logFile='cdo_commands.log')#pythonSet global CDO optionscdo.copy(input:ifile,output:ofile,options:"-f nc4")#rubycdo.copy(input=ifile,output=ofile,options="-f nc4")#pythonSet environment variablescdo.splitname(input:ifile.join(' '),output:'splitTag',env:{'CDO_FILE_SUFFIX'=>'.nc'})#orcdo.env={'CDO_FILE_SUFFIX'=>'.nc'}cdo.splitname(input=' '.join(ifiles),output='splitTag',env={"CDO_FILE_SUFFIX":".nc"})#orcdo.env={'CDO_FILE_SUFFIX':'.nc'}Return multi-dimension arrrayst=cdo.fldmin(:input=>ifile,:returnArray=>true).var('T').get#rb, version <  1.2.0t=cdo.fldmin(:input=>ifile,:returnCdf=>true).var('T').get#rb, version >= 1.2.0t=cdo.fldmin(:input=>ifile,:returnArray=>'T')#rb, version >= 1.2.0t=cdo.fldmin(input=ifile,returnArray=True).variables['T'][:]#py, version <  1.2.0t=cdo.fldmin(input=ifile,returnCdf=True).variables['T'][:]#py, version >= 1.2.0t=cdo.fldmin(input=ifile,returnArray='T')#py, version >= 1.2.0Other options are so-calledmasked arrays(usereturnMaArray) for ruby and python and XArray/XDataset for python-only: usereturnXArrayorreturnXDatasetfor that.*) If you use scipy >= 0.14 as netcdf backend, you have to use following code
instead to avoid possible segmentation faults:cdf=cdo.fldmin(input=ifile,returnCdf=True)temperatures=cdf.variables['T'][:]More examples can be found in test/cdo-examples.rb andon the
homepageAvoid re-processingIf you do not want to re-compute files, you can setthe instance attribute 'forceOutput' to false: this will effect all later
call of that instanceorthe operator option 'forceOutput' to false: this will only effect this
operator call of this instanceFor more information, please have a look at the unit tests.Support, Issues, Bugs, ...Please use the forum or ticket system of CDOs official web page:http://code.mpimet.mpg.de/projects/cdoChangelognext 2.0:reduced usage of keywords:many of them just set return type, so they will go to therun()methodoptions only has effect during run of the tool, so this can also go intorun()the different input types can be handled in something likeinput()or1.6.0:merged pull requests regarding threading, signal and tempfile handlingadded support for xarray/xdataset input AND output at the same time (request by Pauline Millet)1.5.6:slight adoptions for CDO-2.0.0limitted support for python-2.7: many other libs dropped support for it so I can only do limitted testingnew API:cdo.configholds a dictionary/hash with built-in CDO features (availble sind CDO-1.9.x), empty otherwiseremoved cdo.hasLib() and cdo.libsVersion(): relied on unstable output ofcdo -V. use cdo.config insteadinfiles(). This should clean up the lengthy code, which does this
currently1.5.4(python-only):bugfix release for @pgierz regarding #301.5.1(ruby-only):fix some warnings with latest ruby release 2.7.x1.5.0(ruby)/1.5.3(python)API change :simplify the interface:remove returnCdf from constructor, only use it with operator callsremove methods setReturnArray/unsetReturnArray: I fear it's not used anyway, but 'returnArray' in each callremove the optional dependency to scipy since it offers less functionality than netCDF4 and just blows up the codenew attributes: hasNetcdf, hasXArray for checking for the respective supportfix for cdo-1.9.6: allow non-zero return code for diff operatorsthe 1.5.2 relase for python is identical to 1.5.0 (was testing a new setup.py and version cannot be used twice in pipy)1.5.1 had building problems with pip based on python21.4.0API change :theoperatorsatribute is no longer a list, but a dict (python) or hash (ruby) holding the number of output streams as valuefinally fix #16 (missing tempfile generation for more than one output streams)fix #19 (thx @pgierz for the input)1.3.6:bugfix for non-finding the CDO binary on some systemsfix hasCdo (py)add hasCdo (rb)1.3.5:read/write support for XArray datasets - thx to @pinplex!drop ruby support for 1.9 and olderremove module interface from the ruby version1.3.3:return arrays/lists of output files, which are created by split* operators
suggestion from Karl-Hermann Wieners :ocean:NOTE:this is done by simple globbing! Any other files with the appropriate name will be included in the list!usesixfor python2 and 3 compatibility (thanks to @jvegasbsc)drop full support of CDO version older then 1.5.4: undocumented operators
in these version will not be callablenew keyword for operators which write to stdout: autoSplit. When set, each
line will be split with the given value of the keyword to avoid the need
for manual splitting. Nested return arrays of (outer) size 1 are flattened.
See #11, thx to @beatorizu1.3.2improvened stdout/stderr handling, thx to jvegasbsc1.3.1fix environment handling per call (ruby version)1.3.0require ruby-2.*support for upcomming CDO release 1.7.1improve loggin for rubyintroduce logging for pythonunicode bugfix - thanks to Sebastian Illing (illing2005) [python-only]1.2.7Added class interface for ruby version 2.x, mainly for thread safety1.2.6bugfix for autocompletion in interactive usage [python-only]1.2.5bugfix for environment handling (Thanks philipp) [python-only]add logging [ruby-only]1.2.4support python3: Thanks to @jhammanbugfix for scipy: Thanks to @martinclausdocu fixes: Thanks to @guziyallow environment setting via call and object construction (see test_env in test_cdo.py)1.2.3bugfix release: adjust library/feature check to latest cdo-1.6.2  release1.2.2allow arrays in additions to strings for input argumentadd methods for checking the IO libraries of CDO and their versionsoptionally return None on error (suggestion from Alex Loew, python only)1.2.1new return option: Masked Arrays
if the new keyword returnMaArray is given, its value is taken as variable
name and a masked array wrt to its FillValues is returned
contribution for python by Alex Loewerror handling: return stderr in case of non-zero return value + raise exception
contribution for python from Estanislao Gonzalezautocompletion and built-in documentation through help() for interactive use
contribution from Estanislao Gonzalez [python]Added help operator for displaying help interactively [ruby]1.2.0API change:Ruby now uses the same keys like the python interface, i.e. :input and :output
instead of :in and :out:returnArray will accept a variable name, for which the multidimesional
array is returned1.1.0API change:new option :returnCdf : will return the netcdf file handle, which was formerly
done via :returnArraynew options :force : if set to true the cdo call will be run even if the given
output file is presen, default: falseThanks to all contributors!LicenseCdo.{rb,py} makes use of the BSD-3-clause license