Package: easyaccess
easyaccesseasyaccess: an enhanced command line SQL interpreter client for astronomical surveys.Jump to:InstallationUsageConfigurationFAQDescriptioneasyaccessis an enhanced command line interpreter and Python package created to facilitate access to astronomical catalogs stored in SQL Databases. It provides a custom interface with custom commands and was specifically designed to access data from the Dark Energy Survey Oracle database, including autocompletion of tables, columns, users and commands, simple ways to upload and download tables using csv, fits and HDF5 formats, iterators, search and description of tables among others. It can easily be extended to another surveys or SQL databases. The package was completely written in Python and support customized addition of commands and functionalities.For a short tutorial checkhere.DES DR1/DR2 access quickstartTo access the DES public data releases, you first need an account, which you can register yourselfhere. Once you have login credentials for the public DES data server, you can starteasyaccesswith:easyaccess-sdesdrFeaturesNice output format (using pandas)Very flexible configurationSmart tab autocompletion for commands, table names, column names, and file pathsWrite output results to CSV, TAB, FITS, or HDF5 filesLoad tables from CSV, FITS or HDF5 files directly into DB (memory friendly by using number of rows or memory limit)Intrinsic DB commands to describe tables, schema, quota, and moreeasyaccess can be imported as module from Python with a complete Python APIRun commands directly from command lineLoad SQL queries from a file and/or from the editorShow the execution plan of a query if neededPython functions can be run in a inline queryFAQWe have a running list ofFAQwhich we will constantly update, please checkhere.ContributingPlease take a look at ourCode of Conductand ourcontribution guide.CitationIf you useeasyaccessin your research, we encourage you to use this referencehttps://arxiv.org/abs/1810.02721or copy/paste this BibTeX:@ARTICLE{2018arXiv181002721C,author={{Carrasco Kind}, M. and {Drlica-Wagner}, A. and {Koziol}, A.~M.~G. and{Petravick}, D.},title="{easyaccess: Enhanced SQL command line interpreter for astronomical surveys}",journal={arXiv e-prints},keywords={Astrophysics - Instrumentation and Methods for Astrophysics},year=2018,month=Oct,eid={arXiv:1810.02721},pages={arXiv:1810.02721},archivePrefix={arXiv},eprint={1810.02721},primaryClass={astro-ph.IM},adsurl={https://ui.adsabs.harvard.edu/\#abs/2018arXiv181002721C},adsnote={Provided by the SAO/NASA Astrophysics Data System}}See also: Carrasco Kind et al., (2019).easyaccess: Enhanced SQL command line interpreter for astronomical surveys.Journal of Open Source Software, 4(33), 1022,https://doi.org/10.21105/joss.01022InstallationInstallingeasyaccesscan be a little bit tricky given the external libraries required, in particular the Oracle libraries which are free to use. If you are primarily interested inusingtheeasyaccessclient, we recommend running the Docker image as described below.DockerRunningeasyaccessin Docker is easy. Execute the command below to download and run our published image.$dockerrun-it--rm\registry.gitlab.com/des-labs/kubernetes/easyaccess:latest\easyaccess-sdesdr

Enterusername:Enterpassword:ConnectingtoDB**desdr**...
Loadingmetadataintocache..._______\\///..\///._\///.///\\\.///\\\_____///\\_______//DARKENERGYSURVEY`-------`DATAMANAGEMENT

easyaccess1.4.12.TheDESDMDatabaseshell.
_________
DESDR~>SELECTRA,DEC,MAG_AUTO_G,TILENAMEFROMDR2_MAINsample(0.001)FETCHFIRST5ROWSONLY;RADECMAG_AUTO_GTILENAME18.236249-24.02146024.450422DES0032-241528.084798-25.71540126.279263DES0033-254138.142266-35.85492626.509785DES0032-354048.197418-48.27401025.243387DES0030-483158.107404-26.31387624.758778DES0032-2623Alternatively, you may build and run the image yourself using the included Dockerfile with the commands:dockerbuild-tdes-easyaccess.
dockerrun-it--rmdes-easyaccesseasyaccess-sdesdrSource Installationeasyaccessis based heavily on the Oracle python clientcx_Oracle, you can follow the installation instructions fromhere. Forcx_Oracleto work, you will need the Oracle Instant Client packages which can be obtained fromhere.Make sure you have these libraries installed before proceeding to the installation of easyaccess, you can try by opening a Python interpreter and type:importcx_OracleIf you have issues, please check theTroubleshooting pageor ourFAQ page.You can clone this repository and installeasyaccesswith:pythonsetup.pyinstallRequirementsOracle Client> 11g.2 (External library, no python)
Checkherefor instructions on how to install these librariescx_OracleNote that cx_Oracle needs libaio on some Linux systemsNote that cx_Oracle needs libbz2 on some Linux systemsSee thesetup.pyfile for additional requirements.UsageFor a short tutorial and documentation seehere. Note that not all the features are available for public database use.Interactive interpreterAssuming thateasyaccessis in your path, you can enter the interactive interpreter by callingeasyaccesswithout any command line arguments:easyaccessCommand line usageMuch of the functionality provided through the interpreter is also available directly from the command line. To see a list of command-line options, use the--helpoptioneasyaccess--helpRunning SQL commandsOnce inside the interpreter run SQL queries by adding a ";" at the end::DESDB~>select...from...where...;To save the results into a table add ">" after the end of the query (after ";") and namefile at the end of lineDESDB~>select...from...where...;>test.fitsThe file types supported so far are: .csv, .tab, .fits, and .h5. Any other extension is ignored.Load tablesTo load a table it needs to be in a csv format with columns names in the first row
the name of the table is taken from filename or with optional argument --tablenameDESDB~>load_table<filename>--tablename<mytable>--chunksize<numberofrowstoread/upload>--memsize<memoryinMBtoreadatatime>The--chunsizeand--memsizeare optional arguments to facilitate uploading big files.Load SQL queriesTo load SQL queries just run:DESDB~>loadsql<filename.sql>orDESDB~>@filename.sqlThe query format is the same as the interpreter, SQL statement must end with;and to write output files the query must be followed by a standard output redirectcommand > outfileConfigurationThe configuration file is located at$HOME/.easyaccess/config.inibut everything can be configured from inside easyaccess type:DESDB~>helpconfigto see the meanings of all the options, and:DESDB~>configallshowto see the current values, to modify one value, e.g., the prefetch valueDESDB~>configprefetchset50000and to see any particular option (e.g., timeout):DESDB~>configtimeoutshowArchitectureWe have included a simplified UML diagram describing the architecture and dependencies ofeasyaccesswhich shows only the different methods for a given class and the name of the file hosting a given class. The main class,easy_or(), inherits all methods from all different subclasses, making this model flexible and extendable to other surveys or databases. These methods are then converted to command line commands and functions that can be called insideeasyaccess. Given that there are some DES specific functions, we have moved DES methods into a separate classDesActions().