Package: robotframework-sikulixlibrary
robotframework-sikulixlibraryThe all new, modern, SikuliX Robot Framework library for Python 3.x, based on JPype or Py4J Python modules.It can be used with e.g. Robot Framework projects (https://robotframework.org), Robocorp projects (https://robocorp.com) - the easiest approach for beginners or with pure Python projects. Either for Test Automation type of projects, or for open source RPA (Robot Process Automation) projects.JPypeis a Python module to provide full access to Java from within Python.Py4Jenables Python programs running in a Python interpreter to dynamically access Java objects in a JVM.This library is a wrapper to SikuliX that is exposing Java functions as Robot Framework keywords, and it can be enabled to use by
choice any of the JPype or Py4J modules. This is done by creating SIKULI_PY4J environment variable and setting to 1. When not defined or
set to 0, JPype is used instead. Please note that on MacOS, only Py4J can be used, while on Windows or Ubuntu, any of them is working.While in the past the only approach to use Sikuli functionality within Robot Framework was through Remote Server with XML RPC interface (SikuliLibrary), the aim
of this library is to replace that approach and make it a lot easier to use SikuliX within Robot Framework projects with a simple Library statement
(i.e. no need to start remote server and so on).Also with this implementation is very easy to extend the library with new custom keywords, for example with the purpose to
create migration classes to help migrate from current Sikuli libraries or other image recognition alternatives. For practical examples check migrate folder.Seekeyword documentation.Installation instructions (Windows)Python 3.8 or newer, as supported by JPype or Py4JJPype 1.4.1 or newer and JPype project dependencies as explained on project page:https://github.com/jpype-project/jpypeInstall Java 8 or newerWhile not mentioned on JPype page, on a new Windows 10 machine also Visual C++ Redistributable 2015 and newer are needed (e.g. vc_redist.x64.exe)or Py4J 0.10.9.7 or newerSikuliX as a standalone jar from project page:https://raiman.github.io/SikuliX1/downloads.htmlPut jar file in any local directory (e.g. C:\sikulix\sikulix.jar)Install Tesseract 4 for using OCR functionalityPy4J server is enabled from SikuliX 2.0.5 onward and currently advertised as experimental. However, this library is working as expected with Py4J.Recommended to use environment variable SIKULI_HOME that point to sikulix local directorypip install robotframework-sikulixlibraryCheck Known issues section below if libspec is not generated within your environment (RF IDE tool) and thus library keywords are not recognisedWhile JPype JVM is always started automatically, Py4J JVM can be started manually or automatically. To start manually, use the command:java -jar sikulix.jar -p(to start Py4J server) orjava -jar -DsikuliDebug=3 sikulixide.jar -p(useful e.g. for checking sikulix debug info)Installation instructions (MacOS)Mainly the same steps as for Windows above. However, installing Tesseract 4 seems to be challenging since e.g.brew install tesseractwill install Tesseract 5. But at least the following method fromstackoverflowfor installing Tesseract 4.1.3 is working.ExamplesTesting withRobot Framework*** Settings ***LibrarySikuliXLibrarysikuli_path=sikulixide-2.0.5.jar*** Test Cases ***Example TestimagePath add${my_path}settings setMinSimilarity${0.9}app openC:/Windows/System32/notepad.exeregion waitiNotepad.PNGregion pasteWelcome!Testing withPython.fromSikuliXLibraryimportSikuliXLibrarysikuli_path='sikulixide-2.0.5.jar'lib=SikuliXLibrary(sikuli_path)lib.imagePath_add('my_path')lib.settings_set('MinSimilarity',float(0.9))lib.app_open("C:\\Windows\\System32\\notepad.exe")lib.region_wait('iNotepad')lib.region_paste('Welcome!)TestingGit clone, and if not using pip install for this library, then just point PYTHONPATH to local robotframework-sikulixlibrary folder and execute:python testlibrary_win.py(or any .py file from under test directory and for OS of choicerobot --outputdir results/default test_defaultlibrary_win.robot(or any .robot file from under test directory and for OS of choice)or maybe on MacOSpython3 -m robot --outputdir results/default test_defaultlibrary_osx.robotObviously, image files from test/img/MacOS, Ubuntu or Windows might not work on specific environment and would need to be regenerated. Also for these tests SIKULI_PATH is defined and the name of SikuliX issikulixide-2.0.5.jarAdditionally, debugging with some RF supported tools is also possible with this library, for both Robot Framework and Pyton code. Python library debugging was tested with Visual Studio Code with Robot Framework Language Server by Robocorp, by usingdebug test.pyfile. Also Robot Framework test code from within test directory was tested with debugging, with the same tool, by creating a specific configuration within launch.json file (VSCode specific file).Tested on Windows 10 with 1920x1080, Windows Dark mode, Light app mode. Also limited testing with 4k.Tested on MacOS Big Sur with 1440x900, Dark Appearance. Also limited testing with 4k. Text Edit should be put in Plain Text mode.Tested on Ubuntu with 1920x1080 and with Leafpad application.Supported Operating SystemsWindows 10supported, tested with both JPype and Py4JOSXSikuliX works under OSX, however currently there are issues with JPype generally working under OSX:https://github.com/jpype-project/jpype/issues/911Py4J tested and working under MacOS and always enabled without definding SIKULI_PY4J environment variable. However, forcing JPype for experimental purpose is
possible with environment variable SIKULI_PY4J=0.Linuxsupported, tested with Ubuntu 20.04 and Leafpad application. Tested with both JPype and Py4J.due tohttps://github.com/RaiMan/SikuliX1/issues/438, openApp is not currently working with SikuliX 2.0.5, thus it is disabled in the test .py and .robot code for Ubuntu.
This means you have to start the test and open manually Leafpad app in order for tests to succeed.tested with: python3.8, default-jre (openjdk-11-jre), libopencv4.2-java as explained on SikuliX support page, gnome-panel,pip install robotframework-sikulixlibrarystart the tests with e.g.python -m robot --outputdir results/ubuntu test_defaultlibrary_ubuntu.robotorpython testlibrary_ubuntu.pyKnown IssuesWhen using Py4J, libdoc will not generate library documentation within IDE, e.g. Visual Studio Code with Robocorp extension (Robot Framework Language Server). The workaround is to start manually SikuliX Py4J server (see above or runjava -jar sikulix.jar -p), then reload the library to generate the keyword documentation. If not using pip install for this library, then the extension settings.json need to have pythonpath set to local robotframework-sikulixlibrary folder.When generating library documentation within IDE, the library is instantiated with no arguments and it will look for sikulix.jar file within SIKULI_HOME environment variable defined directory, so make sure this file is there even if it is a duplicate of e.g. sikulixide-2.0.5.jar. Otherwise, it is also possible to manually generate the libspec with the following command, and copy it under project folder or PYTHONPATH:python -m robot.libdoc SikuliXLibrary::sikuli_path=path\to\sikulixide-2.0.5.jar SikuliXLibrary.libspecSee this documentation for reference:https://github.com/robocorp/robotframework-lsp/blob/master/robotframework-ls/docs/faq.md