Package: cgshop2020-pyutils
Official Python Utilities for the CG:SHOP 2020 Optimization Competition.WARNING: There is a small bug in the verifier. It does allow leaf points. We updated the source and the linux binary but due to a compiler bug, we cannot create a portable version for OS X. However, you can use the self-compile option. A update for Windows and OS X will hopefully follow soon.We provide basic code to ease your participation in this year'schallenge.
Due to it popularity and simplicity, the choice has fallen on Python.
However, the real code is written in C++ with CGAL to provide the needed accuracy and
efficiency.This python module allows you to easily read the instance and potentially convert them
into an easier format. The JSON format is not as simple as last year's raw format but
it allows to add metadata. Further, it allows you to verify your instances. It uses
the same core as the server so if this code accepts your solution, so will our server.The code is not perfect but we will work to improve this code during the competition.
Feedback is welcome.The C++-core unfortunately messes with the platform independence of Python.
At this point, we already spent much more time into making the module as compatible as
possible possible than into the actual code.
Unfortunately, there are still a lot of ugly workarounds, which we try to remove soon.
The module has been tested on OS X and Linux.Windows is still experimentally.We implemented a fallback mode that in case the native core could not be loaded at least
the basic functionality for reading and writing solutions and instances is available.Please report any bugs you may find.
You can simply write a mail tocgshop-admin@ibr.cs.tu-bs.deand one of our developers will take care of it (also if you have questions or suggestions).https://gitlab.ibr.cs.tu-bs.de/alg/cgshop2020_pyutilsFeaturesReading and writing instance files.Reading and writing solution files.Verifying solutions for feasibility.Simple triangulation solver to get a trivial feasible solution.The exact same implementation will be used on the submission server.
Thus, if this library will accept your solution, so will the submission server.InstallingYou need to have a 64bit operating system (Linux, OS X, Windows) and a 64bit Python
interpreter for full functionality. Unfortunately, it is very easy on Windows to
accidentally install outdated 32bit versions. 32bit users will only be able to use
a limited functionality (or have to compile the core themselves).The module is available through pypi, e.g.,pipinstallcgshop2020-pyutilsor when using the generally recommendedpipenv.pipenvinstallcgshop2020-pyutilsNote that you have to use_instead of-when importing the module.You can also simply copy the foldercgshop2020_pyutilsinto your code folder and use
it like a local module.If you want to work in C++ directly, you can also simply copy ourcpp_core.
If you have some experience with CMake, this should be straightforward.
However, you will be missing the JSON functionality as this is written in pure Python
(which is much simpler than doing this in C++).To update to the newest version, you have to usepipinstallcgshop2020-pyutils--upgradeorpipenvupdatecgshop2020-pyutilsGetting startedWith the module installed, you can now try to create your first solution set (simple triangulations) as follows:# Load an instancefromcgshop2020_pyutilsimportInstanceDatabase,BestSolutionSet,SolutionZipWriter,TrivialTriangulationSolver# load challenge instancesidb=InstanceDatabase("/path/to/zip/")# compute the triangulation for all instancestriangulation_solver=TrivialTriangulationSolver()solutions=BestSolutionSet()forinstanceinidb:solutions.add(triangulation_solver(instance))print(f"Computed triangulation for{instance.name}")# write solutions into zipprint("Creating zip. This can take some time...")withSolutionZipWriter("my_first_upload.zip")aszipper:zipper.add_solutions(solutions)print("You can now upload 'my_first_upload.zip' on","https://cgshop.ibr.cs.tu-bs.de/competition/cg-shop-2020/")You can find further examples in./examplesInstancesYou can easily access the competitions instances by downloading thezip-archive of the first instances batch,
and using theInstanceDatabase.TheInstanceDatabasedoes only need the zip file or the directory of the extracted zip and will
recursively search it for instances of a specific name, e.g.fromcgshop2020_pyutilsimportInstanceDatabaseinstance_database=InstanceDatabase("/path/to/zip")mona_lisa_instance=instance_database["mona_lisa_1000000"]alternatively, you could also simply iterate over all instances:fromcgshop2020_pyutilsimportInstanceDatabaseinstance_database=InstanceDatabase("/path/to/zip")forinstanceininstance_database:print(f"Found instance{instance.name}")The instance class is rather simple:fromcgshop2020_pyutilsimportInstanceDatabaseinstance_database=InstanceDatabase("/path/to/extracted/zip")instance=instance_database["uniform-0000100-2"]print(f"The instance is named{instance.name}.")print(f"It has{len(instance.name)}points.")print(f"The first point is f{instance[0]}.")print("I now list the other points:")forpointininstance:print(f"({point.get_x()},{point.get_y()}")You could also create your own instance byfromcgshop2020_pyutilsimportInstance,Pointinstance=Instance(name="my_instance",points=[Point(1.0,1.0),Point(0.0,3.0),Point(2.0,0.0)])instance.add_point(Point(1.0,5.0))# add an additional pointInstance FormatAn instance in json can look likes this:{"points":[{"i":0,"x":2396.0,"y":5284.0},{"i":1,"x":2656.0,"y":2938.0},{"i":2,"x":4120.0,"y":2278.0},{"i":3,"x":4342.0,"y":102.0},{"i":4,"x":4384.0,"y":2988.0},{"i":5,"x":5136.0,"y":2280.0},{"i":6,"x":6634.0,"y":5416.0},{"i":7,"x":8598.0,"y":2632.0},{"i":8,"x":8898.0,"y":4170.0},{"i":9,"x":11738.0,"y":1550.0}],"type":"Instance","name":"euro-night-0000010","meta":{"comment":"HIP even point set instance (10 points) sampled from image ","faces_in_triangulation":12}}points: a list of points with (i=index, x=x coordinate, y=y coordinate)type: "Instance", just tells you that this json describes an instancemetasome optional metadatacomment: Some comment on the instancefaces_in_triangulation: Number of faces in the delaunay triangulation as trivially achievable objective value.name: name of instance.Last time we used some raw text format which is easier to parse but less powerful.
If you don't want to read json in your software, we recommend to convert the instances
via our python library to a simpler format.fromcgshop2020_pyutilsimportInstanceDatabaseinstance_database=InstanceDatabase("/path/to/extracted/zip")forinstanceininstance_database:withopen(f"{instance.name}.txt","w")assimple_file:forindexinrange(len(instance)):point=instance[index]simple_file.write(f"{index}{point.get_x()}{point.get_y()}\n")For loading an instance from a specific file, you can use theInstanceReader.Solutionfromcgshop2020_pyutilsimportSolution,Edgesolution=Solution(instance="my_instance")solution.add_edge(Edge(0,1))# edge from point with index 0 to point with index 1solution.add_edge(Edge(0,2))# edge from point with index 0 to point with index 2solution.delete_double_edges()# remove redundant edges, as double edges are illegal.foredgeinsolution:print(edge)foriinrange(len(solution)):edge=solution[i]print(edge)you can easily create a valid json file:fromcgshop2020_pyutilsimportSolutionWriterwriter=SolutionWriter()writer.to_json(solution=solution,path="my_solution.json")to create a zip for upload, usefromcgshop2020_pyutilsimportBestSolutionSet,SolutionZipWritersolutions=BestSolutionSet()solutions.add(solution)withSolutionZipWriter("my_upload.zip")aszipper:zipper.add_solutions(solutions)We will probably add a functionality for automatic upload from code in the near future.Meta dataSolutionandInstanceboth have ameta_dataattribute which is a dictionary.
For example you can add a comment bymy_solution.meta_data["comment"]="This is a comment"which will also be saved when converting to JSON.
Everything that can be converted to string will be saved to json and also reloaded
when reading the json.CheckerWe provide an efficient checker for verifying solutions.
In case of infeasibility, the checker also provides a message with an error description.
If the solution is feasible, the objective value is computed.fromcgshop2020_pyutilsimportSolutionCheckerchecker=SolutionChecker()status=checker(instance=instance,solution=solution)print(status.is_feasible())print(status.get_message())print(status.get_objective_value())VisualizationWe provide a simple visualization based on matplotlib to quickly plot instances and solutions.fromcgshop2020_pyutilsimportVisualizervis=Visualizer()vis.visualize_solution(solution=solution,instance=instance)# opens plot if possiblevis.visualize_solution(solution=solution,instance=instance,path="my_fig.pdf")# writes plot to fileTrivial Triangulation SolverWe provide a simple solver that just computes the Delaunay triangulation.fromcgshop2020_pyutilsimportTrivialTriangulationSolversolver=TrivialTriangulationSolver()solution=solver(instance)Check a zipfromcgshop2020_pyutilsimportZipSolutionIterator,SolutionChecker,InstanceDatabasechecker=SolutionChecker()idb=InstanceDatabase("./instances_zipfile")zsi=ZipSolutionIterator()# Also allows to set some constraints on file size, etc.forsolutioninzsi("myzip.zip"):instance=idb[solution.instance_name]assertchecker(instance=instance,solution=solution).is_feasible()Compiling yourselfYou should not need to do this as the module comes with precompiled binaries. However, it is possible to recompile the C++-core on your machine.You need to have the following libraries installed:A C++ compiler e.g. Debian/Ubuntuapt-get install g++CMake Debian/Ubuntu:apt-get install cmake, OS X:brew install cmakeCGAL Debian/Ubuntu:apt-get install cgal, OS X:brew install cgalBoost Debian/Ubuntu:apt-get install boost, OS X:brew install boostPython3 (of course)matplotlib in Python3 (e.g.,pip3 install matplotlib)Otherwise, you likely get an error.Do a manual recompile with:importcgshop2020_pyutilscgshop2020_pyutils.compile_cpp_core()This will only work if you installed into user space (without sudo) as otherwise the
compiler cannot write the files into the module folder.This command actually just executes CMake.
You could also do this by hand.For Windows, CMake might use slightly different paths. In that case, you could simply copy the created.dllinto thebinariesfolder.Changelog0.1.0First published version with native support for OS X and Linux.0.1.1Experimental Windows support0.1.232bit detection and warning.0.1.3fixed a problem with some Windows versions (could not find the supplied dependencies in the same folder)0.1.4Implemented InstanceDatabase, fixed bug with reading solution (thanks to Johannes Obenaus)0.1.5Bugfix in InstanceDatabase0.2.0Zip reading utilities.0.2.1Missing dependency declaration in pip package (chardet).0.3.0Added zip writing utilities and examples.0.3.1Removed potential problem in InstanceDatabase, improved documentation. Fixed a bug in zip reader (which makes me wonder why it worked on the test server?)0.3.2The ZIP module in the Python library has some breaking changes (problems with older versions).0.3.3The potential bugfix of 0.3.1 introduced a real bug with some paths.0.3.4Same as 0.3.3 but this time it should really fix it.0.3.5The InstanceDatabase has a serious memory consumption. Disabled caching by default.0.3.6Some improvements in handling exceptions of bad files.0.3.7Fixed a bug that made loading the C++ core with MS Windows impossible. Thanks for noticing and fixing to Semjon Kerner.0.4.0InstanceDatabase can now read directly from zip.0.4.1Fixed bug in verifier that does allow leaf points. NO BINARY UPDATE FOR OS X AND WINDOWS RIGHT NOW! Thanks to Gil Ben-Shachar for noticing!0.4.2New core compiled for OS X. Version checking of core.0.4.3Fixed problems if meta data is not a dict.0.4.4More flexible regarding the instance name (allowing ".instance" in instance access via InstanceDatabase).Roadmap/Planned featuresProper testing with 100% coverage (currently don't have the manpower for that)Upload of zipsFetching the objective value for your currently best solutions from the server (and skipping upload of worse solutions)