Repository: schemaspy/schemaspy

Issue #360: Error while generating HTML output due to  / in the table names and column names in Hive databases
Created: 2018-06-14T13:24:52Z
I have a Schema in Hive database which contains / in the table name and column name. While generating HTML output for SchemaSpy it fails giving below error:
java.io.IOException: java.io.FileNotFoundException: /localhome/vsatyp/SchemaSpy/npetzall/hive-kerberos/vsatyp_schema/diagrams/orphans/emp/loyee/test.1degree.dot (No such file or directory)
where 'emp/loyee/test' is one table name
Expected Behavior

The SchemaSpy should generate the HTML output with Schema details and original table name should be changed to something else in the generated HTML output...The user should be able to view the table name with / if it is there...
Current Behavior

It is not generating HTML output for the Schema which has / in the table name and column name
Possible Solution

Report should have same tablename as in database only files written should have safenames.
Steps to Reproduce (for bugs)




Create test table in Hive Schema as below:
CREATE TABLE emp/loyee/test (Id INT, First/Name STRING, Last/Name STRING) clustered by (Id) into 2 buckets stored as orc tblproperties ('transactional'='true');


Then below command to create the HTML output:
java -Dhive-kerberos-principal=vsatyp@GLOBAL.COM -Dhive-kerberos-keytab=/localhome/vsatyp/vsatyp.keytab -jar /localhome/vsatyp/SchemaSpy/npetzall/schemaspy-6.0.0-rc2.jar -schemas vsatyp_schema -o /localhome/vsatyp/SchemaSpy/npetzall/hive-kerberos -db ProdHiveDB -u hive -t /localhome/vsatyp/SchemaSpy/npetzall/hive-kerberos.properties -dp /localhome/vsatyp/SchemaSpy/npetzall/hive-kerberos-driverwrapper-0.0.3-hdp-all.jar --logging.level.org.schemaspy=DEBUG


Context

It is not generating HTML output for other Schema as well. I need to generate the HTML Schema representation for all Schema in Hive database.



Database type (e.g. MySQL): Hive


Database driver version (is you use a special one):
hive-kerberos-driverwrapper-0.0.3-hdp-all.jar


Your DDL (sample how to reproduce the issue):
CREATE TABLE emp/loyee/test (Id INT, First/Name STRING, Last/Name STRING) clustered by (Id) into 2 buckets stored as orc tblproperties ('transactional'='true');


Full command including arguments how you run SchemaSpy:
java -Dhive-kerberos-principal=vsatyp@GLOBAL.COM -Dhive-kerberos-keytab=/localhome/vsatyp/vsatyp.keytab -jar /localhome/vsatyp/SchemaSpy/npetzall/schemaspy-6.0.0-rc2.jar -schemas vsatyp_schema -o /localhome/vsatyp/SchemaSpy/npetzall/hive-kerberos -db ProdHiveDB -u hive -t /localhome/vsatyp/SchemaSpy/npetzall/hive-kerberos.properties -dp /localhome/vsatyp/SchemaSpy/npetzall/hive-kerberos-driverwrapper-0.0.3-hdp-all.jar --logging.level.org.schemaspy=DEBUG


Your Environment


Version used:
Environment name and version (e.g. Chrome 39): Chrome Version 67.0.3396.79 (Official Build) (64-bit)
Operating System and version (e.g. Windows 10): Red Hat Enterprise Linux Server 7.5 (Maipo)
Java version: java version "1.8.0_131"

Duplicate issue #635: Failed if table names are non-ascii 
Created: 2019-12-26T07:26:59Z
Some tables in my database have non-ascii (Japanese) names, and Schemaspy fails with an Exception.
It seems handling "{tablename}.1degree.dot" doesn't work well.
java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: ???????.1degree.dot

Expected Behavior
Succeed even if table names are non-ascii.
Current Behavior
Exception occurs.
...Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
Caused by: java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: ???????.1degree.dot
        at sun.nio.fs.UnixPath.encode(UnixPath.java:147)
        at sun.nio.fs.UnixPath.<init>(UnixPath.java:71)
        at sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:281)
        at sun.nio.fs.AbstractPath.resolve(AbstractPath.java:53)
        at org.schemaspy.output.html.mustache.diagrams.MustacheOrphanDiagramFactory.generateOrphanDiagrams(MustacheOrphanDiagramFactory.java:76)
        at org.schemaspy.SchemaAnalyzer.generateHtmlDoc(SchemaAnalyzer.java:366)
        at org.schemaspy.SchemaAnalyzer.analyze(SchemaAnalyzer.java:260)
        at org.schemaspy.SchemaAnalyzer.analyze(SchemaAnalyzer.java:123)
        at org.schemaspy.cli.SchemaSpyRunner.runAnalyzer(SchemaSpyRunner.java:98)
        at org.schemaspy.cli.SchemaSpyRunner.run(SchemaSpyRunner.java:87)
        at org.schemaspy.Main.main(Main.java:55)
        ... 8 more
Steps to Reproduce (for bugs)
DBMS=pgsql

docker run --rm \
  -e LANG=ja_JP.UTF-8 \
  -v "${DST_DIR}:/output" \
  --net="host" schemaspy/schemaspy \
  -t ${DBMS} -host ${_HOST}:${_PORT} -db ${_DB} -u ${_DBUSER} -p ${_DBPASSWORD} -s ${_SCHEMA} 
Your Environment

SchemaSpy in Docker: 6.1.0

Closing comment (2019-12-26T08:14:32Z): This is partially related to #360. The same solution should work. Will do a check on that one, or modify the code to avoid any copyright/license issue.