Repository: hydroshare/hydroshare

Issue:
HISTimeSeries preview figure does not display on www.hydroshare.org (NFIE 1.5.2)
HIS Time Series on NFIE release 1.5.2 (www.hydroshare.org) does not display the time series preview  figure when creating a res. However the code works on park.hydroshare.org.
The figure should be located at "http://www.hydroshare.org/static/img/visualization-*.png" but in our exploring, it seems like it is not generated and stored here at all. @mjstealey, is there a difference in "www" that would prevent the file from being stored here?

In any case we plan on changing the storage of the visualization figures in the creation to a tmp folder using tempfile module like with the other download files for the resource #466 .

Drew and Jeff



Discussion:
@zhiyuli - the code that is on park is based on branch [446-improve-res-mod-perf](https://github.com/hydroshare/hydroshare/pull/483).
This works on park because it is writing directly to the hydroshare directory.

There is fundamentally no difference between www and any other deployments, however your comparison to how park operates isn't valid since it uses your old code.

To debug, download a fresh copy of the hydroshare repository and build the develop branch locally to determine if you observe the same behavior as you are seeing on www.

I do not think .../static/img/visualization... is the right place for a resource to be serving files.  Any files associated with a resource should be in the resource/<GUID> path.  This is notwithstanding changes to avoid the storage of these files for this resource type altogether.

@mjstealey  @dtarb 
Sorry we did not make it clear. 
We know current www and park are running different HISTimeSeries code. However the code related to this issue is the same.
This issue occurs during the resource creation process and only concerns the preview figure files which are stored temporarily and deleted when a resource is created. This is not the visualization file that is currently added as a resource file; we will do away with this along with the other resource files when we refactor the HISTimeSeries code. 
So why can park display the preview figure but not www? WWW and park are in production mode. We are guessing www and park are handling static files in different ways?
Also, my local latest dev code (same code as www, based on “develop” branch) can display it in development mode. 
The current code saves this preview figure to "hydroshare_source_folder/theme/static/img" because files in this folder have urls (http://_.hydroshare.org/static/img/_.png), so frontend can retrieve them via urls and display them.
Yesterday we tried saving the preview figure to /tmp just like what did to resource files on www, but it did not work because this folder is just a system folder without having a url.
Our possible options are:
1) continue saving it in the ‘static’ directory and debug the problem in displaying on “www”
2) save the preview figure in /tmp, find a way that backend can return figure binary (not url) to frontend through ajax, and frontend displays this figure’s binary directly. We have not looked into this much and are not sure how viable this option is.
3) draw this figure on frontend (javascript chart lib). This option will most likely slow down the process significantly esp. when time series are very large as the drawing will be on the front end 
Any suggestion?
4) remove preview graph

Thanks
Drew and Jeff

park has been redeployed using the same code as www.
The only difference between the two is that DEBUG=False on www.

The only valid HIS TS resource on park that I could find was created on June 17: http://park.hydroshare.org/resource/33e3827f2af54af796602865b26d8dfc/

There is also a HIS TS resource on www from June 24: http://hydroshare.org/resource/b9a729c828d147288588ac02b5a7a93b/

I'm not seeing a difference in how these are reflected/displayed in their subsequent landing pages.
There is also a .png file available for download from the resource files in each case.

I do not know if having a different DEBUG setting is effecting your resource creation process, but re-try your code on park and see if it is still operating as expected. If so, you can set DEBUG=False on your local dev environment and see if you can replicate the issue.

The issue is not on the landing page, it occurs in the middle of resource creation process when user choose parameters and prepare to create the resource.
The problem seems to be with the debug setting (see snapshots).

park:
![image](https://cloud.githubusercontent.com/assets/10070599/8362055/f16723e4-1b33-11e5-829c-ca8e4f553b5e.png)

www:
![image](https://cloud.githubusercontent.com/assets/10070599/8362064/f8dbf974-1b33-11e5-9d75-85d6c5439580.png)

local VM (debug=False)
![image](https://cloud.githubusercontent.com/assets/10070599/8362311/4bf21746-1b35-11e5-903f-1b7a46f6482b.png)

@zhiyuli - Okay, this is interesting.

I had run into a similar issue with the rendering of the account login and reset password pages when I first set DEBUG=False on www as well.

In that case it was a matter modifying the base page of what Django was using to display the form which then worked for both DEBUG = True and False.
Reference: https://github.com/hydroshare/hydroshare/commit/717a1b00a4d7bcce14021985d82c8695701394b5

@mjstealey 
A quick test shows:

When Debug = True, both /hydroshare/theme/static/img/ and /hydroshare/hydroshare/static/img/ are mapped as http://...hydroshare.org/static/img/

When Debug = False, only /hydroshare/hydroshare/static/img/ is mapped as http://...hydroshare.org/static/img/
We don't know why it happens. Anyway now we changed it to store the preview figure files in /hydroshare/hydroshare/static/img/ which works for both.


Resolving patch:
diff --git a/ref_ts/ts_utils.py b/ref_ts/ts_utils.py
--- a/ref_ts/ts_utils.py
+++ b/ref_ts/ts_utils.py
@@ -455,7 +455,8 @@ def create_vis(path, site_name, data, xlab, variable_name, units, noDataValue):
     ax.autoscale_view()
 
     ax.grid(True)
-    vis_name = 'visualization-'+site_name+'-'+variable_name+'.png'
+    vis_name = 'vis-'+site_name+'-'+variable_name+'.png'
+    vis_name = vis_name.replace(" ", "")
     vis_path = path + "/" + vis_name
     savefig(vis_path, bbox_inches='tight')
     vis_file = open(vis_path, 'r')
diff --git a/ref_ts/views.py b/ref_ts/views.py
--- a/ref_ts/views.py
+++ b/ref_ts/views.py
@@ -87,7 +87,7 @@ def time_series_from_service(request):
         noDataValue = ts.get('noDataValue', None)
         ts['url'] = url
         ts['ref_type'] = ref_type
-        vis_file = ts_utils.create_vis("theme/static/img/", site, for_graph, 'Date', variable_name, units, noDataValue)
+        vis_file = ts_utils.create_vis("hydroshare/static/img/", site, for_graph, 'Date', variable_name, units, noDataValue)
         vis_file_name = os.path.basename(vis_file.name)
         return json_or_jsonp(request, {'vis_file_name': vis_file_name})
 
@@ -171,10 +171,10 @@ def create_ref_time_series(request, *args, **kwargs):
 
         ts_utils.generate_files(res.short_id,ts)
 
-        for file_name in os.listdir("theme/static/img"):
-            if 'visualization' in file_name:
+        for file_name in os.listdir("hydroshare/static/img"):
+            if 'vis-' in file_name and ".png" in file_name:
                 # open(file_name, 'w')
-                os.remove("theme/static/img/"+file_name)
+                os.remove("hydroshare/static/img/"+file_name)
 
         return HttpResponseRedirect(res.get_absolute_url())
 


Test patch: