Repository: sunpy/sunpy

Issue:
Map.rotate pads with zeros in addition to the value of "missing"
### Description

`Map.rotate()` pads the input data when rotating. The `missing` keyword provides a value for filling the missing values. But this value is not used everywhere where I expect it: there is also a stripe of zero values.

### Expected vs Actual behavior

The value of `missing` should be used in all pixels of the output map where there is no value from the input map (after rotation).

This is not the case: some of these pixels have a value of zero.
 
### Steps to Reproduce

```python
from sunpy.data.sample import AIA_171_ROLL_IMAGE
import sunpy.map
import numpy as np

aiamap = sunpy.map.Map(AIA_171_ROLL_IMAGE)
aia_aligned = aiamap.rotate(order=3, missing=np.nan)
aia_aligned
```
In a notebook, this gives
![image](https://user-images.githubusercontent.com/3624676/109357300-485bf600-7882-11eb-9295-807730e13ced.png)

The red pixels are the expected NaNs, the black stripe is unexpected zeros (it should also be NaNs), as can be displayed with `aia_aligned.peek()`.

The result is the same with `order=1`.

### System Details

#######
General
#######
Time : Friday, 26. February 2021 09:29PM UT
System : Linux
Processor : x86_64
Arch : 64bit
SunPy : 2.1.0
OS: Ubuntu 20.10 groovy (Linux 5.8.0-43-generic x86_64)


##################
Required Libraries
##################
Python: 3.8.6
NumPy: 1.19.4
SciPy: 1.5.4
matplotlib: 3.3.3
Astropy: 4.1
Pandas: 1.1.0
parfive: 1.1.1


#####################
Recommended Libraries
#####################
beautifulsoup: 4.9.1
PyQt4: NOT INSTALLED
PyQt5: 5.15.0
Zeep: 3.4.0
Sqlalchemy: 1.3.19
drms: 0.6.1


Discussion:
Digging deeper into this, I found that this is because of the ```np.nan_to_num``` method which converts all ```nan``` to ```0.0```. All ```nan``` is converted to ```0``` because the following method ```scipy.ndimage.interpolation.affine_transform``` throws an error if the input image contains ```nan``` . 
The partial padding is due to the resized image being padded with ```0``` (black stripes) and the rotated image padded with ```nan``` (this rotated image is the output of ```scipy.ndimage.interpolation.affine_transform``` which pads ```nan``` to the rotated image i.e. the ```missing``` keyword.)
In the case of any other value being passed to ```missing```, ```np.nan_to_num``` doesn't convert them to ```0``` and the rotated image fills the remaining bits in after resizing.
I suppose raising a warning when a user wants to use ```missing=np.nan``` and defaulting to ```missing=0``` would get rid of the red stripes altogether.
@jeffreypaul15 : Some corrections:

- The transformation code defaults to `scikit-image`'s `warp()`, not `scipy`'s `affine_transform()`.
- We don't call `np.nan_to_num()` when `order` is less than 4, as is the case in the reported code block.  It used to be the case that `scikit-image`'s `warp()` preseved NaNs for `order` < 4, but it's possible that that is no longer the case.
> @jeffreypaul15 : Some corrections:
> 
>     * The transformation code defaults to `scikit-image`'s `warp()`, not `scipy`'s `affine_transform()`.
> 
>     * We don't call `np.nan_to_num()` when `order` is less than 4, as is the case in the reported code block.  It used to be the case that `scikit-image`'s `warp()` preseved NaNs for `order` < 4, but it's possible that that is no longer the case.

My apologies, I assumed that it defaults to using ```scipy``` if ```use_scipy``` was not specified. 
Also, when the order is < 4 the plot seems to work fine because ```np.nan_to_num``` is not being used and the missing values are specified accordingly. So the plot while using ```scikit-image``` and ```order=3``` (the code given above) as @ebuchlin mentioned is 
![image](https://user-images.githubusercontent.com/50923538/109537948-873db600-7ae5-11eb-945a-9f675fb382d9.png)

But it seems as though that in this issue, ```scikit-image``` is not installed which defaults to using Scipy which causes the issue of the red and black stripes.

I wanted to know if setting ```missing=0.0``` if ```missing=np.nan``` is a possible solution?

@jeffreypaul15 What version of `scikit-image` are you running?  I am running 0.16.2, and I see the same behavior as the reported bug with the non-preservation of NaNs.  My setup is not falling back to use `scipy`.

> I wanted to know if setting `missing=0.0` if `missing=np.nan` is a possible solution?

I say no
> @jeffreypaul15 What version of `scikit-image` are you running? I am running 0.16.2, and I see the same behavior as the reported bug with the non-preservation of NaNs. My setup is not falling back to use `scipy`.
> 
> > I wanted to know if setting `missing=0.0` if `missing=np.nan` is a possible solution?
> 
> I say no

Okay I think I may have found the cause for this
```python
from astropy.wcs import WCS
wcs = WCS(sunpy.data.sample.AIA_171_ROLL_IMAGE)
data = fits.getdata(sunpy.data.sample.AIA_171_ROLL_IMAGE)
mymap = sunpy.map.Map((data, wcs))  
```
Here ```mymap.data``` is AIA_171_ROLL_IMAGE but of type ```float32```
If this is then rotated with 
```python
aia_aligned = mymap.rotate(order=3, missing=np.nan)
```
The resultant plot would be
![image](https://user-images.githubusercontent.com/50923538/109551736-f079f500-7af6-11eb-8aa4-347f52f0464e.png)

This is because in ```mapbase.py```, ```np.pad``` causes the padding of ```float32``` with ```np.nan``` properly as ```np.nan``` is of ```float``` data type. 
The resultant resized image is 

![image](https://user-images.githubusercontent.com/50923538/109552446-cbd24d00-7af7-11eb-8f4a-295b55df5342.png)

which is then sent to ```transform.py```'s ```affine_transform()``` which then goes through ```skimage``` warp to produce the completely red bounded image.

Now, when using 
```python
from sunpy.data.sample import AIA_171_ROLL_IMAGE
aiamap = sunpy.map.Map(AIA_171_ROLL_IMAGE)
aia_aligned = aiamap.rotate(missing=np.nan, order=3)
```
```aiamap.data``` is of ```>i2``` dataype (2 byte int).

The resultant plot is 

![image](https://user-images.githubusercontent.com/50923538/109552802-400cf080-7af8-11eb-9a49-6a365ac7d121.png)

As ```np.pad``` converts the padding of ```np.nan``` (missing) in ```>i2``` data type to ```0```. 
Which results in the plot 

![image](https://user-images.githubusercontent.com/50923538/109553042-87937c80-7af8-11eb-900e-0127e0c6ac02.png)

Which is then converted to float in ```transform.py```, and then sent to skimage's ```warp``` which causes the bug which is mentioned in this issue.


Solving this would include, instead of converting to ```float64``` (if the image is of type ```>i2```) in ```transform.py```, we can check in ```mapsequence.py``` itself if it is of ```int``` dtype and then proceed to convert it first and then send it to ```transform.py```. 
The required plot would be produced
![image](https://user-images.githubusercontent.com/50923538/109553560-2f10af00-7af9-11eb-969c-2cce984c928c.png)
with the same code 
```python
from sunpy.data.sample import AIA_171_ROLL_IMAGE
import sunpy.map
import numpy as np

aiamap = sunpy.map.Map(AIA_171_ROLL_IMAGE)
aia_aligned = aiamap.rotate(order=3, missing=np.nan)
aia_aligned
``` 
But this would still have the original issue of ```np.nan_to_num``` when using ```scipy``` but would solve it using```skimage```.
Please let me know if this is a viable change. Thanks @ayshih
Ah, yes, the problematic code is in `sunpy.map.GenericMap.rotate()` (as opposed to `sunpy.image.transform.affine_transform()`): https://github.com/sunpy/sunpy/blob/6b3fdfb45d3d8687a16eb193fd87e8a27881d666/sunpy/map/mapbase.py#L1485-L1488

I concur that we ought to convert an integer array to floats prior to the `np.pad()` call, since it's going to get converted to floats anyway in the subsequent `sunpy.image.transform.affine_transform()` call.
> Ah, yes, the problematic code is in `sunpy.map.GenericMap.rotate()` (as opposed to `sunpy.image.transform.affine_transform()`):
> 
> https://github.com/sunpy/sunpy/blob/6b3fdfb45d3d8687a16eb193fd87e8a27881d666/sunpy/map/mapbase.py#L1485-L1488
> 
> I concur that we ought to convert an integer array to floats prior to the `np.pad()` call, since it's going to get converted to floats anyway in the subsequent `sunpy.image.transform.affine_transform()` call.

Shall I make a PR regarding this then?

Sounds good 👍 (and we should add a regression test too)

Resolving patch:
diff --git a/sunpy/map/mapbase.py b/sunpy/map/mapbase.py
--- a/sunpy/map/mapbase.py
+++ b/sunpy/map/mapbase.py
@@ -3,6 +3,7 @@
 """
 import copy
 import html
+import numbers
 import textwrap
 import warnings
 import webbrowser
@@ -1482,6 +1483,12 @@ def rotate(self, angle: u.deg = None, rmatrix=None, order=4, scale=1.0,
         pad_x = int(np.max((diff[1], 0)))
         pad_y = int(np.max((diff[0], 0)))
 
+        if issubclass(self.data.dtype.type, numbers.Integral) and (missing % 1 != 0):
+            warnings.warn(
+                "The specified `missing` value is not an integer, but the data "
+                "array is of integer type, so the output may be strange.",
+                SunpyUserWarning)
+
         new_data = np.pad(self.data,
                           ((pad_y, pad_y), (pad_x, pad_x)),
                           mode='constant',


Test patch:
diff --git a/sunpy/map/tests/test_mapbase.py b/sunpy/map/tests/test_mapbase.py
--- a/sunpy/map/tests/test_mapbase.py
+++ b/sunpy/map/tests/test_mapbase.py
@@ -8,6 +8,7 @@
 import matplotlib.pyplot as plt
 import numpy as np
 import pytest
+from packaging import version
 
 import astropy.units as u
 import astropy.wcs
@@ -740,6 +741,20 @@ def test_rotate(aia171_test_map):
     assert aia171_test_map_crop_rot.data.shape[0] < aia171_test_map_crop_rot.data.shape[1]
 
 
+@pytest.mark.xfail(version.parse(np.__version__) >= version.parse("1.2.0"),
+                   reason="Numpy >= 1.20.0 doesn't allow NaN to int conversion")
+def test_rotate_with_incompatible_missing_dtype():
+    data = np.arange(0, 100).reshape(10, 10)
+    coord = SkyCoord(0 * u.arcsec, 0 * u.arcsec, obstime='2013-10-28',
+                     observer='earth', frame=sunpy.coordinates.Helioprojective)
+    header = sunpy.map.make_fitswcs_header(data, coord)
+    test_map = sunpy.map.Map(data, header)
+    with pytest.warns(SunpyUserWarning,
+                      match="The specified `missing` value is not an integer, but the data "
+                      "array is of integer type, so the output may be strange."):
+        test_map.rotate(order=3, missing=np.nan)
+
+
 def test_rotate_pad_crpix(generic_map):
     rotated_map = generic_map.rotate(30*u.deg)
     # This tests that the reference pixel of the map is in the expected place.