Repository: RadioAstronomySoftwareGroup/pyuvdata
Pull request #641 (2019-07-15 05:23:24)

uvcal ant2ind bug fix
<!--- Provide a general summary of your changes in the Title above -->
Bug fix in UVCal.ant2ind

## Description
<!--- Describe your changes in detail -->
Bug in UVCal.ant2ind caused indexing to always go to zeroth antenna in ant_array. Fixed this to do indexing properly and added a unit test to catch this.

## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here. If this PR closes an issue, put the word 'closes' before the issue link to auto-close the issue when the PR is merged. -->

## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Documentation change (documentation changes only)
- [ ] Version change
- [ ] Build or continuous integration change


## Checklist:
<!--- You may remove the checklists that don't apply to your change type(s) or just leave them empty -->
<!--- Go over all the following points, and replace the space with an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] I have read the [contribution guide](https://github.com/RadioAstronomySoftwareGroup/pyuvdata/blob/master/.github/CONTRIBUTING.md).
- [x] My code follows the code style of this project.

Bug fix checklist:
- [x] My fix includes a new test that breaks as a result of the bug (if possible).
- [ ] All new and existing tests pass in both python 2 and python 3.
- [x] I have updated the [CHANGELOG](https://github.com/RadioAstronomySoftwareGroup/pyuvdata/blob/master/CHANGELOG.md).

New feature checklist:
- [ ] I have added or updated the docstrings associated with my feature using the [numpy docstring format](https://numpydoc.readthedocs.io/en/latest/format.html).
- [ ] I have updated the tutorial to highlight my new feature (if appropriate).
- [ ] I have added tests to cover my new feature.
- [ ] All new and existing tests pass in both python 2 and python 3.
- [ ] I have updated the [CHANGELOG](https://github.com/RadioAstronomySoftwareGroup/pyuvdata/blob/master/CHANGELOG.md).

Breaking change checklist:  
- [ ] I have updated the docstrings associated with my change using the [numpy docstring format](https://numpydoc.readthedocs.io/en/latest/format.html).
- [ ] I have updated the tutorial to reflect my changes (if appropriate).
- [ ] My change includes backwards compatibility and deprecation warnings (if possible).
- [ ] I have added tests to cover my changes.
- [ ] All new and existing tests pass in both python 2 and python 3.
- [ ] I have updated the [CHANGELOG](https://github.com/RadioAstronomySoftwareGroup/pyuvdata/blob/master/CHANGELOG.md).

Documentation change checklist:
- [ ] Any updated docstrings use the [numpy docstring format](https://numpydoc.readthedocs.io/en/latest/format.html).
- [ ] If this is a significant change to the readme or other docs, I have checked that they are rendered properly on ReadTheDocs. (you may need help to get this branch to build on RTD, just ask!)

Version change checklist:
- [ ] I have updated the [CHANGELOG](https://github.com/RadioAstronomySoftwareGroup/pyuvdata/blob/master/CHANGELOG.md) to put all the unreleased changes under the new version (leaving the unreleased section empty).
- [ ] I have noted any dependency changes since the last version and will update the conda package build accordingly.

Build or continuous integration change checklist:
- [ ] If required or optional dependencies have changed (including version numbers), I have updated the readme to reflect this.
- [ ] If this is a new CI setup, I have added the associated badge to the readme and to references/make_index.py (if appropriate).


bhazelton: @nkern can you rebase this on master? We've fixed the hera_qm problem on master.
codecov[bot]: # [Codecov](https://codecov.io/gh/RadioAstronomySoftwareGroup/pyuvdata/pull/641?src=pr&el=h1) Report
> Merging [#641](https://codecov.io/gh/RadioAstronomySoftwareGroup/pyuvdata/pull/641?src=pr&el=desc) into [master](https://codecov.io/gh/RadioAstronomySoftwareGroup/pyuvdata/commit/de776a5981a2ae95a8b0b9e4219ccf9b6091895c?src=pr&el=desc) will **decrease** coverage by `0.17%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/RadioAstronomySoftwareGroup/pyuvdata/pull/641/graphs/tree.svg?width=650&token=4QykcCeB1H&height=150&src=pr)](https://codecov.io/gh/RadioAstronomySoftwareGroup/pyuvdata/pull/641?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #641      +/-   ##
==========================================
- Coverage   98.88%   98.71%   -0.18%     
==========================================
  Files          20       20              
  Lines        8141     8141              
==========================================
- Hits         8050     8036      -14     
- Misses         91      105      +14
```


| [Impacted Files](https://codecov.io/gh/RadioAstronomySoftwareGroup/pyuvdata/pull/641?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pyuvdata/uvcal.py](https://codecov.io/gh/RadioAstronomySoftwareGroup/pyuvdata/pull/641/diff?src=pr&el=tree#diff-cHl1dmRhdGEvdXZjYWwucHk=) | `100% <100%> (ø)` | :arrow_up: |
| [pyuvdata/version.py](https://codecov.io/gh/RadioAstronomySoftwareGroup/pyuvdata/pull/641/diff?src=pr&el=tree#diff-cHl1dmRhdGEvdmVyc2lvbi5weQ==) | `96.61% <0%> (-3.39%)` | :arrow_down: |
| [pyuvdata/uvh5.py](https://codecov.io/gh/RadioAstronomySoftwareGroup/pyuvdata/pull/641/diff?src=pr&el=tree#diff-cHl1dmRhdGEvdXZoNS5weQ==) | `99.19% <0%> (-0.81%)` | :arrow_down: |
| [pyuvdata/utils.py](https://codecov.io/gh/RadioAstronomySoftwareGroup/pyuvdata/pull/641/diff?src=pr&el=tree#diff-cHl1dmRhdGEvdXRpbHMucHk=) | `99.2% <0%> (-0.8%)` | :arrow_down: |
| [pyuvdata/aipy\_extracts.py](https://codecov.io/gh/RadioAstronomySoftwareGroup/pyuvdata/pull/641/diff?src=pr&el=tree#diff-cHl1dmRhdGEvYWlweV9leHRyYWN0cy5weQ==) | `84.9% <0%> (-0.63%)` | :arrow_down: |
| [pyuvdata/uvdata.py](https://codecov.io/gh/RadioAstronomySoftwareGroup/pyuvdata/pull/641/diff?src=pr&el=tree#diff-cHl1dmRhdGEvdXZkYXRhLnB5) | `99.87% <0%> (-0.13%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/RadioAstronomySoftwareGroup/pyuvdata/pull/641?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/RadioAstronomySoftwareGroup/pyuvdata/pull/641?src=pr&el=footer). Last update [de776a5...f3bc99d](https://codecov.io/gh/RadioAstronomySoftwareGroup/pyuvdata/pull/641?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

nkern: @bhazelton @adampbeardsley I think this is ready to go? lmk if there is anything else needed for this one
bhazelton: I'm trying to figure out why the test failed on Mac, python 3.6
bhazelton: I'm having a hard time replicating the error, but I already tried re-running that job and it had the same error, so I don't think it's transient. I also don't see how it's connected to the changes in this PR. I'm going to try rebasing this branch on master, just to see if that somehow fixes it.

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f581386f..3d5fd40f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,8 @@
 All notable changes to this project will be documented in this file.
 
 ## [Unreleased]
+- `utils.uvcalibrate` flag propagation bug fix
+- `UVCal.ant2ind` indexing bug fix
 - `UVCal.get_*` methods for accessing data arrays with antenna-polarization keys
 - `utils.uvcalibrate` for automated calibration of `UVData` by `UVCal`
 
diff --git a/pyuvdata/utils.py b/pyuvdata/utils.py
index 3a669b19..11e78dd6 100644
--- a/pyuvdata/utils.py
+++ b/pyuvdata/utils.py
@@ -464,12 +464,11 @@ def uvcalibrate(uvdata, uvcal, inplace=True, prop_flags=True, flag_missing=True,
                 gain[mask] = 1.0
                 uvdata.flag_array[blt_inds, 0, :, pol_ind] += mask
 
-            else:
-                # apply to data
-                if uvcal.gain_convention == 'multiply':
-                    uvdata.data_array[blt_inds, 0, :, pol_ind] *= gain
-                elif uvcal.gain_convention == 'divide':
-                    uvdata.data_array[blt_inds, 0, :, pol_ind] /= gain
+            # apply to data
+            if uvcal.gain_convention == 'multiply':
+                uvdata.data_array[blt_inds, 0, :, pol_ind] *= gain
+            elif uvcal.gain_convention == 'divide':
+                uvdata.data_array[blt_inds, 0, :, pol_ind] /= gain
 
     # update history
     uvdata.history += "\nCalibrated with pyuvdata.utils.uvcalibrate."
diff --git a/pyuvdata/uvcal.py b/pyuvdata/uvcal.py
index 41f51b17..6f86fa83 100644
--- a/pyuvdata/uvcal.py
+++ b/pyuvdata/uvcal.py
@@ -738,7 +738,7 @@ class UVCal(UVBase):
         if not self._has_key(antnum=antnum):
             raise ValueError("{} not found in ant_array".format(antnum))
 
-        return np.argmin(self.ant_array - antnum)
+        return np.argmin(np.abs(self.ant_array - antnum))
 
     def jpol2ind(self, jpol):
         """


Test patch:
diff --git a/pyuvdata/tests/test_utils.py b/pyuvdata/tests/test_utils.py
index dd405843..01ce1e4e 100644
--- a/pyuvdata/tests/test_utils.py
+++ b/pyuvdata/tests/test_utils.py
@@ -948,7 +948,7 @@ def test_uvcalibrate_apply_gains():
 
     # division calibrate
     uvc.gain_convention = 'divide'
-    uvdcal = uvutils.uvcalibrate(uvd, uvc, prop_flags=False, flag_missing=False, inplace=False)
+    uvdcal = uvutils.uvcalibrate(uvd, uvc, prop_flags=True, flag_missing=False, inplace=False)
     np.testing.assert_array_almost_equal(uvdcal.get_data(key), uvd.get_data(key) / (uvc.get_gains(ant1) * uvc.get_gains(ant2).conj()).T)
 
     # multiplication calibrate
diff --git a/pyuvdata/tests/test_uvcal.py b/pyuvdata/tests/test_uvcal.py
index c4392f22..5f8b14bc 100644
--- a/pyuvdata/tests/test_uvcal.py
+++ b/pyuvdata/tests/test_uvcal.py
@@ -1442,17 +1442,19 @@ def test_uvcal_get_methods():
     uvc = UVCal()
     uvc.read_calfits(os.path.join(DATA_PATH, 'zen.2457698.40355.xx.gain.calfits'))
 
-    # test get methods
-    key = (9, 'Jxx')
+    # test get methods: add in a known value and make sure it is returned
+    key = (10, 'Jxx')
+    uvc.gain_array[1] = 0.0
     d, f, q = uvc.get_gains(key), uvc.get_flags(key), uvc.get_quality(key)
 
     # test shapes
+    assert np.all(np.isclose(d, 0.0))
     assert d.shape == (uvc.Nfreqs, uvc.Ntimes)
     assert f.shape == (uvc.Nfreqs, uvc.Ntimes)
     assert q.shape == (uvc.Nfreqs, uvc.Ntimes)
 
     # test against by-hand indexing
-    np.testing.assert_array_almost_equal(d, uvc.gain_array[uvc.ant_array.tolist().index(9), 0, :, :, uvc.jones_array.tolist().index(-5)])
+    np.testing.assert_array_almost_equal(d, uvc.gain_array[uvc.ant_array.tolist().index(10), 0, :, :, uvc.jones_array.tolist().index(-5)])
 
     # test variable key input
     d2 = uvc.get_gains(*key)
@@ -1461,23 +1463,23 @@ def test_uvcal_get_methods():
     np.testing.assert_array_almost_equal(d, d2)
     d2 = uvc.get_gains(key[:1])
     np.testing.assert_array_almost_equal(d, d2)
-    d2 = uvc.get_gains(9, -5)
+    d2 = uvc.get_gains(10, -5)
     np.testing.assert_array_almost_equal(d, d2)
-    d2 = uvc.get_gains(9, 'x')
+    d2 = uvc.get_gains(10, 'x')
     np.testing.assert_array_almost_equal(d, d2)
 
     # check has_key
-    assert uvc._has_key(antnum=9)
+    assert uvc._has_key(antnum=10)
     assert uvc._has_key(jpol='Jxx')
-    assert uvc._has_key(antnum=9, jpol='Jxx')
-    assert not uvc._has_key(antnum=9, jpol='Jyy')
+    assert uvc._has_key(antnum=10, jpol='Jxx')
+    assert not uvc._has_key(antnum=10, jpol='Jyy')
     assert not uvc._has_key(antnum=101, jpol='Jxx')
 
     # test exceptions
     pytest.raises(ValueError, uvc.get_gains, 1)
-    pytest.raises(ValueError, uvc.get_gains, (9, 'Jyy'))
+    pytest.raises(ValueError, uvc.get_gains, (10, 'Jyy'))
     uvc.cal_type = 'delay'
-    pytest.raises(ValueError, uvc.get_gains, 9)
+    pytest.raises(ValueError, uvc.get_gains, 10)
 
 
 def test_deprecated_x_orientation():