Repository: lightkurve/lightkurve

Issue:
CDIPS light curves can not be stitched
<!-- Fill in the information below before opening an issue. -->

#### Problem description
Light curves from the `CDIPS` author can not be stitched together and produce a lot of warnings/errors when trying. This might indicate something is wrong with the reader for that product. This probably also means these light curves are not usable in general by the community.

#### Example
<!-- Provide a link or minimal code snippet that demonstrates the issue. -->
```python
import lightkurve as lk
lc = lk.search_lightcurve('TIC 30031439', mission="TESS", author='CDIPS').download_all().stitch()
```

```python-traceback
WARNING: column quality has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq1 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq2 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq3 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column rstfc has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column quality has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq1 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq2 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq3 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column rstfc has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column quality has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq1 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq2 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq3 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column rstfc has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column quality has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq1 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq2 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq3 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column rstfc has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column quality has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq1 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq2 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq3 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column rstfc has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column quality has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq1 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq2 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq3 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column rstfc has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column quality has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq1 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq2 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq3 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column rstfc has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column quality has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq1 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq2 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq3 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column rstfc has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column quality has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq1 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq2 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq3 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column rstfc has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column quality has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq1 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq2 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq3 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column rstfc has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column quality has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq1 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq2 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq3 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column rstfc has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column quality has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq1 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq2 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq3 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column rstfc has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column quality has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq1 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq2 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column irq3 has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
WARNING: column rstfc has a unit but is kept as a Column as an attempt to convert it to Quantity failed with:
TypeError('The value must be a valid Python or Numpy numeric type.') [astropy.table.table]
---------------------------------------------------------------------------
UnitConversionError                       Traceback (most recent call last)
File /opt/conda/envs/tess/lib/python3.8/site-packages/astropy/units/quantity.py:959, in Quantity.to_value(self, unit, equivalencies)
    958 try:
--> 959     scale = self.unit._to(unit)
    960 except Exception:
    961     # Short-cut failed; try default (maybe equivalencies help).

File /opt/conda/envs/tess/lib/python3.8/site-packages/astropy/units/core.py:1129, in UnitBase._to(self, other)
   1127         return self_decomposed.scale / other_decomposed.scale
-> 1129 raise UnitConversionError(f"'{self!r}' is not a scaled version of '{other!r}'")

UnitConversionError: 'UnrecognizedUnit(ADU)' is not a scaled version of 'UnrecognizedUnit(ADU)'

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
Cell In[21], line 2
      1 import lightkurve as lk
----> 2 lc = lk.search_lightcurve('TIC 30031439', mission="TESS", author='CDIPS').download_all().stitch()

File /opt/conda/envs/tess/lib/python3.8/site-packages/lightkurve/collections.py:224, in LightCurveCollection.stitch(self, corrector_func)
    218     [
    219         lc.remove_columns(columns_to_remove.intersection(lc.columns))
    220         for lc in lcs
    221     ]
    223 # Need `join_type='inner'` until AstroPy supports masked Quantities
--> 224 return vstack(lcs, join_type="inner", metadata_conflicts="silent")

File /opt/conda/envs/tess/lib/python3.8/site-packages/astropy/table/operations.py:688, in vstack(tables, join_type, metadata_conflicts)
    685     return tables[0]  # no point in stacking a single table
    686 col_name_map = OrderedDict()
--> 688 out = _vstack(tables, join_type, col_name_map, metadata_conflicts)
    690 # Merge table metadata
    691 _merge_table_meta(out, tables, metadata_conflicts=metadata_conflicts)

File /opt/conda/envs/tess/lib/python3.8/site-packages/astropy/table/operations.py:1508, in _vstack(arrays, join_type, col_name_map, metadata_conflicts)
   1506 idx1 = idx0 + len(array)
   1507 if name in array.colnames:
-> 1508     col[idx0:idx1] = array[name]
   1509 else:
   1510     # If col is a Column but not MaskedColumn then upgrade at this point
   1511     # because masking is required.
   1512     if isinstance(col, Column) and not isinstance(col, MaskedColumn):

File /opt/conda/envs/tess/lib/python3.8/site-packages/astropy/units/quantity.py:1301, in Quantity.__setitem__(self, i, value)
   1299 if not self.isscalar and "info" in self.__dict__:
   1300     self.info.adjust_indices(i, value, len(self))
-> 1301 self.view(np.ndarray).__setitem__(i, self._to_own_unit(value))

File /opt/conda/envs/tess/lib/python3.8/site-packages/astropy/units/quantity.py:1631, in Quantity._to_own_unit(self, value, check_precision, unit)
   1629     unit = self.unit
   1630 try:
-> 1631     _value = value.to_value(unit)
   1632 except AttributeError:
   1633     # We're not a Quantity.
   1634     # First remove two special cases (with a fast test):
   1635     # 1) Maybe masked printing? MaskedArray with quantities does not
   1636     # work very well, but no reason to break even repr and str.
   1637     # 2) np.ma.masked? useful if we're a MaskedQuantity.
   1638     if value is np.ma.masked or (
   1639         value is np.ma.masked_print_option and self.dtype.kind == "O"
   1640     ):

File /opt/conda/envs/tess/lib/python3.8/site-packages/astropy/units/quantity.py:962, in Quantity.to_value(self, unit, equivalencies)
    959     scale = self.unit._to(unit)
    960 except Exception:
    961     # Short-cut failed; try default (maybe equivalencies help).
--> 962     value = self._to_value(unit, equivalencies)
    963 else:
    964     value = self.view(np.ndarray)

File /opt/conda/envs/tess/lib/python3.8/site-packages/astropy/units/quantity.py:868, in Quantity._to_value(self, unit, equivalencies)
    865     equivalencies = self._equivalencies
    866 if not self.dtype.names or isinstance(self.unit, StructuredUnit):
    867     # Standard path, let unit to do work.
--> 868     return self.unit.to(
    869         unit, self.view(np.ndarray), equivalencies=equivalencies
    870     )
    872 else:
    873     # The .to() method of a simple unit cannot convert a structured
    874     # dtype, so we work around it, by recursing.
    875     # TODO: deprecate this?
    876     # Convert simple to Structured on initialization?
    877     result = np.empty_like(self.view(np.ndarray))

File /opt/conda/envs/tess/lib/python3.8/site-packages/astropy/units/core.py:1165, in UnitBase.to(self, other, value, equivalencies)
   1163     return UNITY
   1164 else:
-> 1165     return self._get_converter(Unit(other), equivalencies)(value)

File /opt/conda/envs/tess/lib/python3.8/site-packages/astropy/units/core.py:1989, in UnrecognizedUnit._get_converter(self, other, equivalencies)
   1987 def _get_converter(self, other, equivalencies=None):
   1988     self._normalize_equivalencies(equivalencies)
-> 1989     raise ValueError(
   1990         f"The unit {self.name!r} is unrecognized.  It can not be converted "
   1991         "to other units."
   1992     )

ValueError: The unit 'ADU' is unrecognized.  It can not be converted to other units.
```

#### Expected behavior
<!-- Describe the behavior you expected and how it differs from the behavior observed in the example. -->

It should be possible to stitch these light curves

#### Environment

-  platform TIKE
-  lightkurve version 2.3
-  installation method pip


Discussion:

Patch:
diff --git a/src/lightkurve/io/generic.py b/src/lightkurve/io/generic.py
index f571d0e2..d051df68 100644
--- a/src/lightkurve/io/generic.py
+++ b/src/lightkurve/io/generic.py
@@ -67,7 +67,13 @@ def read_generic_lightcurve(
         elif unitstr == "ppm" and repr(tab[colname].unit).startswith("Unrecognized"):
             # Workaround for issue #956
             tab[colname].unit = ppm
-
+        elif unitstr == "ADU":
+            tab[colname].unit = "adu"
+        elif unitstr.lower() == "unitless":
+            tab[colname].unit = ""
+        elif unitstr.lower() == "degcelcius":
+            # CDIPS has non-astropy units
+            tab[colname].unit = "deg_C"
         # Rename columns to lowercase
         tab.rename_column(colname, colname.lower())
 


Test patch:
diff --git a/tests/io/test_cdips.py b/tests/io/test_cdips.py
index c67bff6d..b9f0ea7e 100644
--- a/tests/io/test_cdips.py
+++ b/tests/io/test_cdips.py
@@ -4,7 +4,7 @@ from astropy.io import fits
 import numpy as np
 from numpy.testing import assert_array_equal
 
-from lightkurve import search_lightcurve
+from lightkurve import search_lightcurve, LightCurveCollection
 from lightkurve.io.cdips import read_cdips_lightcurve
 from lightkurve.io.detect import detect_filetype
 
@@ -59,3 +59,7 @@ def test_search_cdips():
     lc = search.download()
     assert type(lc).__name__ == "TessLightCurve"
     assert hasattr(lc, "sector")
+    assert str(lc['bge'].unit) == 'adu'
+    slc = LightCurveCollection([lc, lc]).stitch()
+    assert len(slc) == 2*len(lc)
+