Repository: AllenCellModeling/aicspylibczi (language: python)

Issue:
ERROR: no matching distribution found for aicspylibczi
I'm receiving this error on any 2.7 release while installing as a dependency (or individually) from aicsimageio

This looks exactly the same as an issue from january 2020, except this time with the 2.7 release: #28 

```
ERROR: Could not find a version that satisfies the requirement aicspylibczi~=2.7.5 (from aicsimageio)
ERROR: No matching distribution found for aicspylibczi~=2.7.5
```

similar env as the user earlier:

```
            shell level : 1
       user config file : /Users/bryant.chhun/.condarc
 populated config files : /Users/bryant.chhun/.condarc
          conda version : 4.8.5
    conda-build version : 3.10.5
         python version : 3.6.11.final.0
       virtual packages : __osx=10.13.6
       base environment : /Users/bryant.chhun/anaconda3  (writable)
           channel URLs : https://conda.anaconda.org/talley/osx-64
                          https://conda.anaconda.org/talley/noarch
                          https://conda.anaconda.org/conda-forge/osx-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/osx-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-64
                          https://repo.anaconda.com/pkgs/r/noarch
               platform : osx-64
             user-agent : conda/4.8.5 requests/2.22.0 CPython/3.6.11 Darwin/17.7.0 OSX/10.13.6
                UID:GID : 502:20
             netrc file : None
           offline mode : False
```

```
python 3.7.9
pip 20.3.1
```


Discussion / hints:
Hi @bryantChhun, umm first off it looks like your actually on python 3.6 not 3.7 from your environment output but that shouldn't be a big deal. I would recommend using 3.7 or later generally. Given the output, I'm assuming you're on macOS? 

We use pip for installs and don't directly publish a conda package. 
Try `pip install aicspylibczi` after activating your conda environment and it should work. 
If for some reason it doesn't please respond here and I'll try to get back to you right away. 

@heeler Thanks for the response.  `pip install aicspylibczi` works but defaults to version 2.6.  I noticed the problem when installing `aicsimageio`, which specifies `aicspylibczi >= 2.7.5` as a dependency.

If i try installing 2.7.6 using pip like here https://pypi.org/project/aicspylibczi/2.7.6/, i will get the above error
I'm having a really hard time reproducing your issue. 

can you check to make sure there isn't a pip.conf somewhere that's having impact here? 
```
Per-user:

On Unix the default configuration file is: $HOME/.config/pip/pip.conf which respects the XDG_CONFIG_HOME environment variable.

On macOS the configuration file is $HOME/Library/Application Support/pip/pip.conf if directory $HOME/Library/Application Support/pip exists else $HOME/.config/pip/pip.conf.
There is also a legacy per-user configuration file which is also respected. To find its location:

On Unix and macOS the configuration file is: $HOME/.pip/pip.conf
```

Also so we're on the same page could you try this series of commands 
```
conda create -n testThis python=3.7
conda activate testThis
pip install aicsimageio
``` 
If this doesn't work please post the output back here. 

On every attempt I make it installs aicspylibczi=2.7.6 cleanly. 

I can install aicsimageio using the above commands, and even used `--no-cache-dir` to make sure it's fresh.  It still defaults to 2.6 (and aicsimageio will default to 3.3.1, which has an older dependency)

![image](https://user-images.githubusercontent.com/25647712/104057941-cca9cb00-51a7-11eb-9211-9f4c81b74ac8.png)

If i try to install 3.3.3, I get the same error:

![image](https://user-images.githubusercontent.com/25647712/104059412-27442680-51aa-11eb-8074-c5cd222b0ff3.png)

Additionally, I can not find any pip.conf anywhere in the paths you listed above.

Could this be a mac OS version issue?
It's possible it could be an OS version issue but my build file contains the following statement
```
    IF (APPLE)
        set(CMAKE_OSX_DEPLOYMENT_TARGET "10.13" CACHE STRING "Minimum OS X deployment version" FORCE)
    ENDIF(APPLE)
```
Which I believe sets 10.13 as the minimum supported version. 


Let's try forcing the version for aicspylibczi and then if it works installing aicsimageio
```
conda activate testThis
pip install aicspylibczi==2.7.6
pip install aicsimageio
```


ok I've been dense. I went to manually download the files and realized the file name is explicitly aicspylibczi-2.7.6-cp37-cp37m-macosx_10_14_x86_64.whl  which given that's generated by the python package build tools suggests that 10_14 is the minimum supported macOS level. I have another PR that will become aicspylib-2.7.7 and I will attempt to get it to build for 10.13. I'm not sure if this a consequence of github actions build environment or what but I'll try to get it sorted for you. 
Ah thank you!  I hope this was more helpful than painful.  In the meantime I am more encouraged to upgrade my OS :-)

@heeler We may want to build aicspylibczi on more platforms

https://docs.github.com/en/free-pro-team@latest/actions/reference/specifications-for-github-hosted-runners#supported-runners-and-hardware-resources

`macos-11.0` and `macos-10.15` / `macos-latest` are different.
@JacksonMaxfield This is what throws me though. Generally for building binaries (the C++ target) the compiler flags/libraries you link against (target) sets the minimal macOS version. In C++ build instructions, we explicitly say 10.13. Somewhere along the way, it's now targeting 10.14 in the wheel that gets built. 

To your main point, I have no issue with expanding the build matrix. I'm just not 100% sure where the constraint is. The current version is built on 10.15 but the wheel is compatible with 10.14 and 10.15 just not 10.13. 🤷
Oh yea. My comment wasn't so much about the current 10.13 issue but rather the general "I don't think we are building on 11.0" yet issue.

Fix patch:
diff --git a/.github/workflows/MacOSbacksupport.yml b/.github/workflows/MacOSbacksupport.yml
new file mode 100644
index 00000000..4e93026e
--- /dev/null
+++ b/.github/workflows/MacOSbacksupport.yml
@@ -0,0 +1,45 @@
+name: Wheel build test
+on:
+  push:
+    branches:
+      - 'release/aicspylibczi'
+
+jobs:
+  wheel-setup:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [macOS-10.15]
+
+    steps:
+      - uses: actions/checkout@v1
+        with:
+          submodules: true
+      - name: Build Miniconda
+        run: |
+          wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh -O ~/miniconda.sh
+          bash ~/miniconda.sh -b -p $HOME/miniconda
+          export PATH=~/miniconda/bin:$PATH
+          $HOME/miniconda/bin/conda init bash
+      - name: Build 3.6
+        run: |
+          bash scripts/setupWheelEnv.sh -p 3.6 -f 3_6 -t 10.9
+      - name: Build 3.7
+        run: |
+          bash scripts/setupWheelEnv.sh -p 3.7 -f 3_7 -t 10.9
+      - name: Build 3.8
+        run: |
+          bash scripts/setupWheelEnv.sh -p 3.8 -f 3_8 -t 10.9
+      - name: Build 3.9
+        run: |
+          bash scripts/setupWheelEnv.sh -p 3.9 -f 3_9 -t 10.9
+      - name: Publish to PyPi
+        run: |
+          source ~/.bashrc
+          source ~/.bash_profile
+          export PATH=~/miniconda/bin:$PATH
+          conda activate jamies3_9
+          pip install twine
+          pip install --user --upgrade twine
+          twine upload --repository-url https://upload.pypi.org/legacy/ -u aicspypi -p '${{ secrets.PYPI_TOKEN }}' dist/*
+          ls dist/*
diff --git a/.github/workflows/Publish.yml b/.github/workflows/Publish.yml
index 6d3a9923..63a180c8 100644
--- a/.github/workflows/Publish.yml
+++ b/.github/workflows/Publish.yml
@@ -10,7 +10,7 @@ jobs:
     strategy:
       matrix:
         python-version: [3.6, 3.7, 3.8, 3.9]
-        os: [windows-2019, macOS-10.15, macOS-11.0]
+        os: [windows-2019]
 
     steps:
       - uses: actions/checkout@v1
diff --git a/_aicspylibczi/ImageFactory.cpp b/_aicspylibczi/ImageFactory.cpp
index b2d68e73..2821b97e 100644
--- a/_aicspylibczi/ImageFactory.cpp
+++ b/_aicspylibczi/ImageFactory.cpp
@@ -165,8 +165,8 @@ ImageFactory::constructImage(const std::shared_ptr<libCZI::IBitmapData>& bitmap_
   PixelType pixelType = bitmap_ptr_->GetPixelType();
 
   std::vector<size_t> shape;
-  size_t channels = numberOfChannels(pixelType);
-  if (channels == 3)
+  size_t samples_per_pixel = numberOfChannels(pixelType);
+  if (samples_per_pixel == 3)
     shape.emplace_back(3);
   shape.emplace_back(size_.h);
   shape.emplace_back(size_.w);
@@ -176,7 +176,7 @@ ImageFactory::constructImage(const std::shared_ptr<libCZI::IBitmapData>& bitmap_
     imageFactoryFunction(shape, pixelType, plane_coordinate_, box_, m_imgContainer.get(), mem_index_, index_m_);
   if (image == nullptr)
     throw std::bad_alloc();
-  image->loadImage(bitmap_ptr_, size_, channels);
+  image->loadImage(bitmap_ptr_, size_, samples_per_pixel);
   m_imgContainer->addImage(image);
   return image;
 }
diff --git a/_aicspylibczi/Reader.cpp b/_aicspylibczi/Reader.cpp
index 1d44a79e..9464f78f 100644
--- a/_aicspylibczi/Reader.cpp
+++ b/_aicspylibczi/Reader.cpp
@@ -292,7 +292,7 @@ Reader::readSelected(libCZI::CDimCoordinate& plane_coord_, int index_m_, unsigne
   size_t bgrScaling = ImageFactory::numberOfChannels(m_pixelType);
 
   libCZI::IntRect w_by_h = getSceneYXSize();
-  size_t n_of_pixels = matches.size() * w_by_h.w * w_by_h.h * bgrScaling;
+  size_t n_of_pixels = matches.size() * w_by_h.w * w_by_h.h; // bgrScaling is handled internally * bgrScaling;
   ImageFactory imageFactory(m_pixelType, n_of_pixels);
 
   imageFactory.setMosaic(isMosaic());
@@ -324,7 +324,7 @@ Reader::readSelected(libCZI::CDimCoordinate& plane_coord_, int index_m_, unsigne
 
         auto bitmap = subblock->CreateBitmap();
         libCZI::IntSize size = bitmap->GetSize();
-
+        // constructImage fixes BRG image data now via channels != 3 condition
         imageFactory.constructImage(bitmap, size, &info.coordinate, info.logicalRect, memOffset, info.mIndex);
         return true;
       }));
diff --git a/_aicspylibczi/TypedImage.h b/_aicspylibczi/TypedImage.h
index 45e62757..b3c33016 100644
--- a/_aicspylibczi/TypedImage.h
+++ b/_aicspylibczi/TypedImage.h
@@ -85,12 +85,12 @@ class TypedImage : public Image
   /*!
    * @brief Copy the image from the libCZI bitmap object into this Image object
    * @param bitmap_ptr_ is the image bitmap from libCZI
-   * @param channels_ the number of channels 1 for GrayX, 3 for BgrX etc. (ie
+   * @param samples_per_pixel_ the number of channels 1 for GrayX, 3 for BgrX etc. (ie
    * the number of XY planes required to hold the image)
    */
   void loadImage(const std::shared_ptr<libCZI::IBitmapData>& bitmap_ptr_,
                  libCZI::IntSize size_,
-                 size_t channels_) override;
+                 size_t samples_per_pixel_) override;
 
   // TODO Implement set_sort_order() and operator()<
 
@@ -119,19 +119,21 @@ template<typename T>
 inline void
 TypedImage<T>::loadImage(const std::shared_ptr<libCZI::IBitmapData>& bitmap_ptr_,
                          libCZI::IntSize size_,
-                         size_t channels_)
+                         size_t samples_per_pixel_)
 {
   libCZI::ScopedBitmapLockerP lckScoped{ bitmap_ptr_.get() };
   // WARNING do not compute the end of the array by multiplying stride by
   // height, they are both uint32_t and you'll get an overflow for larger images
   uint8_t* sEnd = static_cast<uint8_t*>(lckScoped.ptrDataRoi) + lckScoped.size;
   if ((lckScoped.stride % size_.w == 0) && ( m_pixelType == libCZI::PixelType::Gray8 || m_pixelType == libCZI::PixelType::Gray16 ||
-      m_pixelType == libCZI::PixelType::Gray32Float)) {
+      m_pixelType == libCZI::PixelType::Gray32Float) &&
+      samples_per_pixel_ != 3 ) {
     std::memcpy(m_array, lckScoped.ptrDataRoi, lckScoped.size);
   }
   else {
-    SourceRange<T> sourceRange(channels_, static_cast<T*>(lckScoped.ptrDataRoi), (T*)(sEnd), lckScoped.stride, size_.w);
-    TargetRange<T> targetRange(channels_, size_.w, size_.h, m_array, m_array + length());
+    SourceRange<T> sourceRange(
+      samples_per_pixel_, static_cast<T*>(lckScoped.ptrDataRoi), (T*)(sEnd), lckScoped.stride, size_.w);
+    TargetRange<T> targetRange(samples_per_pixel_, size_.w, size_.h, m_array, m_array + length());
     for (std::uint32_t h = 0; h < bitmap_ptr_->GetHeight(); ++h) {
       pairedForEach(sourceRange.strideBegin(h),
                     sourceRange.strideEnd(h),
diff --git a/scripts/setupWheelEnv.sh b/scripts/setupWheelEnv.sh
new file mode 100644
index 00000000..a0dd5623
--- /dev/null
+++ b/scripts/setupWheelEnv.sh
@@ -0,0 +1,25 @@
+#!/bin/bash
+
+while getopts p:f:t: option
+do
+  case "${option}"
+    in
+      p) PYTHON_VERSION=${OPTARG};;
+      f) ENV_NAME="jamies${OPTARG}";;
+      t) MACOS_TARGET=${OPTARG};;
+  esac
+done
+
+echo "PYTHON_VERSION=${PYTHON_VERSION}"
+echo "ENV_NAME=${ENV_NAME}"
+echo "MACOS_TARGET=${MACOS_TARGET}"
+
+source ~/.bashrc
+source ~/.bash_profile
+export PATH=~/miniconda/bin:$PATH
+~/miniconda/bin/conda create -y -n ${ENV_NAME} python=${PYTHON_VERSION}
+conda activate ${ENV_NAME}
+export SDKROOT=/Applications/Xcode_12.3.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
+export MACOSX_DEPLOYMENT_TARGET=${MACOS_TARGET}
+pip install wheel
+python setup.py bdist_wheel


Test patch:
diff --git a/aicspylibczi/tests/resources/RGB-8bit_Gplane.npy b/aicspylibczi/tests/resources/RGB-8bit_Gplane.npy
new file mode 100644
index 00000000..7c341faa
Binary files /dev/null and b/aicspylibczi/tests/resources/RGB-8bit_Gplane.npy differ
diff --git a/aicspylibczi/tests/test_czi_file.py b/aicspylibczi/tests/test_czi_file.py
index b91fadbf..4d4ef8ed 100644
--- a/aicspylibczi/tests/test_czi_file.py
+++ b/aicspylibczi/tests/test_czi_file.py
@@ -291,3 +291,15 @@ def test_mosaic_subblock_rect(data_dir, fname, s_index, m_index, expected):
         czi = CziFile(czi_filename=fp)
         data = czi.mosaic_scene_bounding_boxes(s_index)
         assert data == expected
+
+
+@pytest.mark.parametrize("fname, p_index, ans_file", [
+    ("RGB-8bit.czi", 1, "RGB-8bit_Gplane.npy")
+])
+def test_bgr_plane_data_x(data_dir, fname, p_index, ans_file):
+    ans = np.load(data_dir/ans_file)
+    with open(data_dir/fname, 'rb') as fp:
+        czi = CziFile(czi_filename=fp)
+        img, dims = czi.read_image()
+        assert img[0, p_index, :, :].shape == ans.shape
+        np.testing.assert_array_almost_equal(img[0, p_index, :, :], ans)
diff --git a/c_tests/test_Reader.cpp b/c_tests/test_Reader.cpp
index 52eb77f7..35227f44 100644
--- a/c_tests/test_Reader.cpp
+++ b/c_tests/test_Reader.cpp
@@ -405,12 +405,18 @@ TEST_CASE_METHOD(CziMCreator, "test_mosaic_throw", "[Reader_mosaic_throw]")
 class CziBgrCreator
 {
   std::unique_ptr<pylibczi::Reader> m_czi;
-
+  std::vector<int> m_firstPlaneFirstTen = {7, 6, 5, 6, 6, 7, 7, 7, 7, 7};
+  std::vector<int> m_secondPlaneFirstTen = {14,  8,  5,  4,  4,  5,  5,  5,  5,  5};
+  // y=0, x=0:10
 public:
   CziBgrCreator()
     : m_czi(new pylibczi::Reader(L"resources/RGB-8bit.czi"))
   {}
   pylibczi::Reader* get() { return m_czi.get(); }
+  const std::vector<int> & planeZeroFirstTen() { return m_firstPlaneFirstTen; }
+  const std::vector<int> & planeOneFirstTen() { return m_secondPlaneFirstTen; }
+
+
 };
 
 TEST_CASE_METHOD(CziBgrCreator, "test_bgr_read", "[Reader_read_bgr]")
@@ -433,6 +439,8 @@ TEST_CASE_METHOD(CziBgrCreator, "test_bgr_read", "[Reader_read_bgr]")
   REQUIRE(pr.size() == 1);
   REQUIRE(pr.front()->shape() == std::vector<size_t>{ 3, 624, 924 });
   REQUIRE(pr.front()->pixelType() == libCZI::PixelType::Gray8);
+
+
 }
 
 TEST_CASE_METHOD(CziBgrCreator, "test_bgr_flatten", "[Reader_read_flatten_bgr]")
@@ -457,6 +465,18 @@ TEST_CASE_METHOD(CziBgrCreator, "test_bgr_flatten", "[Reader_read_flatten_bgr]")
 
   REQUIRE(shape == shapeAns);
   REQUIRE(pr.front()->pixelType() == libCZI::PixelType::Gray8);
+
+  // [7, 6, 5, 6, 6, 7, 7, 7, 7, 7]
+  auto timage = pylibczi::ImageFactory::getDerived<uint8_t>(pr.front());
+  uint8_t *tmp = timage->getRawPtr();
+  std::vector<uint8_t> data(tmp, tmp+10);
+  auto planeZero = planeZeroFirstTen();
+  auto planeOne = planeOneFirstTen();
+  // check data values
+  for( size_t i = 0; i < 10; i++){
+    REQUIRE(data[i] == planeZero[i]);
+    REQUIRE( (uint8_t)((*timage)[{i, 0, 1}]) == planeOne[i] );
+  }
 }
 
 TEST_CASE_METHOD(CziMCreator, "test_reader_mosaic_subblockinforect", "[Reader_Mosaic_SubblockInfo_Rect]")