Custom Example template in torch Dataset

lampadephoria: <p>In the document,<br>
</p><aside class="onebox allowlistedgeneric">
  <header class="source">
      <a href="https://pytorch.org/cppdocs/api/classtorch_1_1data_1_1datasets_1_1_dataset.html#_CPPv4I00EN5torch4data8datasets7DatasetE" target="_blank" rel="noopener nofollow ugc">pytorch.org</a>
  </header>
  <article class="onebox-body">
    <img src="" class="thumbnail" width="" height="">

<h3><a href="https://pytorch.org/cppdocs/api/classtorch_1_1data_1_1datasets_1_1_dataset.html#_CPPv4I00EN5torch4data8datasets7DatasetE" target="_blank" rel="noopener nofollow ugc">Template Class Dataset — PyTorch master documentation</a></h3>



  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>get() return type is torch::data::Example&lt;&gt;. Example is a template with default types of 2 torch::Tensor.<br>
<code>template&lt;typename  </code>Data<code> = Tensor, typename </code>Target`  = Tensor&gt;</p>
<p>Is there a way to customize the types? I would like get() to return<br>
torch::data::Example&lt;torch::Tensor, std::string&gt;. If in my custom dataset, I define my get() as</p>
<pre><code class="lang-auto">torch::data::Example&lt;torch::Tensor, std::string&gt; get(size_t index) override;
</code></pre>
<p>the compiler complains</p>
<pre><code class="lang-auto">error: return type is neither identical to nor covariant with return type "torch::data::Example&lt;at::Tensor, at::Tensor&gt;" of overridden virtual function "torch::data::datasets::Dataset&lt;Self, SingleExample&gt;::get [with Self=CustomDataset, SingleExample=torch::data::Example&lt;at::Tensor, at::Tensor&gt;]"
      torch::data::Example&lt;torch::Tensor, std::string&gt; get(size_t index) override;
</code></pre>

ptrblck: <p>Maybe you could derive a custom class from <a href="https://github.com/pytorch/pytorch/blob/3f440d74fc2e8efe082d7c47e61d06edf81f45a9/torch/csrc/api/include/torch/data/example.h">here</a> and change the types?<br>
CC <a class="mention" href="/u/yf225">@yf225</a> who would know how to override the types properly <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

lampadephoria: <p>Thank you for the suggestion, <a class="mention" href="/u/ptrblck">@ptrblck</a>.<br>
Here is my CustomExample following the first example in this <a href="https://github.com/pytorch/pytorch/blob/3f440d74fc2e8efe082d7c47e61d06edf81f45a9/torch/csrc/api/include/torch/data/example.h" rel="noopener nofollow ugc">link</a> you suggested.</p>
<pre><code class="lang-auto">template &lt;typename Data = torch::Tensor, typename Path = std::string&gt;
struct CustomExample {
    using DataType = Data;
    using PathType = Path;

    CustomExample() = default;
    CustomExample(Data data, Path path)
        : data(std::move(data)), path(std::move(path)) {}

    operator Data&amp;() { return data; }
    operator const Data&amp;() const { return data; }

    operator Path&amp;() { return path; }
    operator const Path&amp;() const { return path; }

    Data data;
    Path path;
};
</code></pre>
<p>When I replace, however,</p>
<pre><code class="lang-auto">torch::data::Example&lt;&gt; get(size_t index) override;
</code></pre>
<p>with</p>
<pre><code class="lang-auto">CustomExample&lt;&gt; get(size_t index) override;
</code></pre>
<p>I got below errors:</p>
<pre><code class="lang-auto">.../dataset.hpp(76): error: return type is neither identical to nor covariant with return type "torch::data::Example&lt;at::Tensor, at::Tensor&gt;" of overridden virtual function "torch::data::datasets::Dataset&lt;Self, SingleExample&gt;::get [with Self=sas::dlx::CustomDataset, SingleExample=torch::data::Example&lt;at::Tensor, at::Tensor&gt;]"
      CustomExample&lt;&gt; get(size_t index) override;
                      ^

.../dataset.hpp(76): error: member function declared with "override" does not override a base class member
      CustomExample&lt;&gt; get(size_t index) override;
</code></pre>
<p>Where my get() is part of</p>
<pre><code class="lang-auto">class CustomDataset : public torch::data::Dataset&lt;CustomDataset&gt;{...}
</code></pre>
<p>I guess the base class torch::data::Dataset&lt;&gt; has the preset get() member signature so that any inherited class has to maintain the same get() signature. If this is true, is there a way to register a different “get()” function to the torch dataloader?</p>

AntMorais: <p>Hi <a class="mention" href="/u/lampadephoria">@lampadephoria</a>, did you manage to solve this issue? I have a similar problem, except I want to create a custom class with types Data, Target <strong>and</strong> Path. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

glaringlee: <p><a class="mention" href="/u/lampadephoria">@lampadephoria</a>, I think you should remove override since there is no get function in parent class for you to override.<br>
Take a look at this example:<br>
‘<a href="https://github.com/pytorch/examples/blob/master/cpp/custom-dataset/custom-dataset.cpp" class="inline-onebox" rel="noopener nofollow ugc">examples/custom-dataset.cpp at master · pytorch/examples · GitHub</a>’</p>
<p>Example class is there:<br>
‘<a href="https://github.com/pytorch/pytorch/blob/master/torch/csrc/api/include/torch/data/example.h" class="inline-onebox" rel="noopener nofollow ugc">pytorch/example.h at master · pytorch/pytorch · GitHub</a>’</p>

AntMorais: <p><a class="mention" href="/u/glaringlee">@glaringlee</a>, I am trying to return the path of the images in the batch (while training). In order to do that I created a <code>torch::data::PathDataset</code> class with the 2 types in the <code>torch::data::Example</code> class, plus a <code>std::string ImgPath</code>. I followed your suggestion and removed override from the get function. Still I had the same error as <a class="mention" href="/u/lampadephoria">@lampadephoria</a>, which I fixed by creating a <code>torch::data:datasets::CustomDataset</code> class.</p>
<pre><code class="lang-auto">using Data = std::vector&lt;std::pair&lt;std::string, long&gt;&gt;;


namespace torch {
namespace data {


/// A dataset consists of data, an associated target (label) and the associated path.
template &lt;typename Output = torch::Tensor, typename Target = torch::Tensor, typename ImgPath = std::string&gt;
struct PathDataset {
    using DataType = Output;
    using TargetType = Target;
    using ImgPathType = ImgPath;

    PathDataset() = default;
    PathDataset(Output data, Target target, ImgPath img_path) : data(std::move(data)), target(std::move(target)), img_path(std::move(img_path)) {}


    Output data;
    Target target;
    ImgPath img_path;
};


} // data
} // torch


namespace torch{
namespace data{
namespace datasets{

template &lt;typename Self, typename SingleExample = PathDataset&lt;&gt;&gt;
class CustomDataset : public BatchDataset&lt;Self, std::vector&lt;SingleExample&gt;&gt; {
    public:
        using ExampleType = SingleExample;

    /// Returns the example at the given index.
    virtual ExampleType get(size_t index) = 0;
    /// Returns a batch of data.
    /// The default implementation calls `get()` for every requested index
    /// in the batch.
    std::vector&lt;ExampleType&gt; get_batch(ArrayRef&lt;size_t&gt; indices) override {
    std::vector&lt;ExampleType&gt; batch;
    batch.reserve(indices.size());
    for (const auto i : indices) {
        batch.push_back(get(i));
    }
    return batch;
    }
};

}
}
}


class CustomDataset : public torch::data::datasets::CustomDataset&lt;CustomDataset&gt;
{   

    private:
        Data data;
        Options options;
    public:
        explicit CustomDataset(const Data&amp; data) : data(data) {};

        torch::data::PathDataset&lt;&gt; get(size_t index)  {
            std::string path = options.datasetPath + data[index].first;
            auto mat = cv::imread(path);
            assert(!mat.empty());

            cv::resize(mat, mat, cv::Size(options.image_size, options.image_size));
            std::vector&lt;cv::Mat&gt; channels(1);
            cv::split(mat, channels);

            // this is for 1 channel images (e.g. mnist)
            auto pixel_values_tensor = torch::from_blob(
                channels[0].ptr(),
                {options.image_size, options.image_size},
                torch::kUInt8
                );

            auto tdata = pixel_values_tensor
                .view({1, options.image_size, options.image_size})
                .to(torch::kFloat);
            
            auto tlabel = torch::from_blob(&amp;data[index].second, {1}, torch::kLong);
            return {tdata, tlabel, path};
        };

        torch::optional&lt;size_t&gt; size() const {
            return data.size();
        }
        
};

</code></pre>
<p>No errors here, but when I try to create a training set:</p>
<pre><code class="lang-auto">auto train_set = CustomDataset(data.first).map(torch::data::transforms::Stack&lt;&gt;());
</code></pre>
<p>I have the following:</p>
<pre><code class="lang-auto">error: static assertion failed: BatchType type of dataset does not match input type of transform
[build]   110 |           typename TransformType::InputBatchType&gt;::value,
[build]       |                                                    ^~~~~
</code></pre>
<p>and</p>
<pre><code class="lang-auto">error: cannot convert ‘vector&lt;torch::data::PathDataset&lt;&gt;,allocator&lt;torch::data::PathDataset&lt;&gt;&gt;&gt;’ to ‘vector&lt;torch::data::Example&lt;&gt;,allocator&lt;torch::data::Example&lt;&gt;&gt;&gt;’
[build]    75 |     return transform_.apply_batch(dataset_.get_batch(std::move(indices)));
[build]       |                                                                         ^
</code></pre>
<p>I would appreciate it if you could help me or direct me to an example in the documentation.</p>