How to read a dataset in .mat form in pytorch

ky_Pa: <p>I have two datasets in the form of .mat. I want to use the <a href="http://scipy.io" rel="noopener nofollow ugc">scipy.io</a> library and the h5py library to read and apply them to the program, but I don’t know how to operate. Please give pointers, thank you.The code that introduces the data set section is as follows.<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/6/e670105a1ac4fa9fef7133552d015a9d6312bb1e.png" data-download-href="https://discuss.pytorch.org/uploads/default/e670105a1ac4fa9fef7133552d015a9d6312bb1e" title="微信图片_20200302121440"><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/6/e670105a1ac4fa9fef7133552d015a9d6312bb1e.png" alt="微信图片_20200302121440" data-base62-sha1="wSxUFdnvvqkG73PsKX9kWPSRisS" width="690" height="57" data-dominant-color="F2F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">微信图片_20200302121440</span><span class="informations">729×61 3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<pre><code class="lang-auto"> # params for source dataset
    src_dataset = "maria"
    src_encoder_restore = os.path.join(model_root, src_dataset + "-source-encoder-final.pt")
    src_classifier_restore = os.path.join(model_root, src_dataset + "-source-classifier-final.pt")
    src_model_trained = True

    # params for target dataset
    tgt_dataset = "sandy"
    tgt_encoder_restore = os.path.join(model_root, tgt_dataset + "-target-encoder-final.pt")
    tgt_model_trained = True</code></pre>

ptrblck: <p><a href="https://discuss.pytorch.org/t/reading-images-in-mat-format-in-python/55016/2">This post</a> might be helpful, if you would like to read these <code>.mat</code> files lazily.<br>
However, it seems your dataset might be stored completely in the file.<br>
In that case you could read it outside of the <code>Dataset</code> using <code>scipy.io</code>, transform them to tensors via <code>torch.from_numpy</code> and use a <code>TensorDataset</code>.</p>

ky_Pa: <p>I can read .mat into matrix form through <a href="http://scipy.io" rel="nofollow noopener">scipy.io</a>, and then how can I load it out in my program, can you show me the specific code, thank you</p>

ptrblck: <p>Here is a small dummy example:</p>
<pre><code class="lang-python">mat = scipy.io.loadmat('test.mat')
data = mat['data'] # use the key for data here
target = mat['target'] # use the key for target here

data = torch.from_numpy(data).float()
target = torch.from_numpy(target).long() # change type to your use case

dataset = TensorDataset(data, target)
</code></pre>

ky_Pa: <p>Thank you very much, I will try it. The other .mat file is -v7.3 and needs to be read using the h5py library. Please tell me how to do it in detail. Please write it in code.</p>

ky_Pa: <p>May I change it like this? But I will get this error, how can I modify it?<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/1/b10b8c9f91449b4b0b08b2a62980856af9309a0e.png" data-download-href="https://discuss.pytorch.org/uploads/default/b10b8c9f91449b4b0b08b2a62980856af9309a0e" title="微信图片_20200302161240"><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/1/b10b8c9f91449b4b0b08b2a62980856af9309a0e.png" alt="微信图片_20200302161240" data-base62-sha1="pgdfUbqYvDFk1ZgOLNhGIj00Do2" width="690" height="99" data-dominant-color="383933"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">微信图片_20200302161240</span><span class="informations">955×138 8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<h1><a name="params-for-source-dataset-1" class="anchor" href="#params-for-source-dataset-1"></a>params for source dataset</h1>
<pre><code>src_dataset = scipy.io.loadmat('E:\\ADDA\\pytorch-adda-master-lab\\datasets\\lab\\maria\\mat\\test_target_domain_maria.mat')
testdata = src_dataset['testdata'] # use the key for data here
testlabel = src_dataset['testlabel'] # use the key for target here

testdata = torch.from_numpy(testdata).float()
testlabel = torch.from_numpy(testlabel).long() # change type to your use case

dataset = TensorDataset(testdata, testlabel)

src_encoder_restore = os.path.join(model_root, src_dataset + "-source-encoder-final.pt")
src_classifier_restore = os.path.join(model_root, src_dataset + "-source-classifier-final.pt")
src_model_trained = True```
</code></pre>

Mert_Dunver: <p>Hello sir i understood the part about the data part but what are we supposed to use for the target = mat[“target”]  part. Are we supposed to enter the labels here?</p>

ptrblck: <p>Yes,</p>
<pre><code class="lang-python">target = mat['target'] # use the key for target here
</code></pre>
<p>assumes you’ve stored the target values in the <code>.mat</code> file using the <code>'target'</code> key.<br>
If that’s not the case and e.g. you don’t have any targets, you can skip this step.</p>

Mert_Dunver: <p>Can I use the tensor to train my neural network directly or do i need to apply dataloaders.</p>

ptrblck: <p>You can use tensors directly. <code>DataLoader</code>s are able to shuffle, create batches, use multiple workers to load the next batches etc. but are not necessarily needed.</p>