Convert a code from tensorflow/keras to pytorch frame work

username: sumayya
cooked: <p>I want to change the code in this link</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/isi-vista/BusterNet">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/isi-vista/BusterNet" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/344;"><img src="https://opengraph.githubassets.com/9a80815d8759a60595b8433e6cec938202116a05d50de48307e34594841a31af/isi-vista/BusterNet" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/isi-vista/BusterNet" target="_blank" rel="noopener nofollow ugc">GitHub - isi-vista/BusterNet</a></h3>

  <p>Contribute to isi-vista/BusterNet development by creating an account on GitHub.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<p>
into pytorch framework<br>
I start working on this but I don’t know how I will convert lmdb datasets to pytorch tensor without error to start creating (new model and train the datasets) and also using pre-trained model that used in this link</p>
<p>this is my code</p>
<p>from google.colab import drive<br>
drive.mount(‘/content/drive’)</p>
<p>import torch<br>
import torchvision<br>
import numpy as np<br>
import matplotlib.pyplot as plt<br>
import torch.nn as nn<br>
import torch.nn.functional as F<br>
from torchvision.datasets import MNIST<br>
from torchvision.transforms import ToTensor<br>
from torchvision.utils import make_grid<br>
from torch.utils.data.dataloader import DataLoader<br>
from torch.utils.data import random_split<br>
%matplotlib inline</p>
<p>from <strong>future</strong> import print_function<br>
import os<br>
import cv2<br>
import json<br>
import lmdb<br>
import torch<br>
import numpy as np<br>
from matplotlib import pyplot</p>
<p>class USCISI_CMD_API( object ) :<br>
“”" Simple API for reading the USCISI CMD dataset</p>
<pre><code>This API simply loads and parses CMD samples from LMDB
# Example:
```python 
    # get the LMDB file path 
    lmdb_dir = os.path.dirname( os.path.realpath(__file__) )
    # create dataset instance
    dataset = USCISI_CMD_API( lmdb_dir=lmdb_dir, 
                              sample_file=os.path.join( lmdb_dir, 'samples.keys'),
                              differentiate_target=True )
    # retrieve the first 24 samples in the dataset
    samples = dataset( range(24) )
    # visualize these samples
    dataset.visualize_samples( samples )
    # retrieve 24 random samples in the dataset
    samples = dataset( [None]*24 )
    # visualize these samples
    dataset.visualize_samples( samples )

    # get the exact 50th sample in the dataset
    sample = dataset[50]
    # visualize these samples
    dataset.visualize_samples( [sample] )
```
# Arguments:
    lmdb_dir = file path to the dataset LMDB
    sample_file = file path ot the sample list, e.g. samples.keys
    differentiate_target = bool, whether or not generate 3-class target map

# Note:
    1. samples, i.e. the output of "get_samples" or "__call__", is a list of samples
    however, the dimension of each sample may or may not the same
    2. CMD samples are generated upon
       - MIT SUN2012 dataset [https://groups.csail.mit.edu/vision/SUN/]
       - MS COCO dataset [http://cocodataset.org/#termsofuse]
    3. detailed synthesis process can be found in paper

# Citation:
    Yue Wu et.al. "BusterNet: Detecting Image Copy-Move ForgeryWith Source/Target Localization".  
    In: European Conference on Computer Vision (ECCV). Springer. 2018.
# Contact:
    Dr. Yue Wu
    yue_wu@isi.edu
"""
def __init__( self, lmdb_dir, sample_file,train_file,valid_file,test_file, differentiate_target = True ) :
    assert os.path.isdir(lmdb_dir)
    self.lmdb_dir = lmdb_dir
    assert os.path.isfile(sample_file)
    self.sample_keys = self._load_sample_keys(sample_file)
    #add train,valid and test files and load the keys of them
    assert os.path.isfile(train_file)
    self.train_keys = self._load_train_keys(train_file)
    assert os.path.isfile(valid_file)
    self.valid_keys = self._load_valid_keys(valid_file)
    assert os.path.isfile(test_file)
    self.test_keys = self._load_test_keys(test_file)
    
    self.differentiate_target = differentiate_target
    print("INFO: successfully load USC-ISI CMD LMDB with {} keys".format( self.nb_samples ) )
    print("INFO: It have {} train keys".format( len( self.train_keys ) ) )
    print("INFO: It have {} validation keys".format( len( self.valid_keys ) ) )
    print("INFO: It have {} test keys".format( len( self.test_keys ) ) )
def __len__(self):
    return len( self.sample_keys )
@property
def nb_samples( self ) :
    return len( self.sample_keys )
def _load_sample_keys( self, sample_file ) :
    '''Load sample keys from a given sample file
    INPUT:
        sample_file = str, path to sample key file
    OUTPUT:
        keys = list of str, each element is a valid key in LMDB
    '''
    with open( sample_file, 'r' ) as IN :
        keys = [ line.strip() for line in IN.readlines() ]
    return keys
#new methods
def _load_train_keys( self, train_file ) :
    '''Load sample keys from a given sample file
    INPUT:
        sample_file = str, path to sample key file
    OUTPUT:
        keys = list of str, each element is a valid key in LMDB
    '''
    with open( train_file, 'r' ) as IN :
        train_keys = [ line.strip() for line in IN.readlines() ]
    return train_keys
def _load_valid_keys( self, valid_file ) :
    '''Load sample keys from a given sample file
    INPUT:
        sample_file = str, path to sample key file
    OUTPUT:
        keys = list of str, each element is a valid key in LMDB
    '''
    with open( valid_file, 'r' ) as IN :
        valid_keys = [ line.strip() for line in IN.readlines() ]
    return valid_keys
def _load_test_keys( self, test_file ) :
    '''Load sample keys from a given sample file
    INPUT:
        sample_file = str, path to sample key file
    OUTPUT:
        keys = list of str, each element is a valid key in LMDB
    '''
    with open( test_file, 'r' ) as IN :
        test_keys = [ line.strip() for line in IN.readlines() ]
    return test_keys
def _get_image_from_lut( self, lut ) :
    '''Decode image array from LMDB lut
    INPUT:
        lut = dict, raw decoded lut retrieved from LMDB
    OUTPUT:
        image = np.ndarray, dtype='uint8'
    '''
    image_jpeg_buffer = lut['image_jpeg_buffer']
    image = cv2.imdecode( np.array(image_jpeg_buffer).astype('uint8').reshape([-1,1]), 1 )
    image = cv2.resize(image, (128,128), interpolation = cv2.INTER_AREA)
    return image
def _get_mask_from_lut( self, lut ) :
    '''Decode copy-move mask from LMDB lut
    INPUT:
        lut = dict, raw decoded lut retrieved from LMDB
    OUTPUT:
        cmd_mask = np.ndarray, dtype='float32'
                   shape of HxWx1, if differentiate_target=False
                   shape of HxWx3, if differentiate target=True
    NOTE:
        cmd_mask is encoded in the one-hot style, if differentiate target=True.
        color channel, R, G, and B stand for TARGET, SOURCE, and BACKGROUND classes
    '''
    def reconstruct( cnts, h, w, val=1 ) :
        rst = np.zeros([h,w], dtype='uint8')
        cv2.fillPoly( rst, cnts, val )
        return rst 
    h, w = lut['image_height'], lut['image_width']
    src_cnts = [ np.array(cnts).reshape([-1,1,2]) for cnts in lut['source_contour'] ]
    src_mask = reconstruct( src_cnts, h, w, val = 1 )
    tgt_cnts = [ np.array(cnts).reshape([-1,1,2]) for cnts in lut['target_contour'] ]
    tgt_mask = reconstruct( tgt_cnts, h, w, val = 1 )
    if ( self.differentiate_target ) :
        # 3-class target
        background = np.ones([h,w]).astype('uint8') - np.maximum(src_mask, tgt_mask)
        cmd_mask = np.dstack( [tgt_mask, src_mask, background ] ).astype(np.float32)
    else :
        # 2-class target
        cmd_mask = np.maximum(src_mask, tgt_mask).astype(np.float32)
        
    cmd_mask = cv2.resize(cmd_mask, (128,128), interpolation = cv2.INTER_AREA)
    return cmd_mask
def _get_transmat_from_lut( self, lut ) :
    '''Decode transform matrix between SOURCE and TARGET
    INPUT:
        lut = dict, raw decoded lut retrieved from LMDB
    OUTPUT:
        trans_mat = np.ndarray, dtype='float32', size of 3x3
    '''
    trans_mat = lut['transform_matrix']
    return np.array(trans_mat).reshape([3,3])
def _decode_lut_str( self, lut_str ) :
    '''Decode a raw LMDB lut
    INPUT:
        lut_str = str, raw string retrieved from LMDB
    OUTPUT: 
        image = np.ndarray, dtype='uint8', cmd image
        cmd_mask = np.ndarray, dtype='float32', cmd mask
        trans_mat = np.ndarray, dtype='float32', cmd transform matrix
    '''
    # 1. get raw lut
    lut = json.loads(lut_str)
    # 2. reconstruct image
    image = self._get_image_from_lut(lut)
    # 3. reconstruct copy-move masks
    cmd_mask = self._get_mask_from_lut(lut)
    # 4. get transform matrix if necessary
    trans_mat = self._get_transmat_from_lut(lut)
    return ( image, cmd_mask, trans_mat )
def get_one_sample( self, key = None ) :
    '''Get a (random) sample from given key
    INPUT:
        key = str, a sample key or None, if None then use random key
    OUTPUT:
        sample = tuple of (image, cmd_mask, trans_mat)
    '''
    return self.get_samples([key])[0]
def get_samples( self, key_list ) :
    '''Get samples according to a given key list
    INPUT:
        key_list = list, each element is a LMDB key or idx
    OUTPUT:
        sample_list = list, each element is a tuple of (image, cmd_mask, trans_mat)
    '''
    env = lmdb.open( self.lmdb_dir )
    sample_list = []
    with env.begin( write=False ) as txn :
        for key in key_list :
            if not isinstance( key, str ) and isinstance( key, int ):
                idx = key % self.nb_samples
                key = self.sample_keys[idx]
            elif isinstance( key, str ) :
                pass
            else :
                key = np.random.choice(self.sample_keys, 1)[0]
                print("INFO: use random key", key)
            lut_str = txn.get( key.encode(encoding='utf-8', errors='strict') )
            sample = self._decode_lut_str( lut_str )
            sample_list.append( sample )
    return sample_list
def visualize_samples( self, sample_list ) :
    '''Visualize a list of samples
    '''
    for image, cmd_mask, trans_mat in sample_list :
        pyplot.figure(figsize=(10,10))
        pyplot.subplot(121)
        pyplot.imshow( image )
        pyplot.subplot(122)
        pyplot.imshow( cmd_mask )
    return
def pytorch_tensor_samples( self, sample_list ) :
    '''Visualize a list of samples
    '''
    i=0
    All_samples = {}
    for image, cmd_mask, trans_mat in sample_list :
        pytorchTensorSample = torch.tensor([image,cmd_mask])
        pytorchTensorTrans_mat=torch.tensor(trans_mat)
        #pytorchTensorTrans_mat=pytorchTensorTrans_mat.reshape(-1,3,3)
        print(pytorchTensorSample.shape)
        print(pytorchTensorTrans_mat.shape)
        All_samples[i] =[pytorchTensorSample,pytorchTensorTrans_mat]
        i = i + 1
    return All_samples
def __call__( self, key_list ) :
    return self.get_samples( key_list )
def __getitem__( self, key_idx ) :
    return self.get_one_sample( key=key_idx )
</code></pre>
<p>import sys<br>
lmdb_dir = ‘/content/drive/MyDrive/Full dataset/USCISI-CMFD’<br>
sys.path.insert(0,lmdb_dir)<br>
dataset = USCISI_CMD_API( lmdb_dir=lmdb_dir,<br>
sample_file=os.path.join( lmdb_dir, ‘samples.keys’),<br>
train_file=os.path.join( lmdb_dir, ‘train.keys’),<br>
valid_file=os.path.join( lmdb_dir, ‘valid.keys’),<br>
test_file=os.path.join( lmdb_dir, ‘test.keys’ ),<br>
differentiate_target=True )</p>
<p>def simple_cmfd_decoder( busterNetModel, rgb ) :<br>
“”“A simple BusterNet CMFD decoder<br>
“””<br>
# 1. expand an image to a single sample batch<br>
single_sample_batch = np.expand_dims( rgb, axis=0 )<br>
# 2. perform busterNet CMFD<br>
pred = busterNetModel.predict(single_sample_batch)[0]<br>
return pred</p>
<p>def visualize_result( rgb, gt=None, pred=None, figsize=(12,4), title=None ) :<br>
“”“Visualize raw input, ground truth, and BusterNet result<br>
“””<br>
pyplot.figure( figsize=figsize )<br>
pyplot.subplot(131)<br>
pyplot.imshow( rgb )<br>
pyplot.title(‘input image’)<br>
if gt is not None :<br>
pyplot.subplot(132)<br>
pyplot.title(‘ground truth’)<br>
pyplot.imshow(gt)<br>
if pred is not None :<br>
pyplot.subplot(133)<br>
pyplot.imshow(pred)<br>
pyplot.title(‘busterNet pred’)<br>
if title is not None :<br>
pyplot.suptitle( title )</p>
<p>for k in range(10) :<br>
rgb, gt, trans_mat = dataset.get_one_sample()<br>
<span class="hashtag-raw">#pred</span> = simple_cmfd_decoder( busterNetModel, rgb )<br>
visualize_result( rgb, gt)</p>
<p>sample = dataset[100]</p>
<h1><a name="visualize-these-samples-1" class="anchor" href="#visualize-these-samples-1"></a>visualize these samples</h1>
<p>dataset.visualize_samples( [sample] )</p>
<p>sample = dataset[0]<br>
sample1 = dataset[1]</p>
<h1><a name="convert-samples-to-pytorch-tensor-2" class="anchor" href="#convert-samples-to-pytorch-tensor-2"></a>convert samples to pytorch tensor</h1>
<p>pTensor= dataset.pytorch_tensor_samples([sample,sample1])<br>
print(pTensor[0][0].shape)<br>
print(pTensor[1][0].shape)</p>
<p>print(type(dataset))</p>
<p>from torch.utils.data import random_split</p>
<p>val_size = 10000<br>
test_size = 10000<br>
train_size = 80000</p>
<p>train_ds, val_ds, test_ds = random_split(dataset, [train_size, val_size, test_size])<br>
len(train_ds), len(val_ds), len(test_ds)</p>
<p>from torch.utils.data import DataLoader<br>
batch_size=64<br>
train_loader = DataLoader(train_ds, batch_size, shuffle=True)<br>
test_loader = DataLoader(test_ds, batch_size<em>2)<br>
val_loader = DataLoader(val_ds, batch_size</em>2)</p>
<p>import matplotlib.pyplot as plt<br>
from torchvision.utils import make_grid<br>
for images,cmd_mask, trans_mat in train_loader:<br>
print(‘images.shape:’, images.shape)<br>
plt.figure(figsize=(16,8))<br>
plt.axis(‘off’)<br>
plt.imshow(make_grid(images, nrow=16).permute((1, 2, 0)))<br>
break</p>
<p>I don’t know if my starting is correct or not, also I found the following error<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/7/174a5b115710f91fabf92bdf6b5dc5c5ce3ded7e.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/174a5b115710f91fabf92bdf6b5dc5c5ce3ded7e" title="Picture1"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/7/174a5b115710f91fabf92bdf6b5dc5c5ce3ded7e_2_690x415.jpeg" alt="Picture1" data-base62-sha1="3k2hS0Gx1a0w3HZfbIsFtyQ9Qho" width="690" height="415" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/7/174a5b115710f91fabf92bdf6b5dc5c5ce3ded7e_2_690x415.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/1/7/174a5b115710f91fabf92bdf6b5dc5c5ce3ded7e.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/7/174a5b115710f91fabf92bdf6b5dc5c5ce3ded7e.jpeg 2x" data-dominant-color="F6F6F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Picture1</span><span class="informations">944×568 97.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>PyTorch uses the channels-first memory layout by default, so make sure your data is loaded in the shape <code>[batch_size, channels, height, width]</code>. Based on the print statement it seems the channel dimension is the last dim.</p>

username: sumayya
cooked: <p>Thank you for your reply<br>
I try to solve this problem using permute((0, 3, 1,2)) but I still have the error<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/e/7ec5a8ad853e5ec39402cd368eaa3b297c9d24e7.png" data-download-href="https://discuss.pytorch.org/uploads/default/7ec5a8ad853e5ec39402cd368eaa3b297c9d24e7" title="Picture2"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/e/7ec5a8ad853e5ec39402cd368eaa3b297c9d24e7_2_690x320.png" alt="Picture2" data-base62-sha1="i5tDTETjZLw4PzWw5hkbGRCsRKf" width="690" height="320" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/e/7ec5a8ad853e5ec39402cd368eaa3b297c9d24e7_2_690x320.png, https://discuss.pytorch.org/uploads/default/original/3X/7/e/7ec5a8ad853e5ec39402cd368eaa3b297c9d24e7.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/e/7ec5a8ad853e5ec39402cd368eaa3b297c9d24e7.png 2x" data-dominant-color="F5F6F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Picture2</span><span class="informations">933×433 105 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>my problem is the data in the form of lmdb, so I don’t know how I can deal with this form of data and modifying how to load it.</p>

username: ptrblck
cooked: <p>I still think that <code>make_grid</code> raises the error, since <code>images</code> is in a wrong shape.<br>
Use <code>permute</code> on the <code>images</code> tensor and make sure the aforementioned shape is passed to <code>make_grid</code>.</p>

username: sumayya
cooked: <p>I print the shape after use function (permute)<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/1/01221cda11a996b901142275cd98e0774f2873d0.png" data-download-href="https://discuss.pytorch.org/uploads/default/01221cda11a996b901142275cd98e0774f2873d0" title="Picture3"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/1/01221cda11a996b901142275cd98e0774f2873d0_2_690x335.png" alt="Picture3" data-base62-sha1="a1yWthBx8s2lQtii7bAX35h66s" width="690" height="335" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/1/01221cda11a996b901142275cd98e0774f2873d0_2_690x335.png, https://discuss.pytorch.org/uploads/default/original/3X/0/1/01221cda11a996b901142275cd98e0774f2873d0.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/0/1/01221cda11a996b901142275cd98e0774f2873d0.png 2x" data-dominant-color="F5F6F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Picture3</span><span class="informations">919×447 119 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Alexey_Demyanchuk
cooked: <p>You need to reassign variable like this:<br>
<code>images = images.permute((0,3,1,2))</code></p>