foldl/torch-forum huggingface dataset:
Expanding Variable with zero-padding (for DepthConcat)

thnkim: For example, <SCODE># a: 1x2x2, b: 1x4x4
a = Variable(torch.FloatTensor([[[1,2],[3,4]]]))
b = Variable(torch.FloatTensor([[[1,1,1,1],[2,2,2,2],[3,3,3,3],[4,4,4,4]]]))
<ECODE> What I want to get from cat([a, b], 0) is <SCODE>(0 ,.,.) =
0 0 0 0
0 1 2 0
0 3 4 0
0 0 0 0

(1 ,.,.) =
1 1 1 1
2 2 2 2
3 3 3 3
4 4 4 4
<ECODE> I may try to use narrow() and copy_(), but I’m not sure copying to Variable in forward() is ok.

thnkim: 

apaszke: 

thnkim:
How can i change nn.LSTM to eval mode?

yunjey: <SCODE>rnn = nn.LSTM(10, 20, 2)
input = Variable(torch.randn(5, 3, 10))
output, hn = rnn(input)
<ECODE>

apaszke: 

yunjey: I solved this problem as you said. I should have thought a bit more. Thanks.
Is there any different between torch.sigmoid and torch.nn.functional.sigmoid?

Teaonly: 

apaszke: 

fmassa: 

Teaonly: Thanks, i see. More different is help message, there are help text for python object, no text for registered functions in C.

cakeeatingpolarbear: 

Haydnspass: <SCODE>UserWarning: nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.
  warnings.warn("nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.")
<ECODE>

drevicko:
Siamese network not converging

Nilabhra: I have implemented a Siamese network for text similarity. Here is something I observed. When I feed the two sequence batches (one batch of left sequences and another batch of right sequences in separate autograd vars) separately to the LSTM and then compute similarity on the last hidden state of the output, the model works just fine. But If I feed in both the text sequences as a single input batch and then use alternative indexing to separate out the first and second set of sequences (even indices are the left sequence and the odd indices are the right sequence) and then compute similarity, the model doesn’t converge and the outputs are random. Am I somehow losing the gradients when I am slicing the hidden state variable?

apaszke: What version are you running on?

n0obcoder:
CUDA out of memory when a Function is used multiple times

xhwang: Hi, there Sample code as below: <SCODE>
import torch
from torch.autograd import Function
from torch.autograd import Variable


class Identity(Function):

    def forward(self, input):
        return input

    def backward(self, grad_output):
        return grad_output


class Linear(Function):

    def forward(self, input, weight):
        self.save_for_backward(input, weight)
        return input.mm(weight.t())

    def backward(self, grad_output):

        input, weight = self.saved_tensors
        grad_input = grad_weight = None

        if self.needs_input_grad[0]:
            grad_input = grad_output.mm(weight)
        if self.needs_input_grad[1]:
            grad_weight = grad_output.t().mm(input)

        return grad_input, grad_weight


x = Variable(torch.rand(4000, 3000).cuda(), requires_grad=True)
w = Variable(torch.rand(3000, 3000).cuda(), requires_grad=True)

grad_output = torch.rand(4000, 3000).cuda()

lr = 0.01
for i in range(10000):

    # (1) cuda memory stays the same
    # identity = Identity()
    # loss1 = identity(x)
    # loss2 = identity(x)

    # (2) cuda memory continuously increase
    linear = Linear()
    loss1 = linear(x, w)
    loss2 = linear(x, w)

    loss = loss1 + loss2
    loss.backward(grad_output)

    x.data = x.data - lr * x.grad.data
    x.grad.data.zero_()

    if i % 100 == 0:
        print(i)
<ECODE>

apaszke: 

xhwang: Module can be reused. However, when the same Module calls forward() multiple times, it actually creates a new Function object each time, only the Parameters in it (e.g. weight in nn.Linear) are shared. Am I right? Thanks.

apaszke: Yes. Modules are safe to be reused, Functions aren’t.
PyTorch Tutorial for Deep Learning Researchers

yunjey: Hi,

Veril: Yes whoever came up with pytorch’s high level design was a genius. I think its design is objectively superior to any other python framework. In TF or Theano you invariably end up ditching the object oriented style (if you had one to begin at all), in pytorch it makes too much sense to ditch.

smth: We’re glad that you like it.

ninzine: What am I doing wrong?

Mike: Great tutorial!

QuantScientist: Adding my own recommended list:
Torch dependencies when installing from source

Tudor_Berariu: I have installed PyTorch from source using an Anaconda environment. <SCODE>$ ldd torch/lib/libTHC.so.1 
	linux-vdso.so.1 =>  (0x00007fffeebe7000)
	libcudart.so.8.0 => /usr/local/cuda-8.0/targets/x86_64-linux/lib/libcudart.so.8.0 (0x00007fb1aac3a000)
	libcublas.so.8.0 => /usr/local/cuda-8.0/targets/x86_64-linux/lib/libcublas.so.8.0 (0x00007fb1a8289000)
	libTH.so.1 => /home/tudor/pytorch/torch/lib/libTH.so.1 (0x00007fb1a7c92000)
	libcurand.so.8.0 => /usr/local/cuda-8.0/targets/x86_64-linux/lib/libcurand.so.8.0 (0x00007fb1a3d29000)
	libcusparse.so.8.0 => /usr/local/cuda-8.0/targets/x86_64-linux/lib/libcusparse.so.8.0 (0x00007fb1a121a000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fb1a0e98000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fb1a0b8f000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fb1a0978000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb1a05af000)
	/lib64/ld-linux-x86-64.so.2 (0x0000562627347000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fb1a03ab000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fb1a018d000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fb19ff85000)
	libmkl_intel_lp64.so => not found
	libmkl_intel_thread.so => not found
	libmkl_core.so => not found
	libiomp5.so => not found
	libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007fb19fd61000)
<ECODE>

apaszke:
How to do repeat operation like numpy.repeat in pytorch on Variable like tensor?

mderakhshani: Assume that I have got a tensor like below: Are there any good approaches for doing above operation?

Veril: Is this what you are looking for? You may want to copy() it after that if you actually need to change 7x7 panes independently.

mderakhshani: 

SpandanMadan: You should also look into torch.repeat. That’s also similar and useful in some cases!

himat: When should you use repeat vs expand?

Qing_En: ‘repeat’ copies the original data and allocates new memory and ‘expand’ creates a new view on the existing tensor.
torch.cuda.floatTensor type and torch.floatTensor

Shiyu: i have encountered the following error: (float value)
didn’t match because some of the arguments have invalid types: (torch.cuda.FloatTensor) (torch.FloatTensor other)
didn’t match because some of the arguments have invalid types: (torch.cuda.FloatTensor) (float value, torch.FloatTensor other)

smth: Could you try: <SCODE>gradient = torch.ge(inputs.grad.data, 0.0)
<ECODE>

Shiyu: Problem solved, thanks. Here is another problem, how do convert a torch.cuda.floatTensor type tensor to a torch.floatTensor. Thanks.

fmassa: 

Shiyu: Thanks a lot!!!
datas.ImageFolder could be as torch.utils.data.TensorDataset?

vibe: <SCODE> Thank you for opening my new conversation.I have little chances to speak English ,so my English is bad and maybe the way that I express is not appropriate.I am sorry for this.
 I put some pictures in folder and I use the ImageFolder to get my datasat.What i would like to know is if the dataset could bu used in net without changing to   torch.utils.data.TensorDataset ?<ECODE>

fmassa:
Torch.add() error

Shiyu: When I am using torch.add(), I have encountered the following error

apaszke: 

YankuHorn: So - I will be using torch.mul, but the question is in place, and someone has to either implement the correct function or fix the documentation

smth: <SCODE>In [1]: import torch

In [2]: a=torch.randn(10)

In [3]: torch.add(a, 0.1, a)
Out[3]:

 0.9079
 1.1069
-1.5801
-0.3657
-0.6019
-0.5571
 0.5797
 0.2054
-0.9112
-0.7749
[torch.FloatTensor of size 10]

<ECODE>

YankuHorn: Now it is working, but doesn’t do anything: The code I wrote in a file: <SCODE>import torch

q =torch.randn(10)
print "1 ",q
torch.add(q, 0.1, q)
print "2 ",q
exit()
<ECODE>

smth: 

YankuHorn: OK, :embarrassed:, i was in a hurry - looks like it’s working now. Thanks.
Function for Trace

DasPantom: Hello, I am trying to write a Function for torch.trace (seems not to exist so far). Here is my code: <SCODE>class CustomTrace(torch.autograd.Function):
    
    def forward(self, mat):
        self.save_for_backward(mat)
        return torch.Tensor([torch.trace(mat)])
    
    def backward(self, g):
        mat, = self.saved_tensors
        
        return torch.mul(torch.eye(int(mat.size()[0])),g[0])
<ECODE> When I try to check my result, its precision is not optimal, i.e. <SCODE>input = (torch.autograd.Variable(torch.randn(2,2).double(), requires_grad = True),)
torch.autograd.gradcheck(CustomTrace(), input, eps = 1e-4, atol = 1e-3)
<ECODE> returns true. However, <SCODE>input = (torch.autograd.Variable(torch.randn(2,2).double(), requires_grad = True),)
torch.autograd.gradcheck(CustomTrace(), input, eps = 1e-6, atol = 1e-4)
<ECODE> returns false. I would hope that my math is right. Is there any problem with my code or is this for some reason to be expected? Thanks a lot!

smth: You dont need to save the entire mat for backwards, only mat.size()
 
mat.size()[0] -> mat.size(0)
 you dont handle trace for non-square inputs. the output return type is not the same as the input return type, because you use torch.Tensor. Instead use mat.new()
 Overall, here’s a modified version that handles non-square inputs, and implements the other suggestions. <SCODE>class CustomTrace(torch.autograd.Function):
    
    def forward(self, input):
        self.isize = input.size()
        return input.new([torch.trace(input)])
    
    def backward(self, grad_output):
        isize = self.isize
        grad_input = grad_output.new(isize).copy_(torch.eye(*isize))
        grad_input.mul_(grad_output[0])
        return grad_input
<ECODE>

DasPantom: Thanks a lot! Amazing to see how much these few lines of code can be improved. Thanks for pointing out that trace also works for non-square matrices (the mathematician in me assumed that trace only works for square matrices). I will try to implement this. Edit: I see. It already works for non-square matrices. Thanks a lot.

tachim: correlation_product = lambda f, d: torch.mul(d, torch.dot(f, d).expand_as(d)) <SCODE>print(gradcheck(correlation_product, (f, d), eps=1e-6, atol=1e-3))
print(gradcheck(correlation_product, (f, d), eps=1e-6, atol=1e-2))
print(gradcheck(correlation_product, (f, d), eps=1e-6, atol=1e-1))
<ECODE>

smth: you should ideally do gradchecks in double precision. Float precision might not be enough for finite difference to agree with analytical gradient.

tachim: That was exactly the problem – thanks!

Florian_1990: Just mentioning for everyone that gets here using a search engine.
What’s the channel order of images for resnet model in model_zoo?
Category: vision

Yufeng_Ma: BGR or RGB? And also what is the scale of input images? I’ve seen tutorials with both [0,255] and [0,1] scales.

Cysu: 

jhjungCode: here is a tutorial <SCODE>normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])

batch_size = 256 
train_loader = torch.utils.data.DataLoader(
    datasets.ImageFolder(traindir,
                         transforms.Compose([
                             transforms.RandomSizedCrop(224),
                             transforms.RandomHorizontalFlip(),
                             transforms.ToTensor(),
                             normalize,])),
    batch_size=batch_size,
    shuffle=True,
    num_workers=4)
<ECODE>
Is it possible to use third party torch7 packages in pytorch?

kirk86: 

apaszke: No, we don’t support loading Lua modules in Python

kirk86: 

apaszke: No there’s not, but I’m pretty sure you can find something in a (much richer) Python ecosystem. You can easily convert between tensors and numpy arrays.

kirk86: 

apaszke: I’m afraid that won’t work. PyTorch uses a slightly different version of C backends (0 vs 1-based indexing), and when you load Lua packages with lutorpy, it will resolve the Lua backend symbols to 0-based ones, likely leading to errors in weird places.

kirk86: 

apaszke: No, PyTorch packages have been redesigned and rewritten from scratch.

kirk86: 

jekbradbury: I believe the functions you’re looking for are implemented in scipy and/or scikit-learn, and you can use them by calling .numpy and .from_numpy on Torch tensors.

kirk86:
Loading weights for CPU model while trained on GPU

jtremblay: This is not a very complicated issue, but I am not sure what is the best way to load the weights into the cpu when the model was trained on a GPU, thus here is my solution: <SCODE>model = torch.load('mymodel')
self.model = model.cpu().double()
<ECODE>

fmassa: <SCODE>torch.load('my_file.pt', map_location=lambda storage, loc: storage)
<ECODE>

achaiah: 

fmassa: 

achaiah: Perfect, thanks for the clarification.  I ended up using the conversion suggested in your linked post.

jadore801120: Thanks for the suggestion!

tralgu: torch.load('my_file.pt', map_location=lambda storage, loc: storage)

Peter_Ham: I have a related question, I have a shared model trained on GPU, and another process needs this model for inference on CPU. So I use a shared model and use the following command to load this shared model <SCODE>    cpu_model.load_state_dict(gpu_model.cpu().state_dict())
<ECODE> however, this won’t work and returns CUDA error(3), initialization error, what happened?

Marat: If I load model with your hack and set model.train(‘True’) and trying even inference it fails. So it do not work for all cases (finetune on CPU after training on GPU not working).

TSLsun: <SCODE>use_cuda = torch.cuda.is_available()
DEVICE = torch.device('cuda' if use_cuda else 'cpu')   # 'cpu' in this case

cpu_model = your_model()
cpu_model.load_state_dict(torch.load(path_to_your_saved_gpu_model, map_location=DEVICE)
<ECODE>

A_Rza_SH: thanks, worked for me

prashanth:
DQN tutorial with no cheat

alexis-jacq: <SCODE>resize = T.Compose([T.ToPILImage(), T.Scale(40, interpolation=Image.CUBIC), T.ToTensor()])

# This is based on the code from gym.
screen_width = 600
def get_cart_location():
    world_width = env.x_threshold * 2
    scale = screen_width / world_width
    return int(env.state[0] * scale + screen_width / 2.0) # MIDDLE OF CART
    
    
def get_screen():
    screen = env.render(mode='rgb_array').transpose((2, 0, 1)) # transpose into torch order (CHW)
    # Strip off the top and bottom of the screen
    '''
    # this is the trick :
    screen = screen[:, 160:320]
    view_width = 320
    cart_location = get_cart_location()
    if cart_location < view_width // 2:
        slice_range = slice(view_width)
    elif cart_location > (screen_width - view_width // 2):
        slice_range = slice(-view_width,None)
    else:
        slice_range = slice(cart_location - view_width // 2, cart_location + view_width // 2)
    # Strip off the edges, so that we have a square image centered on a cart
    screen = screen[:, :, slice_range]
    '''
    # Convert to float, rescare, convert to torch tensor (this doesn't require a copy)
    screen = np.ascontiguousarray(screen, dtype=np.float32) / 255
    screen = torch.from_numpy(screen)
    # Resize, and add a batch dimension (BCHW)
    return resize(screen).unsqueeze(0)

env.reset()
plt.imshow(get_screen().squeeze(0).permute(1, 2, 0).numpy(), interpolation='none')
plt.show()
<ECODE> … I can’t obtain any learning. I tried several different combinations of parameters, and I also try to change the structure of the network, but no way to reach any acceptable result. I there a way to make it work ?

apaszke: 

alexis-jacq: Haha, I take your response as a challenge! I will find a way.

arc_HITMAN: Hey, Did you find a way to make the agent learn without the trick?
How to change the default device of GPU? device_ids[0]

lynnea718: <SCODE>if output_device is None:
output_device =device_ids[0]
<ECODE> to <SCODE>if output_device is None:
output_device =device_ids[1]
<ECODE>

fmassa: <SCODE>CUDA_VISIBLE_DEVICES=1,2 python myscript.py
<ECODE> so your script will only see GPUs number 1 and 2, and won’t touch the other GPUs

lynnea718: We have 8 gpu. I used the devices[1,2] ,the rest is used by others

colesbury: 

cold_wind: So in the Python script, How to choose the GPU devices. Especially, I want to change GPU devices in the training processing

uzeful: I also want to know how to choose the GPU device in the python script.

Jing: ex: <SCODE>with torch.cuda.device(1):
    w = torch.FloatTensor(2,3).cuda()
    # w was placed in  device_1 by default.
<ECODE> <SCODE>w = torch.FloatTensor(2,3).cuda(2)
# w was placed in device_2

<ECODE>

uzeful: 

jdhao: <SCODE>CUDA_VISIBLE_DEVICES=3; python test.py
<ECODE> <SCODE>import torch
print(torch.cuda.current_device())
<ECODE> the above script still shows that current device is 0.

SimonW: two things you did wrong:

jdhao: 

SimonW: That controls what devices CUDA exposes and PyTorch can’t do nothing in this regards.

hszhao: Hi, you can specify used gpu in python script as following:

jianzhong: it cannot work for me, it always use the first(ie, 0) gpu

kaiyuyue: Thanks, this is the easiest way to solve this problem.

SimonW: It shouldn’t happen. That is a CUDA flag. Once set, PyTorch will never have access to the excluded device(s).

moonlightlane: 

maplewizard: This is a very useful solution, especially you are going to run with someone else’s code without specifying the cuda id.

Sun_ShiJie: <SCODE>torch.cuda.set_device(device)

Sets the current device.

Usage of this function is discouraged in favor of device. In most cases it’s better to use CUDA_VISIBLE_DEVICES environmental variable.

Parameters:	device (torch.device or int) – selected device. This function is a no-op if this argument is negative.
<ECODE>
When cuda() is called, a list of modules cannot shift to the GPU

xinwei_he: Hi, below is my definition of network: <SCODE>class  CNN_Text(nn.Module):
    
    def __init__(self, args):
        super(CNN_Text,self).__init__()
        self.args = args
        
        V = args.embed_num
        D = args.embed_dim
        C = args.class_num
        Ci = 1
        Co = args.kernel_num
        Ks = args.kernel_sizes

        self.embed = nn.Embedding(V, D)
        self.convs1 = [nn.Conv2d(Ci, Co, (K, D)) for K in Ks]

        self.dropout = nn.Dropout(args.dropout)
        self.fc1 = nn.Linear(len(Ks)*Co, C)
<ECODE> However, when i called cuda() on the model, the modules in the  list self.convs1 will not shift to the GPU, how to solve this problem, any ideas ?

jekbradbury: 

xinwei_he: thanksm but it gives me the error <SCODE>AttributeError: 'module' object has no attribute 'ModuleList'
<ECODE> is it a problem related to the version of pytorch?

xinwei_he: Many Thanks, it solves my problem. And Updating my pytorch makes ModuleList available.
State_dict does not contain keys for conv layers in list

isht7: <SCODE>import torch.nn as nn
class Classifier_Module(nn.Module):

    def __init__(self,dilation_series,padding_series):
        super(Classifier_Module, self).__init__()
	    self.conv2d_list = []
	    for dilation,padding in zip(dilation_series,padding_series):
	        self.conv2d_list.append(nn.Conv2d(2048,5,kernel_size=3,stride=1, padding =padding, dilation = dilation,bias = True))

   
    def forward(self, x):
	    out = self.conv2d_list[0](x)
	    for i in range(len(self.conv2d_list)-1):
	        out = self.conv2d_list[i+1](x)+out
        return out


class Module1(nn.Module):
    def __init__(self):
        super(Module1, self).__init__()
        self.layer = self._make_pred_layer(Classifier_Module, [6,12,18,24],[6,12,18,24])

    def _make_pred_layer(self,block, dilation_series, padding_series):
    	return nn.Sequential(block(dilation_series,padding_series))

    def forward(self, x):
        x = self.layer(x)
        return x
<ECODE> The state dictionary of the net does not contain any keys corresponding to the conv2d layers of the Classifier_Module. <SCODE>import sample
model = getattr(sample,'Module1')()
print model   # 1 does not show conv2d list
for keys in model.state_dict().keys():
    print keys    #2 does not show con2d list
print model.layer._modules['0'].conv2d_list  # this shows the conv2d list

<ECODE> How can I fix this issue? Is there any other way to perform a similar function without writing code for each conv2d layer?

jekbradbury: 

isht7: 

apaszke: Yes, you can return multiple outputs form a module
How to get the newest pytorch?

Ke_Bai: I uninstall pytorch from anaconda and tried to compile the source code from github because I want to use the latest part “nn.init”. But things go strange. My python part cannot find torch packages, which means, when we I try to type in Python 3.6.0 | Anaconda 4.3.0 can not find torch module. I have no choice but to install it from Anaconda again and I still cannot use nn.init part. What should I do? p.s. Is there any else way to setup the weight initialization? Thanks.

jekbradbury: 

Ke_Bai: Hope to be useful to others.
‘Criterion’ object has no attribute ‘cuda’!

mderakhshani: Hello there, I wrote  below customized Cost Function for my project: My codes to benefit from these written cost are: Thanks!

fmassa: calling torch.FloatTensor will always put your tensor in the CPU. You probably want to get the device in which the input tensor is by doing something like pred_out.new(), which will generate a new tensor in the same device as pred_out. you are using np.ndarray in parts of your code. While it’s possible to do it, it will avoid parts of your code to run in the GPU Performing for loops for the computations that you want will probably be slower in the GPU than in the CPU. You’d probably want to wrap some inner loops using only batched mathematical operations in tensors so that it can be efficiently computed in the GPU, so that instead of accepting one bounding box, you accept a bunch of them at the same time.

mderakhshani: 

fmassa: <SCODE>output = output.cpu()
loss = criterion(output, target)
<ECODE> and that should work out of the box.
Word Language Model training question

emanjavacas: Given the sequence of characters: “a” “b” “c” “d” … “z” and bptt equal to 3 and ignoring batching for simplicity: first sequence: src=“a”,“b”,“c”; trg=“d” second sequence: src=“d”, “e”, “f”; trg=“g” Perhaps, I am wrong but doesn’t it mean that an amount of data proportional to the value of bptt isn’t being used during training (in the example above sequences src=“b” “c” “d”, trg=“e” and src=“c” “d” “e”, trg=“f” aren’t in the training set)?

smth: b,c,d target=“e” is covered by carrying the hidden state forward between sequences.

emanjavacas: 

emanjavacas: In other words, the parameter bptt seems to be tweaking two things: how many steps back to include in the rnn computational graph. how many examples to take into account for estimate perplexity on. but perhaps I am understanding things in a wrong way…
Clarification - Using backward() on non-scalars

Mattie: Hey, Would anyone be able to point me in the direction of an example if one exists? If not, say I had the following super simple example: x = Variable(torch.FloatTensor([[2,1]]), requires_grad=True) M = Variable(torch.FloatTensor([[1,2],[3,4]])) y = torch.mm(x,M)

colesbury: <SCODE>x = Variable(torch.FloatTensor([[2,1]]), requires_grad=True)
M = Variable(torch.FloatTensor([[1,2],[3,4]]))
y = torch.mm(x, M)
jacobian = torch.FloatTensor(2, 2).zero_()
y.backward(torch.FloatTensor([[1, 0]]), retain_variables=True)
jacobian[:,0] = x.grad.data
x.grad.data.zero_()
y.backward(torch.FloatTensor([[0, 1]]), retain_variables=True)
jacobian[:,1] = x.grad.data
<ECODE> torch.autograd.backward([y], [torch.FloatTensor([[1, 0]])], retain_variables=True)

Mattie: Fab, that’s a great help. Thanks for your time.

linlin: I am a bit confused by this. In this example, isn’t that the “loss” is y? Is that correct? Thanks a lot!

jsm: is used instead of Thank you in advance!

jdhao: 

jsm: 

jdhao: 

linlin: 

yottabytt: 

jdhao: 

saan77: 

Jolyon: 

Huimin_ZENG: