댑덥딥 2주차 정리
Tags: ai, deeplearning, 스터디, 모두를위한딥러닝시즌2
Published: 2023-12-28 09:13:09

> '모두를 위한 딥러닝 시즌 2' 강의를 듣고 공부하는 스터디 입니다. https://deeplearningzerotoall.github.io/season2/lec_tensorflow.html
> 

---

***비대면 13 April, 2023***
# 04-1 multivariate linear regression

### simple linear regression 복습

-하나의 정보→하나의 결과 예측

H(x) = Wx+b


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/alwi6cbh9kx1e47g5y5a.png)


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gagvjhi2uixvfh8gn7ip.png)

### multivariate linear regression

-여러 개의 정보→하나의 결과 예측

H(x) = w1x1+w2x2+w3x3+b

: x가 길이 1000의 vector라면? 식이 하염없이 길어짐→matmul() 사용

cost 함수, 학습 방식 simple linear regression과 동일

```python
#simple과 다른 부분

#데이터 정의
x_train = torch.FloatTensor([[73, 80, 75],[93, 88, 93], [89, 91, 90], [96, 98, 100], [73, 66, 70]])
y_train = torch.FloatTensor([[152],[185],[180],[196],[142]])
#W 정의
W = torch.zeros((3,1), requires_grad=True)
```

### torch.nn module

-모델 생성을 도와줌

nn.Linear(입력 개수, 출력 개수)

hypothesis 계산은 forward()에서

gradient 계산은 backward()에서_pytorch가 알아서 해줌


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1gewlysosgjskwznvo4u.png)

### F.mse_loss

torch.nn.functional에서 다양한 loss function을 제공

- Full code


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jqkd6cfebol7qt20o537.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qg4ru1cxxyj8usg7staa.png)

# 04-2 Loading Data

-엄청난 양의 데이터를 편하게 다루는 방법

데이터가 많으면? 속도가 느리거나 하드웨어적으로 불가능→일부분의 데이터만 학습

### “Minibatch” Gradient Descent

-전체 데이터를 균일하게 나눠 학습하는 방법→업데이트 주기 빨라짐/잘못된 예측을 할 수 있음

데이터 일부만 활용

### PyTorch Dataset module

-원하는 데이터셋 지정

__len__(): 데이터셋의 총 데이터수

__getitem__(): 특정 인덱스에 상응하는 입출력 데이터 torch.tensor 형태로 반환

### DataLoader

-데이터셋 불러오기

batch_size: 통상적으로 2의 제곱수로 설정

suffle 옵션: 데이터가 학습되는 순서를 바꿈

```python
data_loader = torch.utils.DataLoader(
	DataLoader=mnist_train, 
	batch_size=batch_size, 
	shuffle=True, 
	drop_last=True
)
```

- Full code


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lgtg47s3tcy6dbai9hvv.png)

# 05 Logistic Regression

-결과 값이 0 또는 1

-binary classification에서 사용

→ BCE(sigmoid 함수 활용)

### computing hypothesis


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cu01e7nconmt5nz6hw8w.png)

=1/(1+e^(-(XW+b)))

H(x)~=P(x=1;w) = 1-P(x=0;w)

```python
hypothesis = 1/(1+torch.exp(-(x_train.matmul(w)+b)))
#혹은
hypothesis = torch.sigmoid(x_train.matmul(w)+b)
```

### computing cost function


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mdb00x1bi958xhkrrdnj.png)


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qztcc0fuwrv3tsor6ybo.png)

```python
	#bce
F.binary_cross_entropy(hypothesis, y_train)
```

참고) 로지스틱 회귀에서의 cost function 도출 _https://copycode.tistory.com/162


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/elqwz6mkj3kc44ifszq2.png)


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ccnl8ymusiwm93j86v97.png)

⇒두 경우를 하나의 식으로 표현하면 위의 cost(W) 식이 됨

- Full Code


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2jkadqostmh27fsxdf22.png)

### evaluation

```python
prediction = torch.ByteTensor
prediction = hypothesis >= torch.FloatTensor([0.5])      #0.5보다 크면 1, 작으면 0
correct_prediction = prediction.float() == y_train
```

### higher implementation

- Full Code


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qcap9fmkchc5ksco3geg.png)

# 06 Softmax Classification

-Logistic Regression의 연장선

-multi-classifcation 문제에서 사용

-이산확률분포 함수를 근사함

-multinolmial classfication → CE(softmax)

### Softmax

-이전의 사건을 바탕으로 다음 사건이 일어날 확률을 계산


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/phlvi61qyny7oketjc4q.png)

-max값을 뽑는데 soft하게(부드럽게) 뽑음

ex) [1,2,3]에 대해

: 일반적인 max-(0,0,1) / softmax-(0.0900, 0.2447, 0.6652)_합은 1, P(3|1) = 0.6652

-여기에서 ei는 sigmoid 함수로 구한 각각의 값(합이 1이 아님)

#### Cross Entropy

-두 개의 확률 분포의 유사도를 나타내는 수치


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2m6mc4i43bhh487ynrgw.png)

cross entropy를 최소화하면 Q를 P에 근사할 수 있음

### Cross Entropy Loss

#### Low-level Implementation and High-level Implementation


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/va92mruffekn6sx7hyty.png)

Q(x) = prediction

```python
z = torch.rand(3, 5, requires_grad=True)
hypothesis = F.softmax(z, dim=1)

y_one_hot = torch.zeros_like(hypothesis)
y_one_hot.scatter_(1, y.unsqueeze(1), 1)

#(low level)
torch.log(F.softmax(z, dim=1))
cost = (y_one_hot * -torch.log(hypothesis)).sum(dim=1).mean()

#혹은(high level)
F.log_softmax(z, dim=1)
cost = F.nll_loss(F.log_softmax(z, dim=1), y)      //nll:Negative Log Likelihood

#혹은
cost = F.cross_entropy(z, y)      //F.log_softmax() + F.nll_loss()
```

- Full Code

//low level


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0fyvuw7ulwgeor0kg01j.png)

//F.cross_entropy


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nj63e6hf2mytc9h69tmm.png)

---
***대면 15 April, 2023***

###multivariate linear regression_logistic regression_softmax classification
{% embed https://colab.research.google.com/drive/1r2s2X67-WYlj-KLmpgCBKs_tG2HOS8_q?usp=sharing %}