Mini Batch
전체 데이터를 더 작은 단위로 나누어서 해당 단위로 학습하는 개념
미니 배치 학습 시, 미니 배치만큼 가져가서 그에 대한 비용(cost)를 계산하고 경사하강법 수행한다.
그리고 그 다음 미니 배치를 가져가서 경사 하강법을 사용하고 마지막까지 이를 반복.
이렇게 전체 데이터에 대한 학습 1회 끝나면 -> 1 Epoch DONE
*Epoch(에포크) : 전체 train data가 학습에 한번 사용된 주기를 말함
*미니 배치 경사 하강법은 전체 데이터의 일부만을 보고 경사 하강법을 수행하므로 최적값으로 수렴하는 과정에서 값이 조금 헤매기도 하지만 training 속도가 빠름
*Iteration : 한 번의 에포크 내에서 이루어지는 매개변수인 W와 b의 업데이트 횟수.
Data Load
from torch.utils.data import TensorDataset # 텐서데이터셋
from torch.utils.data import DataLoader # 데이터로더
x_train = torch.FloatTensor([[73, 80, 75],
[93, 88, 93],
[89, 91, 90],
[96, 98, 100],
[73, 66, 70]])
y_train = torch.FloatTensor([[152], [185], [180], [196], [142]])
dataset = TensorDataset(x_train, y_train)
파이토치 데이터셋을 만들고 데이터로더 사용.
dataloader = DataLoader(dataset, batch_size = 2, shuffle=True)
사용할 데이터 셋, 미니 배치의 크기 지정
*보통 미니 배치의 크기는 통상적으로 2의 배수로 한다.
*shuffle = True : Epoch마다 데이터셋을 섞어서 데이터가 학습되는 순서 바꿈.
model = nn.Linear(3,1)
optimizer = torch.optim.SGD(model.parameters(), lr=1e-5)
모델과 옵티마이저를 설계한다.
nb_epochs = 50
for epoch in range(nb_epochs + 1):
for batch_idx, samples in enumerate(dataloader):
# print(batch_idx)
# print(samples)
x_train, y_train = samples
# H(x) 계산
prediction = model(x_train)
# cost 계산
cost = F.mse_loss(prediction, y_train)
# cost로 H(x) 계산
optimizer.zero_grad()
cost.backward()
optimizer.step()
print('Epoch {:4d}/{} Batch {}/{} Cost: {:.6f}'.format(
epoch, nb_epochs, batch_idx+1, len(dataloader),
cost.item()
))
*output
더보기
Epoch 0/50 Batch 1/3 Cost: 1.242304
Epoch 0/50 Batch 2/3 Cost: 0.563961
Epoch 0/50 Batch 3/3 Cost: 0.448591
Epoch 1/50 Batch 1/3 Cost: 0.692483
Epoch 1/50 Batch 2/3 Cost: 1.106694
Epoch 1/50 Batch 3/3 Cost: 0.213365
Epoch 2/50 Batch 1/3 Cost: 0.064273
Epoch 2/50 Batch 2/3 Cost: 1.796256
Epoch 2/50 Batch 3/3 Cost: 1.607992
Epoch 3/50 Batch 1/3 Cost: 0.984423
Epoch 3/50 Batch 2/3 Cost: 1.063094
Epoch 3/50 Batch 3/3 Cost: 0.182904
Epoch 4/50 Batch 1/3 Cost: 1.582306
Epoch 4/50 Batch 2/3 Cost: 1.148095
Epoch 4/50 Batch 3/3 Cost: 0.063173
Epoch 5/50 Batch 1/3 Cost: 0.919290
Epoch 5/50 Batch 2/3 Cost: 1.062542
Epoch 5/50 Batch 3/3 Cost: 0.135253
Epoch 6/50 Batch 1/3 Cost: 1.065469
Epoch 6/50 Batch 2/3 Cost: 0.789830
Epoch 6/50 Batch 3/3 Cost: 0.269580
Epoch 7/50 Batch 1/3 Cost: 1.052595
Epoch 7/50 Batch 2/3 Cost: 0.737678
Epoch 7/50 Batch 3/3 Cost: 0.290358
Epoch 8/50 Batch 1/3 Cost: 1.379399
Epoch 8/50 Batch 2/3 Cost: 0.645907
Epoch 8/50 Batch 3/3 Cost: 0.990046
Epoch 9/50 Batch 1/3 Cost: 1.161351
Epoch 9/50 Batch 2/3 Cost: 0.983895
Epoch 9/50 Batch 3/3 Cost: 0.142623
Epoch 10/50 Batch 1/3 Cost: 0.858378
Epoch 10/50 Batch 2/3 Cost: 1.062926
Epoch 10/50 Batch 3/3 Cost: 0.148715
Epoch 11/50 Batch 1/3 Cost: 0.777309
Epoch 11/50 Batch 2/3 Cost: 0.682220
Epoch 11/50 Batch 3/3 Cost: 1.290777
Epoch 12/50 Batch 1/3 Cost: 0.954402
Epoch 12/50 Batch 2/3 Cost: 1.085446
Epoch 12/50 Batch 3/3 Cost: 0.110559
Epoch 13/50 Batch 1/3 Cost: 0.041273
Epoch 13/50 Batch 2/3 Cost: 1.833644
Epoch 13/50 Batch 3/3 Cost: 1.557202
Epoch 14/50 Batch 1/3 Cost: 1.191278
Epoch 14/50 Batch 2/3 Cost: 0.751343
Epoch 14/50 Batch 3/3 Cost: 0.163530
Epoch 15/50 Batch 1/3 Cost: 0.819364
Epoch 15/50 Batch 2/3 Cost: 1.062180
Epoch 15/50 Batch 3/3 Cost: 0.199147
Epoch 16/50 Batch 1/3 Cost: 0.749296
Epoch 16/50 Batch 2/3 Cost: 1.041121
Epoch 16/50 Batch 3/3 Cost: 0.212914
Epoch 17/50 Batch 1/3 Cost: 0.803099
Epoch 17/50 Batch 2/3 Cost: 0.664343
Epoch 17/50 Batch 3/3 Cost: 1.702409
Epoch 18/50 Batch 1/3 Cost: 0.602896
Epoch 18/50 Batch 2/3 Cost: 0.553041
Epoch 18/50 Batch 3/3 Cost: 1.596134
Epoch 19/50 Batch 1/3 Cost: 0.968940
Epoch 19/50 Batch 2/3 Cost: 0.662227
Epoch 19/50 Batch 3/3 Cost: 1.189889
Epoch 20/50 Batch 1/3 Cost: 0.161107
Epoch 20/50 Batch 2/3 Cost: 2.475749
Epoch 20/50 Batch 3/3 Cost: 0.269707
Epoch 21/50 Batch 1/3 Cost: 0.789196
Epoch 21/50 Batch 2/3 Cost: 1.059804
Epoch 21/50 Batch 3/3 Cost: 0.163728
Epoch 22/50 Batch 1/3 Cost: 1.043981
Epoch 22/50 Batch 2/3 Cost: 0.069241
Epoch 22/50 Batch 3/3 Cost: 1.800337
Epoch 23/50 Batch 1/3 Cost: 1.209918
Epoch 23/50 Batch 2/3 Cost: 0.714378
Epoch 23/50 Batch 3/3 Cost: 0.956597
Epoch 24/50 Batch 1/3 Cost: 1.406833
Epoch 24/50 Batch 2/3 Cost: 0.667870
Epoch 24/50 Batch 3/3 Cost: 0.916201
Epoch 25/50 Batch 1/3 Cost: 0.593989
Epoch 25/50 Batch 2/3 Cost: 1.192008
Epoch 25/50 Batch 3/3 Cost: 0.439744
Epoch 26/50 Batch 1/3 Cost: 0.687513
Epoch 26/50 Batch 2/3 Cost: 1.029234
Epoch 26/50 Batch 3/3 Cost: 0.221504
Epoch 27/50 Batch 1/3 Cost: 1.031025
Epoch 27/50 Batch 2/3 Cost: 0.083856
Epoch 27/50 Batch 3/3 Cost: 1.742661
Epoch 28/50 Batch 1/3 Cost: 0.618320
Epoch 28/50 Batch 2/3 Cost: 1.077366
Epoch 28/50 Batch 3/3 Cost: 1.431801
Epoch 29/50 Batch 1/3 Cost: 0.565040
Epoch 29/50 Batch 2/3 Cost: 1.195735
Epoch 29/50 Batch 3/3 Cost: 0.345257
Epoch 30/50 Batch 1/3 Cost: 0.494264
Epoch 30/50 Batch 2/3 Cost: 0.889818
Epoch 30/50 Batch 3/3 Cost: 1.347875
Epoch 31/50 Batch 1/3 Cost: 0.409979
Epoch 31/50 Batch 2/3 Cost: 2.180332
Epoch 31/50 Batch 3/3 Cost: 0.189834
Epoch 32/50 Batch 1/3 Cost: 0.060971
Epoch 32/50 Batch 2/3 Cost: 1.050519
Epoch 32/50 Batch 3/3 Cost: 1.602038
Epoch 33/50 Batch 1/3 Cost: 1.262114
Epoch 33/50 Batch 2/3 Cost: 0.688861
Epoch 33/50 Batch 3/3 Cost: 0.927570
Epoch 34/50 Batch 1/3 Cost: 1.213289
Epoch 34/50 Batch 2/3 Cost: 0.571694
Epoch 34/50 Batch 3/3 Cost: 0.499225
Epoch 35/50 Batch 1/3 Cost: 0.687431
Epoch 35/50 Batch 2/3 Cost: 0.594548
Epoch 35/50 Batch 3/3 Cost: 1.329620
Epoch 36/50 Batch 1/3 Cost: 1.934188
Epoch 36/50 Batch 2/3 Cost: 0.921663
Epoch 36/50 Batch 3/3 Cost: 0.052485
Epoch 37/50 Batch 1/3 Cost: 1.058650
Epoch 37/50 Batch 2/3 Cost: 0.856656
Epoch 37/50 Batch 3/3 Cost: 0.181225
Epoch 38/50 Batch 1/3 Cost: 0.753086
Epoch 38/50 Batch 2/3 Cost: 0.637194
Epoch 38/50 Batch 3/3 Cost: 1.249881
Epoch 39/50 Batch 1/3 Cost: 0.921967
Epoch 39/50 Batch 2/3 Cost: 0.744168
Epoch 39/50 Batch 3/3 Cost: 1.139925
Epoch 40/50 Batch 1/3 Cost: 1.091079
Epoch 40/50 Batch 2/3 Cost: 1.016668
Epoch 40/50 Batch 3/3 Cost: 0.136321
Epoch 41/50 Batch 1/3 Cost: 0.749371
Epoch 41/50 Batch 2/3 Cost: 1.021392
Epoch 41/50 Batch 3/3 Cost: 0.183715
Epoch 42/50 Batch 1/3 Cost: 0.759830
Epoch 42/50 Batch 2/3 Cost: 1.036982
Epoch 42/50 Batch 3/3 Cost: 0.183283
Epoch 43/50 Batch 1/3 Cost: 0.361928
Epoch 43/50 Batch 2/3 Cost: 0.943031
Epoch 43/50 Batch 3/3 Cost: 1.415997
Epoch 44/50 Batch 1/3 Cost: 0.562195
Epoch 44/50 Batch 2/3 Cost: 0.545631
Epoch 44/50 Batch 3/3 Cost: 1.730717
Epoch 45/50 Batch 1/3 Cost: 0.243121
Epoch 45/50 Batch 2/3 Cost: 1.203367
Epoch 45/50 Batch 3/3 Cost: 1.212969
Epoch 46/50 Batch 1/3 Cost: 0.530609
Epoch 46/50 Batch 2/3 Cost: 1.143835
Epoch 46/50 Batch 3/3 Cost: 0.297622
Epoch 47/50 Batch 1/3 Cost: 0.450287
Epoch 47/50 Batch 2/3 Cost: 0.891708
Epoch 47/50 Batch 3/3 Cost: 1.408309
Epoch 48/50 Batch 1/3 Cost: 1.374655
Epoch 48/50 Batch 2/3 Cost: 1.342901
Epoch 48/50 Batch 3/3 Cost: 0.535213
Epoch 49/50 Batch 1/3 Cost: 0.575596
Epoch 49/50 Batch 2/3 Cost: 0.834751
Epoch 49/50 Batch 3/3 Cost: 1.450522
Epoch 50/50 Batch 1/3 Cost: 1.048827
Epoch 50/50 Batch 2/3 Cost: 0.574974
Epoch 50/50 Batch 3/3 Cost: 0.376825
cost값이 점점 작아진다. 에포크를 더 늘리면 cost가 더 작아짐
이제 임의의 값을 넣어 예측값을 확인해보자.
new_var = torch.FloatTensor([[50,39,78]])
pred_y = model(new_var)
print(pred_y)
##################output###################
tensor([[105.0303]], grad_fn=<AddmmBackward0>)
https://deeplearningzerotoall.github.io/season2/lec_pytorch.html
'Pytorch' 카테고리의 다른 글
MNIST 분류 by softmax regression (0) | 2022.12.29 |
---|---|
SoftMax Regression (0) | 2022.12.29 |
Logistic Regression (0) | 2022.12.29 |
Linear Regression (0) | 2022.12.27 |
Tensor manipulation (0) | 2022.12.27 |