Mini Batch and Data Load

Mini Batch 

전체 데이터를 더 작은 단위로 나누어서 해당 단위로 학습하는 개념 

 

 

미니 배치 학습 시, 미니 배치만큼 가져가서 그에 대한 비용(cost)를 계산하고 경사하강법 수행한다.

그리고 그 다음 미니 배치를 가져가서 경사 하강법을 사용하고 마지막까지 이를 반복.

이렇게 전체 데이터에 대한 학습 1회 끝나면 -> 1 Epoch DONE

*Epoch(에포크) : 전체 train data가 학습에 한번 사용된 주기를 말함

*미니 배치 경사 하강법은 전체 데이터의 일부만을 보고 경사 하강법을 수행하므로 최적값으로 수렴하는 과정에서 값이 조금 헤매기도 하지만 training 속도가 빠름

*Iteration : 한 번의 에포크 내에서 이루어지는 매개변수인 W와 b의 업데이트 횟수. 

 

Data Load

from torch.utils.data import TensorDataset # 텐서데이터셋
from torch.utils.data import DataLoader # 데이터로더

x_train  =  torch.FloatTensor([[73,  80,  75], 
                               [93,  88,  93], 
                               [89,  91,  90], 
                               [96,  98,  100],   
                               [73,  66,  70]])  
y_train  =  torch.FloatTensor([[152],  [185],  [180],  [196],  [142]])

dataset = TensorDataset(x_train, y_train)

파이토치 데이터셋을 만들고 데이터로더 사용. 

dataloader = DataLoader(dataset, batch_size = 2, shuffle=True)

사용할 데이터 셋, 미니 배치의 크기 지정

*보통 미니 배치의 크기는 통상적으로 2의 배수로 한다. 

*shuffle = True : Epoch마다 데이터셋을 섞어서 데이터가 학습되는 순서 바꿈. 

model = nn.Linear(3,1)
optimizer = torch.optim.SGD(model.parameters(), lr=1e-5)

모델과 옵티마이저를 설계한다. 

 

nb_epochs = 50
for epoch in range(nb_epochs + 1):
  for batch_idx, samples in enumerate(dataloader):
    # print(batch_idx)
    # print(samples)
    x_train, y_train = samples
    # H(x) 계산
    prediction = model(x_train)

    # cost 계산
    cost = F.mse_loss(prediction, y_train)

    # cost로 H(x) 계산
    optimizer.zero_grad()
    cost.backward()
    optimizer.step()

    print('Epoch {:4d}/{} Batch {}/{} Cost: {:.6f}'.format(
        epoch, nb_epochs, batch_idx+1, len(dataloader),
        cost.item()
        ))

 

*output

 

Epoch    0/50 Batch 1/3 Cost: 1.242304
Epoch    0/50 Batch 2/3 Cost: 0.563961
Epoch    0/50 Batch 3/3 Cost: 0.448591
Epoch    1/50 Batch 1/3 Cost: 0.692483
Epoch    1/50 Batch 2/3 Cost: 1.106694
Epoch    1/50 Batch 3/3 Cost: 0.213365
Epoch    2/50 Batch 1/3 Cost: 0.064273
Epoch    2/50 Batch 2/3 Cost: 1.796256
Epoch    2/50 Batch 3/3 Cost: 1.607992
Epoch    3/50 Batch 1/3 Cost: 0.984423
Epoch    3/50 Batch 2/3 Cost: 1.063094
Epoch    3/50 Batch 3/3 Cost: 0.182904
Epoch    4/50 Batch 1/3 Cost: 1.582306
Epoch    4/50 Batch 2/3 Cost: 1.148095
Epoch    4/50 Batch 3/3 Cost: 0.063173
Epoch    5/50 Batch 1/3 Cost: 0.919290
Epoch    5/50 Batch 2/3 Cost: 1.062542
Epoch    5/50 Batch 3/3 Cost: 0.135253
Epoch    6/50 Batch 1/3 Cost: 1.065469
Epoch    6/50 Batch 2/3 Cost: 0.789830
Epoch    6/50 Batch 3/3 Cost: 0.269580
Epoch    7/50 Batch 1/3 Cost: 1.052595
Epoch    7/50 Batch 2/3 Cost: 0.737678
Epoch    7/50 Batch 3/3 Cost: 0.290358
Epoch    8/50 Batch 1/3 Cost: 1.379399
Epoch    8/50 Batch 2/3 Cost: 0.645907
Epoch    8/50 Batch 3/3 Cost: 0.990046
Epoch    9/50 Batch 1/3 Cost: 1.161351
Epoch    9/50 Batch 2/3 Cost: 0.983895
Epoch    9/50 Batch 3/3 Cost: 0.142623
Epoch   10/50 Batch 1/3 Cost: 0.858378
Epoch   10/50 Batch 2/3 Cost: 1.062926
Epoch   10/50 Batch 3/3 Cost: 0.148715
Epoch   11/50 Batch 1/3 Cost: 0.777309
Epoch   11/50 Batch 2/3 Cost: 0.682220
Epoch   11/50 Batch 3/3 Cost: 1.290777
Epoch   12/50 Batch 1/3 Cost: 0.954402
Epoch   12/50 Batch 2/3 Cost: 1.085446
Epoch   12/50 Batch 3/3 Cost: 0.110559
Epoch   13/50 Batch 1/3 Cost: 0.041273
Epoch   13/50 Batch 2/3 Cost: 1.833644
Epoch   13/50 Batch 3/3 Cost: 1.557202
Epoch   14/50 Batch 1/3 Cost: 1.191278
Epoch   14/50 Batch 2/3 Cost: 0.751343
Epoch   14/50 Batch 3/3 Cost: 0.163530
Epoch   15/50 Batch 1/3 Cost: 0.819364
Epoch   15/50 Batch 2/3 Cost: 1.062180
Epoch   15/50 Batch 3/3 Cost: 0.199147
Epoch   16/50 Batch 1/3 Cost: 0.749296
Epoch   16/50 Batch 2/3 Cost: 1.041121
Epoch   16/50 Batch 3/3 Cost: 0.212914
Epoch   17/50 Batch 1/3 Cost: 0.803099
Epoch   17/50 Batch 2/3 Cost: 0.664343
Epoch   17/50 Batch 3/3 Cost: 1.702409
Epoch   18/50 Batch 1/3 Cost: 0.602896
Epoch   18/50 Batch 2/3 Cost: 0.553041
Epoch   18/50 Batch 3/3 Cost: 1.596134
Epoch   19/50 Batch 1/3 Cost: 0.968940
Epoch   19/50 Batch 2/3 Cost: 0.662227
Epoch   19/50 Batch 3/3 Cost: 1.189889
Epoch   20/50 Batch 1/3 Cost: 0.161107
Epoch   20/50 Batch 2/3 Cost: 2.475749
Epoch   20/50 Batch 3/3 Cost: 0.269707
Epoch   21/50 Batch 1/3 Cost: 0.789196
Epoch   21/50 Batch 2/3 Cost: 1.059804
Epoch   21/50 Batch 3/3 Cost: 0.163728
Epoch   22/50 Batch 1/3 Cost: 1.043981
Epoch   22/50 Batch 2/3 Cost: 0.069241
Epoch   22/50 Batch 3/3 Cost: 1.800337
Epoch   23/50 Batch 1/3 Cost: 1.209918
Epoch   23/50 Batch 2/3 Cost: 0.714378
Epoch   23/50 Batch 3/3 Cost: 0.956597
Epoch   24/50 Batch 1/3 Cost: 1.406833
Epoch   24/50 Batch 2/3 Cost: 0.667870
Epoch   24/50 Batch 3/3 Cost: 0.916201
Epoch   25/50 Batch 1/3 Cost: 0.593989
Epoch   25/50 Batch 2/3 Cost: 1.192008
Epoch   25/50 Batch 3/3 Cost: 0.439744
Epoch   26/50 Batch 1/3 Cost: 0.687513
Epoch   26/50 Batch 2/3 Cost: 1.029234
Epoch   26/50 Batch 3/3 Cost: 0.221504
Epoch   27/50 Batch 1/3 Cost: 1.031025
Epoch   27/50 Batch 2/3 Cost: 0.083856
Epoch   27/50 Batch 3/3 Cost: 1.742661
Epoch   28/50 Batch 1/3 Cost: 0.618320
Epoch   28/50 Batch 2/3 Cost: 1.077366
Epoch   28/50 Batch 3/3 Cost: 1.431801
Epoch   29/50 Batch 1/3 Cost: 0.565040
Epoch   29/50 Batch 2/3 Cost: 1.195735
Epoch   29/50 Batch 3/3 Cost: 0.345257
Epoch   30/50 Batch 1/3 Cost: 0.494264
Epoch   30/50 Batch 2/3 Cost: 0.889818
Epoch   30/50 Batch 3/3 Cost: 1.347875
Epoch   31/50 Batch 1/3 Cost: 0.409979
Epoch   31/50 Batch 2/3 Cost: 2.180332
Epoch   31/50 Batch 3/3 Cost: 0.189834
Epoch   32/50 Batch 1/3 Cost: 0.060971
Epoch   32/50 Batch 2/3 Cost: 1.050519
Epoch   32/50 Batch 3/3 Cost: 1.602038
Epoch   33/50 Batch 1/3 Cost: 1.262114
Epoch   33/50 Batch 2/3 Cost: 0.688861
Epoch   33/50 Batch 3/3 Cost: 0.927570
Epoch   34/50 Batch 1/3 Cost: 1.213289
Epoch   34/50 Batch 2/3 Cost: 0.571694
Epoch   34/50 Batch 3/3 Cost: 0.499225
Epoch   35/50 Batch 1/3 Cost: 0.687431
Epoch   35/50 Batch 2/3 Cost: 0.594548
Epoch   35/50 Batch 3/3 Cost: 1.329620
Epoch   36/50 Batch 1/3 Cost: 1.934188
Epoch   36/50 Batch 2/3 Cost: 0.921663
Epoch   36/50 Batch 3/3 Cost: 0.052485
Epoch   37/50 Batch 1/3 Cost: 1.058650
Epoch   37/50 Batch 2/3 Cost: 0.856656
Epoch   37/50 Batch 3/3 Cost: 0.181225
Epoch   38/50 Batch 1/3 Cost: 0.753086
Epoch   38/50 Batch 2/3 Cost: 0.637194
Epoch   38/50 Batch 3/3 Cost: 1.249881
Epoch   39/50 Batch 1/3 Cost: 0.921967
Epoch   39/50 Batch 2/3 Cost: 0.744168
Epoch   39/50 Batch 3/3 Cost: 1.139925
Epoch   40/50 Batch 1/3 Cost: 1.091079
Epoch   40/50 Batch 2/3 Cost: 1.016668
Epoch   40/50 Batch 3/3 Cost: 0.136321
Epoch   41/50 Batch 1/3 Cost: 0.749371
Epoch   41/50 Batch 2/3 Cost: 1.021392
Epoch   41/50 Batch 3/3 Cost: 0.183715
Epoch   42/50 Batch 1/3 Cost: 0.759830
Epoch   42/50 Batch 2/3 Cost: 1.036982
Epoch   42/50 Batch 3/3 Cost: 0.183283
Epoch   43/50 Batch 1/3 Cost: 0.361928
Epoch   43/50 Batch 2/3 Cost: 0.943031
Epoch   43/50 Batch 3/3 Cost: 1.415997
Epoch   44/50 Batch 1/3 Cost: 0.562195
Epoch   44/50 Batch 2/3 Cost: 0.545631
Epoch   44/50 Batch 3/3 Cost: 1.730717
Epoch   45/50 Batch 1/3 Cost: 0.243121
Epoch   45/50 Batch 2/3 Cost: 1.203367
Epoch   45/50 Batch 3/3 Cost: 1.212969
Epoch   46/50 Batch 1/3 Cost: 0.530609
Epoch   46/50 Batch 2/3 Cost: 1.143835
Epoch   46/50 Batch 3/3 Cost: 0.297622
Epoch   47/50 Batch 1/3 Cost: 0.450287
Epoch   47/50 Batch 2/3 Cost: 0.891708
Epoch   47/50 Batch 3/3 Cost: 1.408309
Epoch   48/50 Batch 1/3 Cost: 1.374655
Epoch   48/50 Batch 2/3 Cost: 1.342901
Epoch   48/50 Batch 3/3 Cost: 0.535213
Epoch   49/50 Batch 1/3 Cost: 0.575596
Epoch   49/50 Batch 2/3 Cost: 0.834751
Epoch   49/50 Batch 3/3 Cost: 1.450522
Epoch   50/50 Batch 1/3 Cost: 1.048827
Epoch   50/50 Batch 2/3 Cost: 0.574974
Epoch   50/50 Batch 3/3 Cost: 0.376825

cost값이 점점 작아진다. 에포크를 더 늘리면 cost가 더 작아짐

이제 임의의 값을 넣어 예측값을 확인해보자.

new_var = torch.FloatTensor([[50,39,78]])
pred_y = model(new_var)
print(pred_y)

##################output###################

tensor([[105.0303]], grad_fn=<AddmmBackward0>)

 

 

https://wikidocs.net/55580

https://deeplearningzerotoall.github.io/season2/lec_pytorch.html