Skip to content
Snippets Groups Projects
Select Git revision
  • dd5d15be5b3e94c0995ee31c9c2314335d855b60
  • main default protected
2 results

train_results.txt

Blame
  • train_results.txt 8.59 KiB
    Training with 7 classes:  T2star: 25, T2w: 1156, FLAIRCE: 1126, FLAIR: 5950, T1w: 5881, OTHER: 211, T1wCE: 5947
    Machine: Cremi/falcon core:24, CPU: i9-14900, 64Go, GPU: RTX 4060 (8Go)
    
    
    fix parameters:
        criterion_balanced = nn.CrossEntropyLoss(weight = class_weights_tensor)
        optimizer_Adam = optim.Adam(model.parameters(), 1e-3)
        scaler = torch.cuda.amp.GradScaler()
        epochs = 30
        BATCH_SIZE = 64
        WORKERS = 8
    
    MODEL_NAME = 'efficientnet_b0'
        time: 33 min 25 sec
        Best epoch: 26
            validation accuracy: 0.9303
            validation loss: 1.6010
            test accuracy: 98.68%
            balanced accuracy: 94.15%
            size: 16MB
        checkpoints: /net/travail/bformanek/checkpoints/transfer_checkpoints_efficientnet_b0_adam_amp_criterion_balanced
    
    MODEL_NAME = 'resnet50'
        time: 56 min 39 sec
        Best epoch: 18 (epoch 22 fails to be loaded)
            validation accuracy: 0.9237
            validation loss: 1.6202
            test accuracy: 98.10%
            balanced accuracy: 94.37%
            size: 91MB
        checkpoints: /net/travail/bformanek/checkpoints/transfer_checkpoints_resnet50_adam_amp_criterion_balanced
    
    MODEL_NAME = 'resnet18'
        time: 21 min 24 sec
        Best epoch: 25
            validation accuracy: 0.9321
            validation loss: 1.8383
            test accuracy: 98.77%
            balanced accuracy: 99.11%
            size: 43MB
        checkpoints: /net/travail/bformanek/checkpoints/transfer_checkpoints_resnet18_adam_amp_criterion_balanced
        Inference:
            - time on GPU:  ms
            - GPU memory use:  bytes
            - time on CPU:  ms
            
    MODEL_NAME = 'resnet18.a2_in1k'
        time: 20 min 6 sec
        Best epoch: 12
            validation accuracy: 0.9317
            validation loss: 1.2443
            test accuracy: 98.77%
            balanced accuracy: 99.18%
            size: 43 MB
        checkpoints: /net/travail/bformanek/checkpoints/transfer_checkpoints_resnet18.a2_in1k
        Inference:
            - time on GPU:  ms
            - GPU memory use:  bytes
            - time on CPU:  ms
    
    MODEL_NAME = 'resnet18.fb_ssl_yfcc100m_ft_in1k'
        time: 20 min 4 sec
        Best epoch: 22
            validation accuracy: 0.9227
            validation loss: 1.8298
            test accuracy: 97.49%
            balanced accuracy: 97.30%
            size: 43MB
        checkpoints: /net/travail/bformanek/checkpoints/transfer_checkpoints_resnet18.fb_ssl_yfcc100m_ft_in1k
        Inference:
            - time on GPU:  ms
            - GPU memory use:  bytes
            - time on CPU:  ms
    
    MODEL_NAME = 'mobilenetv3_small_100.lamb_in1k'
        time: 7 min 32 sec
        Best epoch: 28
            validation accuracy: 0.9206
            validation loss: 1.8223
            test accuracy: 97.01%
            balanced accuracy: 97.25%
            size: 6 MB
        checkpoints: /net/travail/bformanek/checkpoints/transfer_checkpoints_mobilenetv3_small_100.lamb_in1k
        Inference:
            - time on GPU:  ms
            - GPU memory use:  bytes
            - time on CPU:  ms
            
    MODEL_NAME = 'mobilenetv4_conv_large.e500_r256_in1k'
        time: 46 min 50 sec
        Best epoch: 4
            validation accuracy: 0.9216
            validation loss: 1.0341
            test accuracy: 97.27%
            balanced accuracy: 97.44%
            size: 121 MB
        checkpoints: /net/travail/bformanek/checkpoints/transfer_checkpoints_mobilenetv4_conv_large.e500_r256_in1k
        Inference:
            - time on GPU:  sec
            - GPU memory use:  KB
            - time on CPU:  sec
      
    MODEL_NAME = 'mobilenetv4_hybrid_medium.e500_r224_in1k'
        time: 30  min 3 sec
        Best epoch: 7
            validation accuracy: 0.9112
            validation loss: 0.8010
            test accuracy: 97.04%
            balanced accuracy: 87.13%
            size: 39 MB
        checkpoints: /net/travail/bformanek/checkpoints/transfer_checkpoints_mobilenetv4_hybrid_medium.e500_r224_in1k
        Inference:
            - time on GPU:  sec
            - GPU memory use:  KB
            - time on CPU:  sec
    
    -------------------------------------------------------------------------------------------------------------
    ADD Augmentation to train images:
        
    
    MODEL_NAME = 'resnet18'
    transforms.v2.RandomHorizontalFlip(p=0.5),
    transforms.v2.RandomRotation(degrees=15),
    transforms.v2.ColorJitter(contrast=0.3),
    transforms.v2.GaussianBlur(7, sigma=2),
    RandomResample(scale_factor=4),
        time: ~20 min (I forgot to save)
        Best epoch: 21
            validation accuracy: 0.5566
            validation loss: 4.0260
            test accuracy: 57.37%
            balanced accuracy: 42.34%
            size: 43MB
        checkpoints: /net/travail/bformanek/checkpoints/transfer_checkpoints_resnet18_augmentation1
        note: poor results on validation and test sets
    
    augmentation2: lower random contrast, blur and resample 
    MODEL_NAME = 'resnet18'
    transforms.v2.RandomHorizontalFlip(p=0.5),
    transforms.v2.RandomRotation(degrees=15),
    transforms.v2.ColorJitter(contrast=0.1),
    transforms.v2.GaussianBlur(7, sigma=1),
    RandomResample(scale_factor=2),
        time: 20 min 44 sec
        Best epoch: 19
            validation accuracy: 0.8666
            validation loss: 1.1987
            test accuracy: 91.48%
            balanced accuracy: 92.11%
            size: 43MB
        checkpoints: /net/travail/bformanek/checkpoints/transfer_checkpoints_resnet18_augmentation2
        note: much better results, but only around 90%, this cen be the result of generalization, but there are distinct faulty cells in the confusion matrix 
            2 confusion nodes: t2 <-> t2*, t1<->flair -> can still be the problem of contrast manipulation 
    
    augmentation3: without contrast change, have higehr blur (from augmentation1) and same resample (augmentation2)
    MODEL_NAME = 'resnet18'
    transforms.v2.RandomHorizontalFlip(p=0.5),
    transforms.v2.RandomRotation(degrees=15),
    transforms.v2.GaussianBlur(7, sigma=2),
    RandomResample(scale_factor=2),
    transforms.ToTensor()
        time: 20 min 36 sec
        Best epoch: 3
            validation accuracy: 0.5524
            validation loss: 2.5870
            test accuracy: 59.24%
            balanced accuracy: 46.14%
            size: 43MB
        checkpoints: /net/travail/bformanek/checkpoints/transfer_checkpoints_resnet18_augmentation3
        note: essentially no training, everithing is detected as t2* or t1
    
    augmentation4: without contrast change, blur or resampling, just rotation and flip
    MODEL_NAME = 'resnet18'
    transforms.v2.RandomHorizontalFlip(p=0.5),
    transforms.v2.RandomRotation(degrees=15),
    transforms.ToTensor()
        time: 20 min 22 sec
        Best epoch: 24
            validation accuracy: 0.9216
            validation loss: 1.6285
            test accuracy: 98.53%
            balanced accuracy: 98.77%
            size: 43MB
        checkpoints: /net/travail/bformanek/checkpoints/transfer_checkpoints_resnet18_augmentation4
        note: as expected: much better results, but the learning curves still show overfitting
    
    augmentation using horizontal and vertical flips: 
    MODEL_NAME = 'resnet18_flips'
    transforms.v2.RandomHorizontalFlip(p=0.5),
    transforms.v2.RandomVerticalFlip(p=0.5),
    transforms.ToTensor()
        time: 20 min 16 sec
        Best epoch: 22
            validation accuracy: 0.9294
            validation loss: 1.9463
            test accuracy: 98.71%
            balanced accuracy: 98.73%
            size: 43MB
        checkpoints: /net/travail/bformanek/checkpoints/transfer_checkpoints_resnet18_flips
    
    augmentation using horizontal, vertical flips and 90 degrees random rotation: 
    MODEL_NAME = 'resnet18_flips_90'
    transforms.v2.RandomHorizontalFlip(p=0.5),
    transforms.v2.RandomVerticalFlip(p=0.5),
    transforms.v2.RandomRotation(degrees=90, expand=True),
    transforms.ToTensor()
        time: 20 min 19 sec
        Best epoch: 13
            validation accuracy: 89.24%
            validation loss: 1.8287
            test accuracy: 93.27%
            balanced accuracy: 95.00%
            size: 43
        checkpoints: /net/travail/bformanek/checkpoints/transfer_checkpoints_resnet18_flips_90
    
    augmentation using horizontal, vertical flips and 180 degrees random rotation: 
    MODEL_NAME = 'resnet18_flips_180'
    transforms.v2.RandomHorizontalFlip(p=0.5),
    transforms.v2.RandomVerticalFlip(p=0.5),
    transforms.v2.RandomRotation(degrees=180, expand=True),
    transforms.ToTensor()
        time: 20 min 12 sec
        Best epoch: 22
            validation accuracy: 1.7595
            validation loss: 90.17%
            test accuracy: 95.95%
            balanced accuracy: 95.60%
            size: 43
        checkpoints: /net/travail/bformanek/checkpoints/transfer_checkpoints_resnet18_flips_180
    
    augmentation using horizontal, vertical flips and 360 degrees random rotation: 
    MODEL_NAME = 'resnet18_flips_360'
    transforms.v2.RandomHorizontalFlip(p=0.5),
    transforms.v2.RandomVerticalFlip(p=0.5),
    transforms.v2.RandomRotation(degrees=360, expand=True),
    transforms.ToTensor()
        time: 20 min 12 sec
        Best epoch: 22
            validation accuracy: 1.8767
            validation loss: 90.58%
            test accuracy: 95.13%
            balanced accuracy: 95.60%
            size: 43
        checkpoints: /net/travail/bformanek/checkpoints/transfer_checkpoints_resnet18_flips_360