Open In Colab

Open In Colab
Code 
# Clone the entire repository
!git clone https://github.com/emilianodesu/MLA2.git

# Install required packages using the requirements.txt file inside the cloned directory
%pip install torch torchinfo torchvision

# Change the current working directory to the cloned repository folder
import os
os.chdir('MLA2')

1. Imports

Code 
import os
import random
import re
import json
import csv
from pathlib import Path
import torch
from torchinfo import summary
from preprocessing_utils import prepare_images, summarize_split, preview_random_images, get_dataloaders, show_batch
from models_utils import train_model, test_model, plot_confusion_matrix, plot_training_process, predict_image
from mlp import MLPBaseline
Code 
print(torch.__version__)
# Device setup
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"Using device: {device}")
2.8.0+cpu
Using device: cpu

## 2. Getting Images Ready

The dataset used in this project is the Chinese MNIST dataset, which contains images of handwritten Chinese numerals (digits 0-9) written by multiple individuals. Each image is labeled with its corresponding digit and additional label. The dataset is organized as follows:

  • Images: Located in the data/c_mnist folder, each image file is named according to its suite, sample, and code.

  • Labels: Provided in the data/chinese_mnist.csv file, which includes the digit value, character, and code for each image.

In this section, the following steps are performed:

  1. Download and Extraction: The dataset is downloaded (if not already present) and extracted to the data directory.

  2. Splitting: The images are split into training, validation, and test sets according to a specified ratio, with each split organized into subfolders by class.

  3. Summary and Preview: The script prints a summary of the number of images and classes in each split, and displays a random sample of images from the dataset for visual inspection.

The default ratio is training 80%, validation 10% and testing 10%.

Code 
root = prepare_images()
print(f"Dataset ready at: {root}")
Dataset already exists, skipping download.
Train/Val/Test folders already exist. (Set overwrite=True to recreate them)
Dataset ready at: data
Code 
summarize_split(str(root))
train: 12000 images, 15 classes
{'0': 800, '1': 800, '10': 800, '100': 800, '1000': 800, '10000': 800, '100000000': 800, '2': 800, '3': 800, '4': 800, '5': 800, '6': 800, '7': 800, '8': 800, '9': 800}
val: 1500 images, 15 classes
{'0': 100, '1': 100, '10': 100, '100': 100, '1000': 100, '10000': 100, '100000000': 100, '2': 100, '3': 100, '4': 100, '5': 100, '6': 100, '7': 100, '8': 100, '9': 100}
test: 1500 images, 15 classes
{'0': 100, '1': 100, '10': 100, '100': 100, '1000': 100, '10000': 100, '100000000': 100, '2': 100, '3': 100, '4': 100, '5': 100, '6': 100, '7': 100, '8': 100, '9': 100}
Code 
preview_random_images(data_dir=str(root / "c_mnist"), n_images=9, grid_size=(3, 3))

3. Preprocessing data

This section prepares image tensors and DataLoaders used for training and evaluation.

What we do here:

  • Always convert images to grayscale (1 channel) first, then resize to a fixed image_size (e.g., 64x64).
  • Training-only augmentation (when augment=True): small random rotation (±10°) and RandomAffine with up to 5% translation. Validation and test receive no augmentation.
  • Convert images to tensors with transforms.ToTensor() so pixel values are in [0, 1].
  • Optional normalization (when normalize=True): applies transforms.Normalize((mean,), (std,)) for the single grayscale channel. Defaults are mean=0, std=1. Use the same mean/std for train/val/test when enabled.
  • Optional flattening (when flatten=True): flattens the tensor to a 1-D vector via a Lambda step. Keep flatten=True for MLPs; set flatten=False for CNNs.
  • The overall transform order is: Grayscale → Resize → [Augment (train only)] → ToTensor → [Normalize] → [Flatten].

How datasets and loaders are built:

  • Create ImageFolder datasets from data/train, data/val, and data/test; class labels come from the subfolder names created in the previous step.
  • Wrap datasets with PyTorch DataLoaders using the chosen batch_size. The training loader is shuffled; val/test are not.

Inputs / outputs: - Inputs: images on disk under data/{train,val,test}. - Outputs: dataloaders and datasets dicts where dataloaders['train'] yields (X, y) batches ready for training.

Notes:

  • Normalization is off by default. Enable normalize=True to get zero-mean/unit-variance inputs based on the provided mean/std (single-channel).
  • Augmentation is intentionally light to preserve digit structure; adjust or disable as needed.
  • If splits are missing, call prepare_images(overwrite=False) to (re)create them.
Code 
dataloaders, datasets = get_dataloaders(data_dir="data", batch_size=64, image_size=(64, 64), augment=True)
labels = datasets['train'].classes
print("DataLoaders ready.")
print(f"Classes: {labels}")
DataLoaders ready.
Classes: ['0', '1', '10', '100', '1000', '10000', '100000000', '2', '3', '4', '5', '6', '7', '8', '9']
Code 
show_batch(datasets['train'], n=12, cols=4)

## 4. Parameters and Architectures

We now use a configurable Multi-Layer Perceptron (MLP) to classify Chinese MNIST digits. Images are 64×64 grayscale and flattened to 4096 input features. The task has 15 classes (based on the chosen labeling scheme during dataset preparation).

  • INPUT_DIM = 64 * 64 (4096)
    Flattened size of each grayscale image. The MLP expects a 1D vector per sample. Flattening happens in the preprocessing transforms when flatten=True.
  • NUM_CLASSES = 15
    Output layer size; the model returns logits of shape (batch_size, 15) consumed by CrossEntropyLoss.
  • BATCH_SIZE = [32, 256]
    We evaluate both a smaller and a larger batch size. Small batches introduce gradient noise (potentially better generalization); large batches are computationally efficient and yield smoother updates.
  • EPOCHS = 50
    Upper bound for training. Early stopping (via PATIENCE) prevents over-training when validation accuracy plateaus.
  • LEARNING_RATE = [0.01, 0.001, 0.0005]
    A coarse-to-fine sweep: high (0.01) for rapid progress, mid (0.001) for stability, and low (0.0005) for fine convergence—tested for BOTH optimizers.
  • WEIGHT_DECAY = 5e-4
    Acts as L2 regularization on weights; helps prevent overfitting on fully-connected layers which can easily memorize pixel-level noise.
  • MOMENTUM = 0.9 (SGD only)
    Speeds up traversal across ravines and dampens oscillations. Traditional and effective default for vision tasks.
  • PATIENCE = 5
    If validation accuracy does not improve for 5 consecutive epochs, training stops early and the best checkpoint is kept.

Architecture Search Parameters

Instead of a single fixed layout, we explore multiple MLP configurations. Each architecture is encoded in a descriptive name and saved to disk using a consistent pattern.

  • MLP_HIDDEN_LAYERS = [1, 2, 3, 4]
    Depth controls representational hierarchy. Too shallow → underfit; too deep for this dataset → diminishing returns / overfit.
  • MLP_HIDDEN_UNITS = [1024, 512, 256, 128]
    Candidate widths for layers. Wider early layers allow rich feature mixing; later shrinking layers encourage abstraction and compression.
  • DROPOUT_RATE = 0.5
    Applied after each hidden layer to reduce co-adaptation and mitigate overfitting, especially important in dense networks without spatial inductive bias.

Architecture Set (architectures list)

Example entries (shown later in code):
- mlp_1x512 → 1 hidden layer of 512 units
- mlp_2x256 → 2 hidden layers, each 256 units (symmetric)
- mlp_3x512-256-128 → 3 hidden layers of decreasing width
- (You can easily extend with mlp_4x1024-512-256-128)

Each name encodes:
mlp_{depth}x{unitsPattern} where unitsPattern is either a single integer (repeated width) or a hyphen-separated list of layer sizes.

Checkpoint Naming Convention

During training we save best-validation checkpoints per (architecture, batch size, optimizer, learning rate):

checkpoints/mlp/{architecture_name}_{batch_size}_{optimizer}_{learning_rate}.pth
# Example: checkpoints/mlp/mlp_3x512-256-128_32_adam_0.001.pth

If an optimizer segment is absent (fallback cases), filenames still follow the prefix and structural pattern.

Summary

Category Key Choices Justification
Input Flattened 64×64 grayscale Simplifies model to dense layers
Depth 1–4 layers Explore capacity vs overfit
Width 1024→128 progression Wide early abstraction, tapered compression
Regularization Dropout 0.5, weight decay 5e-4 Prevent overfitting in dense nets
Optimizers Adam, SGD+Momentum Compare adaptive vs classical momentum
LR Sweep 0.01 / 0.001 / 0.0005 Coarse → fine convergence control
Early Stop Patience 5 Efficiency & generalization
Persistence JSON + CSV per run Reproducible offline analysis
Naming mlp_{depth}x{units} Transparent architecture encoding
Code 
sample_batch = next(iter(dataloaders["train"]))[0]
# For flattened 64x64 images
INPUT_DIM = sample_batch.shape[1] if sample_batch.ndim == 2 else sample_batch[0].numel()
NUM_CLASSES = len(labels)

BATCH_SIZE = [32, 256]
EPOCHS = 50
PATIENCE = 5
LEARNING_RATE = [0.01, 0.001, 0.0005]
WEIGHT_DECAY = 5e-4
MOMENTUM = 0.9 # For SGD

MLP_HIDDEN_LAYERS = [1, 2, 3, 4]
MLP_HIDDEN_UNITS = [1024, 512, 256, 128]
DROPOUT_RATE = 0.5

We define ranges for batch size, learning rate, hidden layers, and hidden units to allow hyperparameter search to find the best combination. Other parameters are fixed based on common defaults and prior experience with similar tasks. These are the architectures we will explore:

Code 
architectures = [
    {"name": "mlp_1x512", "hidden_layers": MLP_HIDDEN_LAYERS[0], "hidden_units": MLP_HIDDEN_UNITS[1], "dropout": DROPOUT_RATE},
    {"name": "mlp_2x256", "hidden_layers": MLP_HIDDEN_LAYERS[1], "hidden_units": MLP_HIDDEN_UNITS[2], "dropout": DROPOUT_RATE},
    {"name": "mlp_3x512-256-128", "hidden_layers": MLP_HIDDEN_LAYERS[2], "hidden_units": MLP_HIDDEN_UNITS[1:], "dropout": DROPOUT_RATE}
]

5. Loss and Optimizer

Loss

The standard loss for multi-class classification is Cross-Entropy Loss (nn.CrossEntropyLoss in PyTorch). This loss measures the difference between the predicted class probabilities (from model logits) and the true class labels. It combines a softmax activation with negative log-likelihood, encouraging the model to assign high probability to the correct class and penalizing incorrect predictions. Cross-entropy is robust, differentiable, and well-suited for tasks where each input belongs to exactly one of several classes, such as digit recognition.

Optimizers

Stochastic Gradient Descent (SGD) updates model parameters by moving them in the direction that reduces the loss, using gradients computed from random mini-batches of data. This introduces beneficial noise, helping the optimizer escape local minima and improving generalization. SGD is simple, memory-efficient, and, when combined with momentum, can accelerate convergence and smooth parameter updates. However, it requires careful tuning of the learning rate and may converge slowly, especially on problems with complex or poorly scaled loss surfaces.

Adam (Adaptive Moment Estimation) extends SGD by maintaining running averages of both the gradients and their squared values for each parameter, enabling adaptive learning rates. Adam typically converges faster and is less sensitive to the initial learning rate, making it robust and easy to use for a wide range of problems. It is especially effective for large datasets and high-dimensional parameter spaces.

Code 
# Model definition
mlp_model = MLPBaseline(input_dim=INPUT_DIM, num_classes=NUM_CLASSES, hidden_layers=MLP_HIDDEN_LAYERS[2], hidden_units=MLP_HIDDEN_UNITS[1:], dropout=DROPOUT_RATE)

# Loss and optimizers
criterion = torch.nn.CrossEntropyLoss()

## 6. Models Summary

Code 
print(f"MLP summary (input_dim={INPUT_DIM}, num_classes={NUM_CLASSES})")
summary(mlp_model, input_size=(1, INPUT_DIM), col_names=("input_size", "output_size", "num_params"))
MLP summary (input_dim=4096, num_classes=15)
===================================================================================================================
Layer (type:depth-idx)                   Input Shape               Output Shape              Param #
===================================================================================================================
MLPBaseline                              [1, 4096]                 [1, 15]                   --
├─Sequential: 1-1                        [1, 4096]                 [1, 15]                   --
│    └─Linear: 2-1                       [1, 4096]                 [1, 512]                  2,097,664
│    └─BatchNorm1d: 2-2                  [1, 512]                  [1, 512]                  1,024
│    └─ReLU: 2-3                         [1, 512]                  [1, 512]                  --
│    └─Dropout: 2-4                      [1, 512]                  [1, 512]                  --
│    └─Linear: 2-5                       [1, 512]                  [1, 256]                  131,328
│    └─BatchNorm1d: 2-6                  [1, 256]                  [1, 256]                  512
│    └─ReLU: 2-7                         [1, 256]                  [1, 256]                  --
│    └─Dropout: 2-8                      [1, 256]                  [1, 256]                  --
│    └─Linear: 2-9                       [1, 256]                  [1, 128]                  32,896
│    └─BatchNorm1d: 2-10                 [1, 128]                  [1, 128]                  256
│    └─ReLU: 2-11                        [1, 128]                  [1, 128]                  --
│    └─Dropout: 2-12                     [1, 128]                  [1, 128]                  --
│    └─Linear: 2-13                      [1, 128]                  [1, 15]                   1,935
===================================================================================================================
Total params: 2,265,615
Trainable params: 2,265,615
Non-trainable params: 0
Total mult-adds (Units.MEGABYTES): 2.27
===================================================================================================================
Input size (MB): 0.02
Forward/backward pass size (MB): 0.01
Params size (MB): 9.06
Estimated Total Size (MB): 9.09
===================================================================================================================

7. Training

First, we are going to create different dataloaders for each batch size and training the model for each batch size.

Code 
dataloaders_dict = {}
for bs in BATCH_SIZE:
    dl, _ = get_dataloaders(data_dir="data", batch_size=bs, image_size=(64, 64), augment=True)
    dataloaders_dict[bs] = dl
    print(f"DataLoaders for batch size {bs} ready.")
DataLoaders for batch size 32 ready.
DataLoaders for batch size 256 ready.

To enable offline analysis (without re‑running experiments), all per‑epoch metrics from every run are persisted immediately after training finishes.

The following artifacts are written:

  • Per‑run JSON (directory: metrics/mlp/)
    File name pattern: {run_id}.json where run_id = {model_type}_bs{batch}_{optimizer}_lr{lr} (dots in the LR replaced by underscores).
    Contents include: model/architecture label, optimizer, batch size, learning rate, plus full epoch lists for: train_accuracy, train_loss, val_accuracy, val_loss.
  • Global detailed CSV: metrics/mlp/runs_detailed.csv
    A tidy, row‑wise log with one row per epoch across all runs. Columns:
    run_id, epoch, optimizer, batch_size, learning_rate, train_acc, train_loss, val_acc, val_loss.

During training we maintain dictionaries keyed by (batch_size, learning_rate) to hold the epoch sequences for each optimizer. This avoids recomputation and lets us generate plots after all runs finish.

For each optimizer (SGD, Adam) we store:

  • total_acc_train_*: List of training accuracy values per epoch.
  • total_loss_train_*: List of training loss values per epoch.
  • total_acc_validation_*: List of validation accuracy values per epoch.
  • total_loss_validation_*: List of validation loss values per epoch.

(“*” is either _sgd or _adam).

Each list length equals the number of completed epochs for that run (may be < EPOCHS if early stopping triggered). These structures feed the plotting utilities and are also passed to the persistence function right after a run concludes.

Data Integrity Notes:

  • Persist is atomic per run: JSON then CSV append. If interrupted mid‑experiment, completed runs remain intact.
  • Re‑running the same configuration will overwrite the JSON for that run_id but append additional rows to the CSV (you can later filter duplicates by keeping only the max epoch per run_id).
  • Learning rates are string‑formatted in file names (with . swapped to _) to ensure filesystem safety.

Quick Recap

  1. Train model with a (batch size, optimizer, learning rate) triple.
  2. Collect epoch curves in memory.
  3. Save JSON snapshot + append epoch rows to the global CSV.
  4. Use saved data later for plotting, model selection, reporting.
Code 
# Ensure metrics directory exists
os.makedirs("metrics/mlp", exist_ok=True)

def persist_run_metrics(model_type:str, optimizer_name:str, batch_size:int, lr:float,
                         acc_train, loss_train, acc_val, loss_val):
    """Persist per-epoch metrics for a single run to disk.
    Creates/updates:
      - JSON file with full metric lists
      - CSV row-wise log (one row per epoch)
    """
    run_id = f"{model_type}_bs{batch_size}_{optimizer_name}_lr{lr}".replace('.','_')
    base_dir = Path("metrics/mlp")
    base_dir.mkdir(parents=True, exist_ok=True)
    # JSON aggregate
    json_path = base_dir / f"{run_id}.json"
    payload = {
        "model": model_type,
        "optimizer": optimizer_name,
        "batch_size": batch_size,
        "learning_rate": lr,
        "train_accuracy": acc_train,
        "train_loss": loss_train,
        "val_accuracy": acc_val,
        "val_loss": loss_val,
    }
    with open(json_path, 'w', encoding='utf-8') as f:
        json.dump(payload, f, indent=2)
    # CSV per-epoch (append)
    csv_path = base_dir / "runs_detailed.csv"
    file_exists = csv_path.exists()
    with open(csv_path, 'a', newline='', encoding='utf-8') as f:
        writer = csv.writer(f)
        if not file_exists:
            writer.writerow(["run_id","epoch","optimizer","batch_size","learning_rate","train_acc","train_loss","val_acc","val_loss"])
        for epoch_idx, (ta, tl, va, vl) in enumerate(zip(acc_train, loss_train, acc_val, loss_val), start=1):
            writer.writerow([run_id, epoch_idx, optimizer_name, batch_size, lr, ta, tl, va, vl])
    print(f"Metrics persisted for {run_id} -> {json_path}")
Code 
total_acc_train_sgd = {}
total_acc_validation_sgd = {}
total_loss_train_sgd = {}
total_loss_validation_sgd = {}

total_acc_train_adam = {}
total_acc_validation_adam = {}
total_loss_train_adam = {}
total_loss_validation_adam = {}

ARCHITECTURE_IDX = 2 # Change to select different architecture

for bs in BATCH_SIZE:
    print(f"\nTraining with batch size: {bs}")
    dataloaders = dataloaders_dict[bs]
    run_idx = 1
    for lr in LEARNING_RATE:
        # Adam run
        print(f"  Run {run_idx}: Adam optimizer with lr={lr}")
        model_adam = MLPBaseline(input_dim=INPUT_DIM, num_classes=NUM_CLASSES, hidden_layers=architectures[ARCHITECTURE_IDX]["hidden_layers"], hidden_units=architectures[ARCHITECTURE_IDX]["hidden_units"]).to(device)
        adam_opt = torch.optim.Adam(model_adam.parameters(), lr=lr, weight_decay=WEIGHT_DECAY)
        (
            acc_train_adam,
            loss_train_adam,
            acc_val_adam,
            loss_val_adam,
        ) = train_model(
            model_adam,
            dataloaders,
            adam_opt,
            criterion,
            epochs=EPOCHS,
            patience=PATIENCE,
            save_path=f"checkpoints/mlp/{architectures[ARCHITECTURE_IDX]['name']}_{bs}_adam_{lr}.pth",
        )
        total_acc_train_adam[(bs, lr)] = acc_train_adam
        total_loss_train_adam[(bs, lr)] = loss_train_adam
        total_acc_validation_adam[(bs, lr)] = acc_val_adam
        total_loss_validation_adam[(bs, lr)] = loss_val_adam
        persist_run_metrics(architectures[ARCHITECTURE_IDX]["name"], "adam", bs, lr, acc_train_adam, loss_train_adam, acc_val_adam, loss_val_adam)

        # SGD run
        print(f"  Run {run_idx}: SGD optimizer with lr={lr}, momentum={MOMENTUM}")
        model_sgd = MLPBaseline(input_dim=INPUT_DIM, num_classes=NUM_CLASSES, hidden_layers=architectures[ARCHITECTURE_IDX]["hidden_layers"], hidden_units=architectures[ARCHITECTURE_IDX]["hidden_units"]).to(device)
        sgd_opt = torch.optim.SGD(model_sgd.parameters(), lr=lr, momentum=MOMENTUM, weight_decay=WEIGHT_DECAY)
        (
            acc_train_sgd,
            loss_train_sgd,
            acc_val_sgd,
            loss_val_sgd,
        ) = train_model(
            model_sgd,
            dataloaders,
            sgd_opt,
            criterion,
            epochs=EPOCHS,
            patience=PATIENCE,
            save_path=f"checkpoints/mlp/{architectures[ARCHITECTURE_IDX]['name']}_{bs}_sgd_{lr}.pth",
        )
        total_acc_train_sgd[(bs, lr)] = acc_train_sgd
        total_loss_train_sgd[(bs, lr)] = loss_train_sgd
        total_acc_validation_sgd[(bs, lr)] = acc_val_sgd
        total_loss_validation_sgd[(bs, lr)] = loss_val_sgd
        persist_run_metrics(architectures[ARCHITECTURE_IDX]["name"], "sgd", bs, lr, acc_train_sgd, loss_train_sgd, acc_val_sgd, loss_val_sgd)

        run_idx += 1

Training with batch size: 32
  Run 1: Adam optimizer with lr=0.01
Epoch 01 | Train Loss: 2.0887 | Train Acc: 0.3137 | Val Loss: 1.4094 | Val Acc: 0.5480
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.01.pth
Saved new best checkpoint (epoch 1, val_acc=0.5480)
------------------------------------------------------------------------------------------
Epoch 02 | Train Loss: 1.8337 | Train Acc: 0.3857 | Val Loss: 1.3075 | Val Acc: 0.5613
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.01.pth
Saved new best checkpoint (epoch 2, val_acc=0.5613)
------------------------------------------------------------------------------------------
Epoch 03 | Train Loss: 1.8072 | Train Acc: 0.3948 | Val Loss: 1.2636 | Val Acc: 0.5933
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.01.pth
Saved new best checkpoint (epoch 3, val_acc=0.5933)
------------------------------------------------------------------------------------------
Epoch 04 | Train Loss: 1.7888 | Train Acc: 0.4088 | Val Loss: 1.2740 | Val Acc: 0.5760
------------------------------------------------------------------------------------------
Epoch 05 | Train Loss: 1.8066 | Train Acc: 0.3991 | Val Loss: 1.3464 | Val Acc: 0.5647
------------------------------------------------------------------------------------------
Epoch 06 | Train Loss: 1.7944 | Train Acc: 0.4042 | Val Loss: 1.3256 | Val Acc: 0.5653
------------------------------------------------------------------------------------------
Epoch 07 | Train Loss: 1.8026 | Train Acc: 0.4031 | Val Loss: 1.2968 | Val Acc: 0.5800
------------------------------------------------------------------------------------------
Epoch 08 | Train Loss: 1.7912 | Train Acc: 0.3991 | Val Loss: 1.2705 | Val Acc: 0.5927
------------------------------------------------------------------------------------------
Early stopping at epoch 8. Best val acc: 0.5933 (epoch 3)
Training finished. Best val acc: 0.5933 (epoch 3)
==========================================================================================
Metrics persisted for mlp_3x512-256-128_bs32_adam_lr0_01 -> metrics\mlp\mlp_3x512-256-128_bs32_adam_lr0_01.json
  Run 1: SGD optimizer with lr=0.01, momentum=0.9
Epoch 01 | Train Loss: 2.1563 | Train Acc: 0.2812 | Val Loss: 1.3876 | Val Acc: 0.5173
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 1, val_acc=0.5173)
------------------------------------------------------------------------------------------
Epoch 02 | Train Loss: 1.7428 | Train Acc: 0.4112 | Val Loss: 1.1035 | Val Acc: 0.6320
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 2, val_acc=0.6320)
------------------------------------------------------------------------------------------
Epoch 03 | Train Loss: 1.5708 | Train Acc: 0.4669 | Val Loss: 0.9541 | Val Acc: 0.6827
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 3, val_acc=0.6827)
------------------------------------------------------------------------------------------
Epoch 04 | Train Loss: 1.4747 | Train Acc: 0.5003 | Val Loss: 0.8670 | Val Acc: 0.7247
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 4, val_acc=0.7247)
------------------------------------------------------------------------------------------
Epoch 05 | Train Loss: 1.3849 | Train Acc: 0.5301 | Val Loss: 0.7693 | Val Acc: 0.7553
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 5, val_acc=0.7553)
------------------------------------------------------------------------------------------
Epoch 06 | Train Loss: 1.3105 | Train Acc: 0.5514 | Val Loss: 0.7289 | Val Acc: 0.7720
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 6, val_acc=0.7720)
------------------------------------------------------------------------------------------
Epoch 07 | Train Loss: 1.2674 | Train Acc: 0.5701 | Val Loss: 0.6767 | Val Acc: 0.7807
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 7, val_acc=0.7807)
------------------------------------------------------------------------------------------
Epoch 08 | Train Loss: 1.2469 | Train Acc: 0.5774 | Val Loss: 0.6378 | Val Acc: 0.7993
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 8, val_acc=0.7993)
------------------------------------------------------------------------------------------
Epoch 09 | Train Loss: 1.1901 | Train Acc: 0.5984 | Val Loss: 0.6326 | Val Acc: 0.7933
------------------------------------------------------------------------------------------
Epoch 10 | Train Loss: 1.1548 | Train Acc: 0.6041 | Val Loss: 0.5802 | Val Acc: 0.8200
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 10, val_acc=0.8200)
------------------------------------------------------------------------------------------
Epoch 11 | Train Loss: 1.1307 | Train Acc: 0.6158 | Val Loss: 0.5570 | Val Acc: 0.8207
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 11, val_acc=0.8207)
------------------------------------------------------------------------------------------
Epoch 12 | Train Loss: 1.1058 | Train Acc: 0.6206 | Val Loss: 0.5461 | Val Acc: 0.8327
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 12, val_acc=0.8327)
------------------------------------------------------------------------------------------
Epoch 13 | Train Loss: 1.0748 | Train Acc: 0.6349 | Val Loss: 0.5163 | Val Acc: 0.8307
------------------------------------------------------------------------------------------
Epoch 14 | Train Loss: 1.0400 | Train Acc: 0.6454 | Val Loss: 0.4957 | Val Acc: 0.8427
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 14, val_acc=0.8427)
------------------------------------------------------------------------------------------
Epoch 15 | Train Loss: 1.0242 | Train Acc: 0.6498 | Val Loss: 0.4880 | Val Acc: 0.8433
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 15, val_acc=0.8433)
------------------------------------------------------------------------------------------
Epoch 16 | Train Loss: 1.0215 | Train Acc: 0.6534 | Val Loss: 0.4966 | Val Acc: 0.8293
------------------------------------------------------------------------------------------
Epoch 17 | Train Loss: 1.0104 | Train Acc: 0.6565 | Val Loss: 0.4623 | Val Acc: 0.8460
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 17, val_acc=0.8460)
------------------------------------------------------------------------------------------
Epoch 18 | Train Loss: 0.9738 | Train Acc: 0.6619 | Val Loss: 0.4508 | Val Acc: 0.8587
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 18, val_acc=0.8587)
------------------------------------------------------------------------------------------
Epoch 19 | Train Loss: 0.9693 | Train Acc: 0.6687 | Val Loss: 0.4387 | Val Acc: 0.8580
------------------------------------------------------------------------------------------
Epoch 20 | Train Loss: 0.9520 | Train Acc: 0.6757 | Val Loss: 0.4231 | Val Acc: 0.8673
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 20, val_acc=0.8673)
------------------------------------------------------------------------------------------
Epoch 21 | Train Loss: 0.9500 | Train Acc: 0.6757 | Val Loss: 0.3999 | Val Acc: 0.8773
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 21, val_acc=0.8773)
------------------------------------------------------------------------------------------
Epoch 22 | Train Loss: 0.9267 | Train Acc: 0.6912 | Val Loss: 0.4058 | Val Acc: 0.8667
------------------------------------------------------------------------------------------
Epoch 23 | Train Loss: 0.9323 | Train Acc: 0.6819 | Val Loss: 0.3933 | Val Acc: 0.8820
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 23, val_acc=0.8820)
------------------------------------------------------------------------------------------
Epoch 24 | Train Loss: 0.9226 | Train Acc: 0.6870 | Val Loss: 0.3901 | Val Acc: 0.8873
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 24, val_acc=0.8873)
------------------------------------------------------------------------------------------
Epoch 25 | Train Loss: 0.8936 | Train Acc: 0.6960 | Val Loss: 0.3828 | Val Acc: 0.8900
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 25, val_acc=0.8900)
------------------------------------------------------------------------------------------
Epoch 26 | Train Loss: 0.8996 | Train Acc: 0.6946 | Val Loss: 0.3864 | Val Acc: 0.8727
------------------------------------------------------------------------------------------
Epoch 27 | Train Loss: 0.8939 | Train Acc: 0.6979 | Val Loss: 0.3832 | Val Acc: 0.8727
------------------------------------------------------------------------------------------
Epoch 28 | Train Loss: 0.8966 | Train Acc: 0.6955 | Val Loss: 0.3706 | Val Acc: 0.8893
------------------------------------------------------------------------------------------
Epoch 29 | Train Loss: 0.8842 | Train Acc: 0.7008 | Val Loss: 0.3578 | Val Acc: 0.8993
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.01.pth
Saved new best checkpoint (epoch 29, val_acc=0.8993)
------------------------------------------------------------------------------------------
Epoch 30 | Train Loss: 0.8753 | Train Acc: 0.7007 | Val Loss: 0.3582 | Val Acc: 0.8973
------------------------------------------------------------------------------------------
Epoch 31 | Train Loss: 0.8803 | Train Acc: 0.7026 | Val Loss: 0.3624 | Val Acc: 0.8873
------------------------------------------------------------------------------------------
Epoch 32 | Train Loss: 0.8580 | Train Acc: 0.7092 | Val Loss: 0.3460 | Val Acc: 0.8980
------------------------------------------------------------------------------------------
Epoch 33 | Train Loss: 0.8746 | Train Acc: 0.7033 | Val Loss: 0.3858 | Val Acc: 0.8847
------------------------------------------------------------------------------------------
Epoch 34 | Train Loss: 0.8719 | Train Acc: 0.7083 | Val Loss: 0.3559 | Val Acc: 0.8820
------------------------------------------------------------------------------------------
Early stopping at epoch 34. Best val acc: 0.8993 (epoch 29)
Training finished. Best val acc: 0.8993 (epoch 29)
==========================================================================================
Metrics persisted for mlp_3x512-256-128_bs32_sgd_lr0_01 -> metrics\mlp\mlp_3x512-256-128_bs32_sgd_lr0_01.json
  Run 2: Adam optimizer with lr=0.001
Epoch 01 | Train Loss: 2.1616 | Train Acc: 0.2975 | Val Loss: 1.4338 | Val Acc: 0.5507
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 1, val_acc=0.5507)
------------------------------------------------------------------------------------------
Epoch 02 | Train Loss: 1.6872 | Train Acc: 0.4481 | Val Loss: 1.0624 | Val Acc: 0.6693
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 2, val_acc=0.6693)
------------------------------------------------------------------------------------------
Epoch 03 | Train Loss: 1.5022 | Train Acc: 0.5022 | Val Loss: 0.9072 | Val Acc: 0.7313
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 3, val_acc=0.7313)
------------------------------------------------------------------------------------------
Epoch 04 | Train Loss: 1.4032 | Train Acc: 0.5289 | Val Loss: 0.8150 | Val Acc: 0.7507
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 4, val_acc=0.7507)
------------------------------------------------------------------------------------------
Epoch 05 | Train Loss: 1.2977 | Train Acc: 0.5636 | Val Loss: 0.7337 | Val Acc: 0.7913
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 5, val_acc=0.7913)
------------------------------------------------------------------------------------------
Epoch 06 | Train Loss: 1.2610 | Train Acc: 0.5791 | Val Loss: 0.6884 | Val Acc: 0.7860
------------------------------------------------------------------------------------------
Epoch 07 | Train Loss: 1.2174 | Train Acc: 0.5929 | Val Loss: 0.6581 | Val Acc: 0.7987
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 7, val_acc=0.7987)
------------------------------------------------------------------------------------------
Epoch 08 | Train Loss: 1.1831 | Train Acc: 0.6004 | Val Loss: 0.6143 | Val Acc: 0.8080
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 8, val_acc=0.8080)
------------------------------------------------------------------------------------------
Epoch 09 | Train Loss: 1.1284 | Train Acc: 0.6188 | Val Loss: 0.6023 | Val Acc: 0.8140
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 9, val_acc=0.8140)
------------------------------------------------------------------------------------------
Epoch 10 | Train Loss: 1.1071 | Train Acc: 0.6300 | Val Loss: 0.5616 | Val Acc: 0.8267
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 10, val_acc=0.8267)
------------------------------------------------------------------------------------------
Epoch 11 | Train Loss: 1.0941 | Train Acc: 0.6286 | Val Loss: 0.5410 | Val Acc: 0.8313
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 11, val_acc=0.8313)
------------------------------------------------------------------------------------------
Epoch 12 | Train Loss: 1.0563 | Train Acc: 0.6411 | Val Loss: 0.5172 | Val Acc: 0.8360
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 12, val_acc=0.8360)
------------------------------------------------------------------------------------------
Epoch 13 | Train Loss: 1.0329 | Train Acc: 0.6516 | Val Loss: 0.5142 | Val Acc: 0.8200
------------------------------------------------------------------------------------------
Epoch 14 | Train Loss: 1.0186 | Train Acc: 0.6572 | Val Loss: 0.4986 | Val Acc: 0.8533
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 14, val_acc=0.8533)
------------------------------------------------------------------------------------------
Epoch 15 | Train Loss: 1.0260 | Train Acc: 0.6538 | Val Loss: 0.4758 | Val Acc: 0.8573
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 15, val_acc=0.8573)
------------------------------------------------------------------------------------------
Epoch 16 | Train Loss: 1.0010 | Train Acc: 0.6629 | Val Loss: 0.4597 | Val Acc: 0.8520
------------------------------------------------------------------------------------------
Epoch 17 | Train Loss: 0.9994 | Train Acc: 0.6605 | Val Loss: 0.4658 | Val Acc: 0.8580
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 17, val_acc=0.8580)
------------------------------------------------------------------------------------------
Epoch 18 | Train Loss: 0.9633 | Train Acc: 0.6757 | Val Loss: 0.4586 | Val Acc: 0.8613
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 18, val_acc=0.8613)
------------------------------------------------------------------------------------------
Epoch 19 | Train Loss: 0.9813 | Train Acc: 0.6683 | Val Loss: 0.4406 | Val Acc: 0.8720
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 19, val_acc=0.8720)
------------------------------------------------------------------------------------------
Epoch 20 | Train Loss: 0.9641 | Train Acc: 0.6727 | Val Loss: 0.4508 | Val Acc: 0.8647
------------------------------------------------------------------------------------------
Epoch 21 | Train Loss: 0.9680 | Train Acc: 0.6773 | Val Loss: 0.4314 | Val Acc: 0.8720
------------------------------------------------------------------------------------------
Epoch 22 | Train Loss: 0.9429 | Train Acc: 0.6837 | Val Loss: 0.4258 | Val Acc: 0.8660
------------------------------------------------------------------------------------------
Epoch 23 | Train Loss: 0.9712 | Train Acc: 0.6743 | Val Loss: 0.3967 | Val Acc: 0.8873
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.001.pth
Saved new best checkpoint (epoch 23, val_acc=0.8873)
------------------------------------------------------------------------------------------
Epoch 24 | Train Loss: 0.9342 | Train Acc: 0.6829 | Val Loss: 0.4216 | Val Acc: 0.8713
------------------------------------------------------------------------------------------
Epoch 25 | Train Loss: 0.9398 | Train Acc: 0.6839 | Val Loss: 0.4074 | Val Acc: 0.8780
------------------------------------------------------------------------------------------
Epoch 26 | Train Loss: 0.9111 | Train Acc: 0.6947 | Val Loss: 0.3911 | Val Acc: 0.8760
------------------------------------------------------------------------------------------
Epoch 27 | Train Loss: 0.9322 | Train Acc: 0.6893 | Val Loss: 0.3958 | Val Acc: 0.8833
------------------------------------------------------------------------------------------
Epoch 28 | Train Loss: 0.9193 | Train Acc: 0.6899 | Val Loss: 0.3950 | Val Acc: 0.8813
------------------------------------------------------------------------------------------
Early stopping at epoch 28. Best val acc: 0.8873 (epoch 23)
Training finished. Best val acc: 0.8873 (epoch 23)
==========================================================================================
Metrics persisted for mlp_3x512-256-128_bs32_adam_lr0_001 -> metrics\mlp\mlp_3x512-256-128_bs32_adam_lr0_001.json
  Run 2: SGD optimizer with lr=0.001, momentum=0.9
Epoch 01 | Train Loss: 2.5802 | Train Acc: 0.1522 | Val Loss: 2.1878 | Val Acc: 0.3220
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 1, val_acc=0.3220)
------------------------------------------------------------------------------------------
Epoch 02 | Train Loss: 2.2551 | Train Acc: 0.2634 | Val Loss: 1.9174 | Val Acc: 0.4180
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 2, val_acc=0.4180)
------------------------------------------------------------------------------------------
Epoch 03 | Train Loss: 2.0661 | Train Acc: 0.3172 | Val Loss: 1.6591 | Val Acc: 0.4920
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 3, val_acc=0.4920)
------------------------------------------------------------------------------------------
Epoch 04 | Train Loss: 1.9238 | Train Acc: 0.3674 | Val Loss: 1.4818 | Val Acc: 0.5453
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 4, val_acc=0.5453)
------------------------------------------------------------------------------------------
Epoch 05 | Train Loss: 1.7968 | Train Acc: 0.4055 | Val Loss: 1.3247 | Val Acc: 0.5907
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 5, val_acc=0.5907)
------------------------------------------------------------------------------------------
Epoch 06 | Train Loss: 1.7149 | Train Acc: 0.4324 | Val Loss: 1.1918 | Val Acc: 0.6413
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 6, val_acc=0.6413)
------------------------------------------------------------------------------------------
Epoch 07 | Train Loss: 1.6401 | Train Acc: 0.4510 | Val Loss: 1.1171 | Val Acc: 0.6573
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 7, val_acc=0.6573)
------------------------------------------------------------------------------------------
Epoch 08 | Train Loss: 1.5689 | Train Acc: 0.4761 | Val Loss: 1.0382 | Val Acc: 0.6927
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 8, val_acc=0.6927)
------------------------------------------------------------------------------------------
Epoch 09 | Train Loss: 1.5103 | Train Acc: 0.4960 | Val Loss: 0.9651 | Val Acc: 0.7160
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 9, val_acc=0.7160)
------------------------------------------------------------------------------------------
Epoch 10 | Train Loss: 1.4716 | Train Acc: 0.5037 | Val Loss: 0.9161 | Val Acc: 0.7227
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 10, val_acc=0.7227)
------------------------------------------------------------------------------------------
Epoch 11 | Train Loss: 1.4395 | Train Acc: 0.5132 | Val Loss: 0.8691 | Val Acc: 0.7447
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 11, val_acc=0.7447)
------------------------------------------------------------------------------------------
Epoch 12 | Train Loss: 1.4038 | Train Acc: 0.5240 | Val Loss: 0.8433 | Val Acc: 0.7467
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 12, val_acc=0.7467)
------------------------------------------------------------------------------------------
Epoch 13 | Train Loss: 1.3810 | Train Acc: 0.5373 | Val Loss: 0.8080 | Val Acc: 0.7540
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 13, val_acc=0.7540)
------------------------------------------------------------------------------------------
Epoch 14 | Train Loss: 1.3487 | Train Acc: 0.5445 | Val Loss: 0.7770 | Val Acc: 0.7667
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 14, val_acc=0.7667)
------------------------------------------------------------------------------------------
Epoch 15 | Train Loss: 1.3131 | Train Acc: 0.5599 | Val Loss: 0.7451 | Val Acc: 0.7653
------------------------------------------------------------------------------------------
Epoch 16 | Train Loss: 1.2804 | Train Acc: 0.5714 | Val Loss: 0.6975 | Val Acc: 0.7847
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 16, val_acc=0.7847)
------------------------------------------------------------------------------------------
Epoch 17 | Train Loss: 1.2706 | Train Acc: 0.5718 | Val Loss: 0.7022 | Val Acc: 0.7887
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 17, val_acc=0.7887)
------------------------------------------------------------------------------------------
Epoch 18 | Train Loss: 1.2486 | Train Acc: 0.5750 | Val Loss: 0.6683 | Val Acc: 0.8000
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 18, val_acc=0.8000)
------------------------------------------------------------------------------------------
Epoch 19 | Train Loss: 1.2324 | Train Acc: 0.5838 | Val Loss: 0.6682 | Val Acc: 0.8000
------------------------------------------------------------------------------------------
Epoch 20 | Train Loss: 1.2141 | Train Acc: 0.5873 | Val Loss: 0.6370 | Val Acc: 0.8107
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 20, val_acc=0.8107)
------------------------------------------------------------------------------------------
Epoch 21 | Train Loss: 1.2134 | Train Acc: 0.5893 | Val Loss: 0.6217 | Val Acc: 0.8047
------------------------------------------------------------------------------------------
Epoch 22 | Train Loss: 1.1769 | Train Acc: 0.6035 | Val Loss: 0.6089 | Val Acc: 0.8280
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 22, val_acc=0.8280)
------------------------------------------------------------------------------------------
Epoch 23 | Train Loss: 1.1660 | Train Acc: 0.6054 | Val Loss: 0.6048 | Val Acc: 0.8227
------------------------------------------------------------------------------------------
Epoch 24 | Train Loss: 1.1584 | Train Acc: 0.6092 | Val Loss: 0.5869 | Val Acc: 0.8253
------------------------------------------------------------------------------------------
Epoch 25 | Train Loss: 1.1411 | Train Acc: 0.6137 | Val Loss: 0.5733 | Val Acc: 0.8360
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 25, val_acc=0.8360)
------------------------------------------------------------------------------------------
Epoch 26 | Train Loss: 1.1303 | Train Acc: 0.6188 | Val Loss: 0.5483 | Val Acc: 0.8413
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 26, val_acc=0.8413)
------------------------------------------------------------------------------------------
Epoch 27 | Train Loss: 1.1090 | Train Acc: 0.6255 | Val Loss: 0.5623 | Val Acc: 0.8287
------------------------------------------------------------------------------------------
Epoch 28 | Train Loss: 1.1108 | Train Acc: 0.6242 | Val Loss: 0.5545 | Val Acc: 0.8307
------------------------------------------------------------------------------------------
Epoch 29 | Train Loss: 1.0964 | Train Acc: 0.6278 | Val Loss: 0.5347 | Val Acc: 0.8420
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 29, val_acc=0.8420)
------------------------------------------------------------------------------------------
Epoch 30 | Train Loss: 1.0842 | Train Acc: 0.6279 | Val Loss: 0.5177 | Val Acc: 0.8420
------------------------------------------------------------------------------------------
Epoch 31 | Train Loss: 1.0711 | Train Acc: 0.6382 | Val Loss: 0.5167 | Val Acc: 0.8547
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 31, val_acc=0.8547)
------------------------------------------------------------------------------------------
Epoch 32 | Train Loss: 1.0551 | Train Acc: 0.6431 | Val Loss: 0.5331 | Val Acc: 0.8280
------------------------------------------------------------------------------------------
Epoch 33 | Train Loss: 1.0455 | Train Acc: 0.6458 | Val Loss: 0.5099 | Val Acc: 0.8513
------------------------------------------------------------------------------------------
Epoch 34 | Train Loss: 1.0364 | Train Acc: 0.6507 | Val Loss: 0.5042 | Val Acc: 0.8553
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 34, val_acc=0.8553)
------------------------------------------------------------------------------------------
Epoch 35 | Train Loss: 1.0256 | Train Acc: 0.6502 | Val Loss: 0.4815 | Val Acc: 0.8560
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 35, val_acc=0.8560)
------------------------------------------------------------------------------------------
Epoch 36 | Train Loss: 1.0268 | Train Acc: 0.6510 | Val Loss: 0.4721 | Val Acc: 0.8580
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 36, val_acc=0.8580)
------------------------------------------------------------------------------------------
Epoch 37 | Train Loss: 1.0016 | Train Acc: 0.6659 | Val Loss: 0.4763 | Val Acc: 0.8567
------------------------------------------------------------------------------------------
Epoch 38 | Train Loss: 1.0063 | Train Acc: 0.6552 | Val Loss: 0.4626 | Val Acc: 0.8580
------------------------------------------------------------------------------------------
Epoch 39 | Train Loss: 0.9941 | Train Acc: 0.6611 | Val Loss: 0.4462 | Val Acc: 0.8713
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 39, val_acc=0.8713)
------------------------------------------------------------------------------------------
Epoch 40 | Train Loss: 0.9854 | Train Acc: 0.6630 | Val Loss: 0.4623 | Val Acc: 0.8500
------------------------------------------------------------------------------------------
Epoch 41 | Train Loss: 0.9862 | Train Acc: 0.6642 | Val Loss: 0.4502 | Val Acc: 0.8593
------------------------------------------------------------------------------------------
Epoch 42 | Train Loss: 0.9723 | Train Acc: 0.6730 | Val Loss: 0.4481 | Val Acc: 0.8593
------------------------------------------------------------------------------------------
Epoch 43 | Train Loss: 0.9744 | Train Acc: 0.6647 | Val Loss: 0.4407 | Val Acc: 0.8607
------------------------------------------------------------------------------------------
Epoch 44 | Train Loss: 0.9638 | Train Acc: 0.6696 | Val Loss: 0.4348 | Val Acc: 0.8720
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 44, val_acc=0.8720)
------------------------------------------------------------------------------------------
Epoch 45 | Train Loss: 0.9706 | Train Acc: 0.6718 | Val Loss: 0.4221 | Val Acc: 0.8780
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 45, val_acc=0.8780)
------------------------------------------------------------------------------------------
Epoch 46 | Train Loss: 0.9651 | Train Acc: 0.6730 | Val Loss: 0.4249 | Val Acc: 0.8713
------------------------------------------------------------------------------------------
Epoch 47 | Train Loss: 0.9482 | Train Acc: 0.6805 | Val Loss: 0.4318 | Val Acc: 0.8680
------------------------------------------------------------------------------------------
Epoch 48 | Train Loss: 0.9625 | Train Acc: 0.6754 | Val Loss: 0.4253 | Val Acc: 0.8713
------------------------------------------------------------------------------------------
Epoch 49 | Train Loss: 0.9328 | Train Acc: 0.6799 | Val Loss: 0.4184 | Val Acc: 0.8773
------------------------------------------------------------------------------------------
Epoch 50 | Train Loss: 0.9309 | Train Acc: 0.6863 | Val Loss: 0.4029 | Val Acc: 0.8827
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.001.pth
Saved new best checkpoint (epoch 50, val_acc=0.8827)
------------------------------------------------------------------------------------------
Training finished. Best val acc: 0.8827 (epoch 50)
==========================================================================================
Metrics persisted for mlp_3x512-256-128_bs32_sgd_lr0_001 -> metrics\mlp\mlp_3x512-256-128_bs32_sgd_lr0_001.json
  Run 3: Adam optimizer with lr=0.0005
Epoch 01 | Train Loss: 2.3217 | Train Acc: 0.2462 | Val Loss: 1.7169 | Val Acc: 0.5007
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 1, val_acc=0.5007)
------------------------------------------------------------------------------------------
Epoch 02 | Train Loss: 1.8489 | Train Acc: 0.3928 | Val Loss: 1.2487 | Val Acc: 0.6307
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 2, val_acc=0.6307)
------------------------------------------------------------------------------------------
Epoch 03 | Train Loss: 1.6055 | Train Acc: 0.4738 | Val Loss: 1.0419 | Val Acc: 0.6847
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 3, val_acc=0.6847)
------------------------------------------------------------------------------------------
Epoch 04 | Train Loss: 1.4739 | Train Acc: 0.5103 | Val Loss: 0.9081 | Val Acc: 0.7140
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 4, val_acc=0.7140)
------------------------------------------------------------------------------------------
Epoch 05 | Train Loss: 1.3705 | Train Acc: 0.5433 | Val Loss: 0.8315 | Val Acc: 0.7447
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 5, val_acc=0.7447)
------------------------------------------------------------------------------------------
Epoch 06 | Train Loss: 1.3103 | Train Acc: 0.5557 | Val Loss: 0.7322 | Val Acc: 0.7740
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 6, val_acc=0.7740)
------------------------------------------------------------------------------------------
Epoch 07 | Train Loss: 1.2598 | Train Acc: 0.5760 | Val Loss: 0.6794 | Val Acc: 0.7853
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 7, val_acc=0.7853)
------------------------------------------------------------------------------------------
Epoch 08 | Train Loss: 1.2080 | Train Acc: 0.5961 | Val Loss: 0.6390 | Val Acc: 0.8007
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 8, val_acc=0.8007)
------------------------------------------------------------------------------------------
Epoch 09 | Train Loss: 1.1686 | Train Acc: 0.6041 | Val Loss: 0.6161 | Val Acc: 0.8200
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 9, val_acc=0.8200)
------------------------------------------------------------------------------------------
Epoch 10 | Train Loss: 1.1364 | Train Acc: 0.6158 | Val Loss: 0.5899 | Val Acc: 0.8233
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 10, val_acc=0.8233)
------------------------------------------------------------------------------------------
Epoch 11 | Train Loss: 1.1046 | Train Acc: 0.6265 | Val Loss: 0.5547 | Val Acc: 0.8313
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 11, val_acc=0.8313)
------------------------------------------------------------------------------------------
Epoch 12 | Train Loss: 1.1006 | Train Acc: 0.6264 | Val Loss: 0.5510 | Val Acc: 0.8440
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 12, val_acc=0.8440)
------------------------------------------------------------------------------------------
Epoch 13 | Train Loss: 1.0514 | Train Acc: 0.6481 | Val Loss: 0.5188 | Val Acc: 0.8420
------------------------------------------------------------------------------------------
Epoch 14 | Train Loss: 1.0331 | Train Acc: 0.6511 | Val Loss: 0.4963 | Val Acc: 0.8560
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 14, val_acc=0.8560)
------------------------------------------------------------------------------------------
Epoch 15 | Train Loss: 1.0134 | Train Acc: 0.6541 | Val Loss: 0.5191 | Val Acc: 0.8480
------------------------------------------------------------------------------------------
Epoch 16 | Train Loss: 0.9998 | Train Acc: 0.6633 | Val Loss: 0.4843 | Val Acc: 0.8587
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 16, val_acc=0.8587)
------------------------------------------------------------------------------------------
Epoch 17 | Train Loss: 0.9700 | Train Acc: 0.6668 | Val Loss: 0.4606 | Val Acc: 0.8713
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 17, val_acc=0.8713)
------------------------------------------------------------------------------------------
Epoch 18 | Train Loss: 0.9735 | Train Acc: 0.6743 | Val Loss: 0.4585 | Val Acc: 0.8653
------------------------------------------------------------------------------------------
Epoch 19 | Train Loss: 0.9567 | Train Acc: 0.6726 | Val Loss: 0.4504 | Val Acc: 0.8653
------------------------------------------------------------------------------------------
Epoch 20 | Train Loss: 0.9399 | Train Acc: 0.6773 | Val Loss: 0.4315 | Val Acc: 0.8760
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 20, val_acc=0.8760)
------------------------------------------------------------------------------------------
Epoch 21 | Train Loss: 0.9478 | Train Acc: 0.6768 | Val Loss: 0.4449 | Val Acc: 0.8680
------------------------------------------------------------------------------------------
Epoch 22 | Train Loss: 0.9213 | Train Acc: 0.6906 | Val Loss: 0.4239 | Val Acc: 0.8720
------------------------------------------------------------------------------------------
Epoch 23 | Train Loss: 0.9169 | Train Acc: 0.6886 | Val Loss: 0.4136 | Val Acc: 0.8840
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 23, val_acc=0.8840)
------------------------------------------------------------------------------------------
Epoch 24 | Train Loss: 0.9072 | Train Acc: 0.6907 | Val Loss: 0.4026 | Val Acc: 0.8847
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 24, val_acc=0.8847)
------------------------------------------------------------------------------------------
Epoch 25 | Train Loss: 0.9144 | Train Acc: 0.6932 | Val Loss: 0.3984 | Val Acc: 0.8860
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 25, val_acc=0.8860)
------------------------------------------------------------------------------------------
Epoch 26 | Train Loss: 0.8955 | Train Acc: 0.7000 | Val Loss: 0.3921 | Val Acc: 0.8867
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 26, val_acc=0.8867)
------------------------------------------------------------------------------------------
Epoch 27 | Train Loss: 0.8799 | Train Acc: 0.6968 | Val Loss: 0.3916 | Val Acc: 0.8793
------------------------------------------------------------------------------------------
Epoch 28 | Train Loss: 0.8711 | Train Acc: 0.7047 | Val Loss: 0.3932 | Val Acc: 0.8787
------------------------------------------------------------------------------------------
Epoch 29 | Train Loss: 0.8672 | Train Acc: 0.7070 | Val Loss: 0.3782 | Val Acc: 0.8853
------------------------------------------------------------------------------------------
Epoch 30 | Train Loss: 0.8469 | Train Acc: 0.7164 | Val Loss: 0.3762 | Val Acc: 0.8733
------------------------------------------------------------------------------------------
Epoch 31 | Train Loss: 0.8607 | Train Acc: 0.7103 | Val Loss: 0.3590 | Val Acc: 0.8980
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 31, val_acc=0.8980)
------------------------------------------------------------------------------------------
Epoch 32 | Train Loss: 0.8418 | Train Acc: 0.7106 | Val Loss: 0.3682 | Val Acc: 0.8813
------------------------------------------------------------------------------------------
Epoch 33 | Train Loss: 0.8383 | Train Acc: 0.7168 | Val Loss: 0.3630 | Val Acc: 0.8880
------------------------------------------------------------------------------------------
Epoch 34 | Train Loss: 0.8392 | Train Acc: 0.7137 | Val Loss: 0.3479 | Val Acc: 0.8987
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 34, val_acc=0.8987)
------------------------------------------------------------------------------------------
Epoch 35 | Train Loss: 0.8304 | Train Acc: 0.7200 | Val Loss: 0.3487 | Val Acc: 0.8907
------------------------------------------------------------------------------------------
Epoch 36 | Train Loss: 0.8252 | Train Acc: 0.7210 | Val Loss: 0.3558 | Val Acc: 0.8973
------------------------------------------------------------------------------------------
Epoch 37 | Train Loss: 0.8192 | Train Acc: 0.7216 | Val Loss: 0.3567 | Val Acc: 0.8900
------------------------------------------------------------------------------------------
Epoch 38 | Train Loss: 0.8121 | Train Acc: 0.7274 | Val Loss: 0.3386 | Val Acc: 0.8960
------------------------------------------------------------------------------------------
Epoch 39 | Train Loss: 0.8142 | Train Acc: 0.7242 | Val Loss: 0.3265 | Val Acc: 0.9113
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 39, val_acc=0.9113)
------------------------------------------------------------------------------------------
Epoch 40 | Train Loss: 0.8019 | Train Acc: 0.7244 | Val Loss: 0.3198 | Val Acc: 0.9133
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 40, val_acc=0.9133)
------------------------------------------------------------------------------------------
Epoch 41 | Train Loss: 0.7992 | Train Acc: 0.7344 | Val Loss: 0.3281 | Val Acc: 0.9067
------------------------------------------------------------------------------------------
Epoch 42 | Train Loss: 0.8021 | Train Acc: 0.7247 | Val Loss: 0.3245 | Val Acc: 0.9093
------------------------------------------------------------------------------------------
Epoch 43 | Train Loss: 0.7950 | Train Acc: 0.7317 | Val Loss: 0.3357 | Val Acc: 0.8953
------------------------------------------------------------------------------------------
Epoch 44 | Train Loss: 0.8079 | Train Acc: 0.7305 | Val Loss: 0.3216 | Val Acc: 0.9073
------------------------------------------------------------------------------------------
Epoch 45 | Train Loss: 0.7876 | Train Acc: 0.7362 | Val Loss: 0.3170 | Val Acc: 0.9193
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_adam_0.0005.pth
Saved new best checkpoint (epoch 45, val_acc=0.9193)
------------------------------------------------------------------------------------------
Epoch 46 | Train Loss: 0.7883 | Train Acc: 0.7292 | Val Loss: 0.3245 | Val Acc: 0.9073
------------------------------------------------------------------------------------------
Epoch 47 | Train Loss: 0.7956 | Train Acc: 0.7276 | Val Loss: 0.3124 | Val Acc: 0.9133
------------------------------------------------------------------------------------------
Epoch 48 | Train Loss: 0.7730 | Train Acc: 0.7392 | Val Loss: 0.3073 | Val Acc: 0.9020
------------------------------------------------------------------------------------------
Epoch 49 | Train Loss: 0.7746 | Train Acc: 0.7425 | Val Loss: 0.2971 | Val Acc: 0.9107
------------------------------------------------------------------------------------------
Epoch 50 | Train Loss: 0.7816 | Train Acc: 0.7354 | Val Loss: 0.2974 | Val Acc: 0.9120
------------------------------------------------------------------------------------------
Early stopping at epoch 50. Best val acc: 0.9193 (epoch 45)
Training finished. Best val acc: 0.9193 (epoch 45)
==========================================================================================
Metrics persisted for mlp_3x512-256-128_bs32_adam_lr0_0005 -> metrics\mlp\mlp_3x512-256-128_bs32_adam_lr0_0005.json
  Run 3: SGD optimizer with lr=0.0005, momentum=0.9
Epoch 01 | Train Loss: 2.6741 | Train Acc: 0.1208 | Val Loss: 2.3573 | Val Acc: 0.3000
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 1, val_acc=0.3000)
------------------------------------------------------------------------------------------
Epoch 02 | Train Loss: 2.4269 | Train Acc: 0.2003 | Val Loss: 2.1609 | Val Acc: 0.3707
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 2, val_acc=0.3707)
------------------------------------------------------------------------------------------
Epoch 03 | Train Loss: 2.2767 | Train Acc: 0.2524 | Val Loss: 2.0074 | Val Acc: 0.4247
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 3, val_acc=0.4247)
------------------------------------------------------------------------------------------
Epoch 04 | Train Loss: 2.1732 | Train Acc: 0.2888 | Val Loss: 1.8432 | Val Acc: 0.4780
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 4, val_acc=0.4780)
------------------------------------------------------------------------------------------
Epoch 05 | Train Loss: 2.0829 | Train Acc: 0.3190 | Val Loss: 1.7355 | Val Acc: 0.4960
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 5, val_acc=0.4960)
------------------------------------------------------------------------------------------
Epoch 06 | Train Loss: 1.9877 | Train Acc: 0.3492 | Val Loss: 1.6074 | Val Acc: 0.5347
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 6, val_acc=0.5347)
------------------------------------------------------------------------------------------
Epoch 07 | Train Loss: 1.9162 | Train Acc: 0.3725 | Val Loss: 1.4957 | Val Acc: 0.5440
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 7, val_acc=0.5440)
------------------------------------------------------------------------------------------
Epoch 08 | Train Loss: 1.8443 | Train Acc: 0.3932 | Val Loss: 1.3891 | Val Acc: 0.5847
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 8, val_acc=0.5847)
------------------------------------------------------------------------------------------
Epoch 09 | Train Loss: 1.7773 | Train Acc: 0.4147 | Val Loss: 1.2938 | Val Acc: 0.6133
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 9, val_acc=0.6133)
------------------------------------------------------------------------------------------
Epoch 10 | Train Loss: 1.7118 | Train Acc: 0.4384 | Val Loss: 1.2427 | Val Acc: 0.6300
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 10, val_acc=0.6300)
------------------------------------------------------------------------------------------
Epoch 11 | Train Loss: 1.6688 | Train Acc: 0.4493 | Val Loss: 1.1861 | Val Acc: 0.6500
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 11, val_acc=0.6500)
------------------------------------------------------------------------------------------
Epoch 12 | Train Loss: 1.6179 | Train Acc: 0.4650 | Val Loss: 1.1086 | Val Acc: 0.6727
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 12, val_acc=0.6727)
------------------------------------------------------------------------------------------
Epoch 13 | Train Loss: 1.5762 | Train Acc: 0.4826 | Val Loss: 1.0624 | Val Acc: 0.6807
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 13, val_acc=0.6807)
------------------------------------------------------------------------------------------
Epoch 14 | Train Loss: 1.5501 | Train Acc: 0.4858 | Val Loss: 1.0224 | Val Acc: 0.7047
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 14, val_acc=0.7047)
------------------------------------------------------------------------------------------
Epoch 15 | Train Loss: 1.5220 | Train Acc: 0.4951 | Val Loss: 0.9726 | Val Acc: 0.7140
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 15, val_acc=0.7140)
------------------------------------------------------------------------------------------
Epoch 16 | Train Loss: 1.4823 | Train Acc: 0.5087 | Val Loss: 0.9296 | Val Acc: 0.7240
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 16, val_acc=0.7240)
------------------------------------------------------------------------------------------
Epoch 17 | Train Loss: 1.4622 | Train Acc: 0.5082 | Val Loss: 0.8982 | Val Acc: 0.7267
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 17, val_acc=0.7267)
------------------------------------------------------------------------------------------
Epoch 18 | Train Loss: 1.4418 | Train Acc: 0.5183 | Val Loss: 0.8568 | Val Acc: 0.7367
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 18, val_acc=0.7367)
------------------------------------------------------------------------------------------
Epoch 19 | Train Loss: 1.4122 | Train Acc: 0.5320 | Val Loss: 0.8465 | Val Acc: 0.7427
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 19, val_acc=0.7427)
------------------------------------------------------------------------------------------
Epoch 20 | Train Loss: 1.3957 | Train Acc: 0.5356 | Val Loss: 0.8237 | Val Acc: 0.7567
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 20, val_acc=0.7567)
------------------------------------------------------------------------------------------
Epoch 21 | Train Loss: 1.3689 | Train Acc: 0.5398 | Val Loss: 0.8059 | Val Acc: 0.7533
------------------------------------------------------------------------------------------
Epoch 22 | Train Loss: 1.3534 | Train Acc: 0.5413 | Val Loss: 0.7814 | Val Acc: 0.7647
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 22, val_acc=0.7647)
------------------------------------------------------------------------------------------
Epoch 23 | Train Loss: 1.3328 | Train Acc: 0.5553 | Val Loss: 0.7820 | Val Acc: 0.7653
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 23, val_acc=0.7653)
------------------------------------------------------------------------------------------
Epoch 24 | Train Loss: 1.3340 | Train Acc: 0.5509 | Val Loss: 0.7342 | Val Acc: 0.7827
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 24, val_acc=0.7827)
------------------------------------------------------------------------------------------
Epoch 25 | Train Loss: 1.3013 | Train Acc: 0.5590 | Val Loss: 0.7389 | Val Acc: 0.7733
------------------------------------------------------------------------------------------
Epoch 26 | Train Loss: 1.2988 | Train Acc: 0.5656 | Val Loss: 0.7288 | Val Acc: 0.7807
------------------------------------------------------------------------------------------
Epoch 27 | Train Loss: 1.2760 | Train Acc: 0.5713 | Val Loss: 0.6925 | Val Acc: 0.7840
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 27, val_acc=0.7840)
------------------------------------------------------------------------------------------
Epoch 28 | Train Loss: 1.2408 | Train Acc: 0.5788 | Val Loss: 0.6816 | Val Acc: 0.7973
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 28, val_acc=0.7973)
------------------------------------------------------------------------------------------
Epoch 29 | Train Loss: 1.2453 | Train Acc: 0.5789 | Val Loss: 0.6694 | Val Acc: 0.8087
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 29, val_acc=0.8087)
------------------------------------------------------------------------------------------
Epoch 30 | Train Loss: 1.2239 | Train Acc: 0.5901 | Val Loss: 0.6564 | Val Acc: 0.7993
------------------------------------------------------------------------------------------
Epoch 31 | Train Loss: 1.2136 | Train Acc: 0.5908 | Val Loss: 0.6513 | Val Acc: 0.7953
------------------------------------------------------------------------------------------
Epoch 32 | Train Loss: 1.2053 | Train Acc: 0.5978 | Val Loss: 0.6428 | Val Acc: 0.8033
------------------------------------------------------------------------------------------
Epoch 33 | Train Loss: 1.1979 | Train Acc: 0.5932 | Val Loss: 0.6236 | Val Acc: 0.8013
------------------------------------------------------------------------------------------
Epoch 34 | Train Loss: 1.1926 | Train Acc: 0.5956 | Val Loss: 0.6147 | Val Acc: 0.8120
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 34, val_acc=0.8120)
------------------------------------------------------------------------------------------
Epoch 35 | Train Loss: 1.1849 | Train Acc: 0.5992 | Val Loss: 0.6114 | Val Acc: 0.8100
------------------------------------------------------------------------------------------
Epoch 36 | Train Loss: 1.1602 | Train Acc: 0.6080 | Val Loss: 0.6054 | Val Acc: 0.8147
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 36, val_acc=0.8147)
------------------------------------------------------------------------------------------
Epoch 37 | Train Loss: 1.1685 | Train Acc: 0.6075 | Val Loss: 0.5910 | Val Acc: 0.8300
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 37, val_acc=0.8300)
------------------------------------------------------------------------------------------
Epoch 38 | Train Loss: 1.1467 | Train Acc: 0.6131 | Val Loss: 0.5979 | Val Acc: 0.8147
------------------------------------------------------------------------------------------
Epoch 39 | Train Loss: 1.1380 | Train Acc: 0.6120 | Val Loss: 0.5697 | Val Acc: 0.8280
------------------------------------------------------------------------------------------
Epoch 40 | Train Loss: 1.1390 | Train Acc: 0.6132 | Val Loss: 0.5713 | Val Acc: 0.8267
------------------------------------------------------------------------------------------
Epoch 41 | Train Loss: 1.1244 | Train Acc: 0.6181 | Val Loss: 0.5625 | Val Acc: 0.8227
------------------------------------------------------------------------------------------
Epoch 42 | Train Loss: 1.1198 | Train Acc: 0.6252 | Val Loss: 0.5576 | Val Acc: 0.8353
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 42, val_acc=0.8353)
------------------------------------------------------------------------------------------
Epoch 43 | Train Loss: 1.1025 | Train Acc: 0.6270 | Val Loss: 0.5535 | Val Acc: 0.8260
------------------------------------------------------------------------------------------
Epoch 44 | Train Loss: 1.0948 | Train Acc: 0.6255 | Val Loss: 0.5359 | Val Acc: 0.8333
------------------------------------------------------------------------------------------
Epoch 45 | Train Loss: 1.1072 | Train Acc: 0.6279 | Val Loss: 0.5201 | Val Acc: 0.8420
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 45, val_acc=0.8420)
------------------------------------------------------------------------------------------
Epoch 46 | Train Loss: 1.0896 | Train Acc: 0.6338 | Val Loss: 0.5209 | Val Acc: 0.8513
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 46, val_acc=0.8513)
------------------------------------------------------------------------------------------
Epoch 47 | Train Loss: 1.0793 | Train Acc: 0.6278 | Val Loss: 0.5100 | Val Acc: 0.8540
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
Saved new best checkpoint (epoch 47, val_acc=0.8540)
------------------------------------------------------------------------------------------
Epoch 48 | Train Loss: 1.0578 | Train Acc: 0.6418 | Val Loss: 0.5177 | Val Acc: 0.8433
------------------------------------------------------------------------------------------
Epoch 49 | Train Loss: 1.0667 | Train Acc: 0.6432 | Val Loss: 0.5063 | Val Acc: 0.8480
------------------------------------------------------------------------------------------
Epoch 50 | Train Loss: 1.0749 | Train Acc: 0.6370 | Val Loss: 0.5134 | Val Acc: 0.8527
------------------------------------------------------------------------------------------
Training finished. Best val acc: 0.8540 (epoch 47)
==========================================================================================
Metrics persisted for mlp_3x512-256-128_bs32_sgd_lr0_0005 -> metrics\mlp\mlp_3x512-256-128_bs32_sgd_lr0_0005.json

Training with batch size: 256
  Run 1: Adam optimizer with lr=0.01
Epoch 01 | Train Loss: 2.1007 | Train Acc: 0.3044 | Val Loss: 1.3211 | Val Acc: 0.5700
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.01.pth
Saved new best checkpoint (epoch 1, val_acc=0.5700)
------------------------------------------------------------------------------------------
Epoch 02 | Train Loss: 1.5496 | Train Acc: 0.4735 | Val Loss: 0.9792 | Val Acc: 0.6887
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.01.pth
Saved new best checkpoint (epoch 2, val_acc=0.6887)
------------------------------------------------------------------------------------------
Epoch 03 | Train Loss: 1.3921 | Train Acc: 0.5320 | Val Loss: 0.8935 | Val Acc: 0.6940
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.01.pth
Saved new best checkpoint (epoch 3, val_acc=0.6940)
------------------------------------------------------------------------------------------
Epoch 04 | Train Loss: 1.2793 | Train Acc: 0.5633 | Val Loss: 0.7717 | Val Acc: 0.7653
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.01.pth
Saved new best checkpoint (epoch 4, val_acc=0.7653)
------------------------------------------------------------------------------------------
Epoch 05 | Train Loss: 1.2260 | Train Acc: 0.5770 | Val Loss: 0.7448 | Val Acc: 0.7407
------------------------------------------------------------------------------------------
Epoch 06 | Train Loss: 1.1836 | Train Acc: 0.5972 | Val Loss: 0.7030 | Val Acc: 0.7687
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.01.pth
Saved new best checkpoint (epoch 6, val_acc=0.7687)
------------------------------------------------------------------------------------------
Epoch 07 | Train Loss: 1.1706 | Train Acc: 0.5977 | Val Loss: 0.6674 | Val Acc: 0.7833
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.01.pth
Saved new best checkpoint (epoch 7, val_acc=0.7833)
------------------------------------------------------------------------------------------
Epoch 08 | Train Loss: 1.1445 | Train Acc: 0.6086 | Val Loss: 0.6870 | Val Acc: 0.7740
------------------------------------------------------------------------------------------
Epoch 09 | Train Loss: 1.1550 | Train Acc: 0.6082 | Val Loss: 0.6640 | Val Acc: 0.7880
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.01.pth
Saved new best checkpoint (epoch 9, val_acc=0.7880)
------------------------------------------------------------------------------------------
Epoch 10 | Train Loss: 1.1316 | Train Acc: 0.6163 | Val Loss: 0.6589 | Val Acc: 0.7787
------------------------------------------------------------------------------------------
Epoch 11 | Train Loss: 1.1205 | Train Acc: 0.6162 | Val Loss: 0.6472 | Val Acc: 0.8053
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.01.pth
Saved new best checkpoint (epoch 11, val_acc=0.8053)
------------------------------------------------------------------------------------------
Epoch 12 | Train Loss: 1.1170 | Train Acc: 0.6187 | Val Loss: 0.6251 | Val Acc: 0.7773
------------------------------------------------------------------------------------------
Epoch 13 | Train Loss: 1.1063 | Train Acc: 0.6194 | Val Loss: 0.6365 | Val Acc: 0.7867
------------------------------------------------------------------------------------------
Epoch 14 | Train Loss: 1.0944 | Train Acc: 0.6235 | Val Loss: 0.6716 | Val Acc: 0.7847
------------------------------------------------------------------------------------------
Epoch 15 | Train Loss: 1.0881 | Train Acc: 0.6312 | Val Loss: 0.6009 | Val Acc: 0.8047
------------------------------------------------------------------------------------------
Epoch 16 | Train Loss: 1.1131 | Train Acc: 0.6184 | Val Loss: 0.6168 | Val Acc: 0.8033
------------------------------------------------------------------------------------------
Early stopping at epoch 16. Best val acc: 0.8053 (epoch 11)
Training finished. Best val acc: 0.8053 (epoch 11)
==========================================================================================
Metrics persisted for mlp_3x512-256-128_bs256_adam_lr0_01 -> metrics\mlp\mlp_3x512-256-128_bs256_adam_lr0_01.json
  Run 1: SGD optimizer with lr=0.01, momentum=0.9
Epoch 01 | Train Loss: 2.6199 | Train Acc: 0.1384 | Val Loss: 2.4216 | Val Acc: 0.2487
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 1, val_acc=0.2487)
------------------------------------------------------------------------------------------
Epoch 02 | Train Loss: 2.2463 | Train Acc: 0.2611 | Val Loss: 1.8971 | Val Acc: 0.4293
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 2, val_acc=0.4293)
------------------------------------------------------------------------------------------
Epoch 03 | Train Loss: 2.0200 | Train Acc: 0.3412 | Val Loss: 1.6358 | Val Acc: 0.4933
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 3, val_acc=0.4933)
------------------------------------------------------------------------------------------
Epoch 04 | Train Loss: 1.8434 | Train Acc: 0.3917 | Val Loss: 1.4125 | Val Acc: 0.5607
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 4, val_acc=0.5607)
------------------------------------------------------------------------------------------
Epoch 05 | Train Loss: 1.7048 | Train Acc: 0.4374 | Val Loss: 1.2427 | Val Acc: 0.6067
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 5, val_acc=0.6067)
------------------------------------------------------------------------------------------
Epoch 06 | Train Loss: 1.5896 | Train Acc: 0.4699 | Val Loss: 1.0810 | Val Acc: 0.6727
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 6, val_acc=0.6727)
------------------------------------------------------------------------------------------
Epoch 07 | Train Loss: 1.4951 | Train Acc: 0.5000 | Val Loss: 0.9945 | Val Acc: 0.6900
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 7, val_acc=0.6900)
------------------------------------------------------------------------------------------
Epoch 08 | Train Loss: 1.4244 | Train Acc: 0.5238 | Val Loss: 0.9234 | Val Acc: 0.7087
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 8, val_acc=0.7087)
------------------------------------------------------------------------------------------
Epoch 09 | Train Loss: 1.3503 | Train Acc: 0.5429 | Val Loss: 0.8704 | Val Acc: 0.7227
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 9, val_acc=0.7227)
------------------------------------------------------------------------------------------
Epoch 10 | Train Loss: 1.2953 | Train Acc: 0.5572 | Val Loss: 0.7984 | Val Acc: 0.7527
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 10, val_acc=0.7527)
------------------------------------------------------------------------------------------
Epoch 11 | Train Loss: 1.2611 | Train Acc: 0.5726 | Val Loss: 0.7612 | Val Acc: 0.7600
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 11, val_acc=0.7600)
------------------------------------------------------------------------------------------
Epoch 12 | Train Loss: 1.2415 | Train Acc: 0.5773 | Val Loss: 0.7136 | Val Acc: 0.7680
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 12, val_acc=0.7680)
------------------------------------------------------------------------------------------
Epoch 13 | Train Loss: 1.1903 | Train Acc: 0.5975 | Val Loss: 0.6853 | Val Acc: 0.7893
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 13, val_acc=0.7893)
------------------------------------------------------------------------------------------
Epoch 14 | Train Loss: 1.1562 | Train Acc: 0.6068 | Val Loss: 0.6638 | Val Acc: 0.7860
------------------------------------------------------------------------------------------
Epoch 15 | Train Loss: 1.1326 | Train Acc: 0.6053 | Val Loss: 0.6244 | Val Acc: 0.8080
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 15, val_acc=0.8080)
------------------------------------------------------------------------------------------
Epoch 16 | Train Loss: 1.1004 | Train Acc: 0.6220 | Val Loss: 0.6072 | Val Acc: 0.8107
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 16, val_acc=0.8107)
------------------------------------------------------------------------------------------
Epoch 17 | Train Loss: 1.0812 | Train Acc: 0.6332 | Val Loss: 0.5845 | Val Acc: 0.8120
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 17, val_acc=0.8120)
------------------------------------------------------------------------------------------
Epoch 18 | Train Loss: 1.0545 | Train Acc: 0.6392 | Val Loss: 0.5730 | Val Acc: 0.8060
------------------------------------------------------------------------------------------
Epoch 19 | Train Loss: 1.0380 | Train Acc: 0.6459 | Val Loss: 0.5533 | Val Acc: 0.8320
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 19, val_acc=0.8320)
------------------------------------------------------------------------------------------
Epoch 20 | Train Loss: 1.0080 | Train Acc: 0.6524 | Val Loss: 0.5340 | Val Acc: 0.8320
------------------------------------------------------------------------------------------
Epoch 21 | Train Loss: 1.0088 | Train Acc: 0.6494 | Val Loss: 0.5138 | Val Acc: 0.8433
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 21, val_acc=0.8433)
------------------------------------------------------------------------------------------
Epoch 22 | Train Loss: 0.9800 | Train Acc: 0.6635 | Val Loss: 0.5215 | Val Acc: 0.8347
------------------------------------------------------------------------------------------
Epoch 23 | Train Loss: 0.9688 | Train Acc: 0.6695 | Val Loss: 0.4933 | Val Acc: 0.8453
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 23, val_acc=0.8453)
------------------------------------------------------------------------------------------
Epoch 24 | Train Loss: 0.9426 | Train Acc: 0.6703 | Val Loss: 0.4845 | Val Acc: 0.8467
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 24, val_acc=0.8467)
------------------------------------------------------------------------------------------
Epoch 25 | Train Loss: 0.9500 | Train Acc: 0.6738 | Val Loss: 0.4758 | Val Acc: 0.8527
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 25, val_acc=0.8527)
------------------------------------------------------------------------------------------
Epoch 26 | Train Loss: 0.9208 | Train Acc: 0.6849 | Val Loss: 0.4718 | Val Acc: 0.8553
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 26, val_acc=0.8553)
------------------------------------------------------------------------------------------
Epoch 27 | Train Loss: 0.9295 | Train Acc: 0.6853 | Val Loss: 0.4553 | Val Acc: 0.8567
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 27, val_acc=0.8567)
------------------------------------------------------------------------------------------
Epoch 28 | Train Loss: 0.9100 | Train Acc: 0.6901 | Val Loss: 0.4472 | Val Acc: 0.8640
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 28, val_acc=0.8640)
------------------------------------------------------------------------------------------
Epoch 29 | Train Loss: 0.9045 | Train Acc: 0.6867 | Val Loss: 0.4389 | Val Acc: 0.8640
------------------------------------------------------------------------------------------
Epoch 30 | Train Loss: 0.8791 | Train Acc: 0.6938 | Val Loss: 0.4237 | Val Acc: 0.8727
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 30, val_acc=0.8727)
------------------------------------------------------------------------------------------
Epoch 31 | Train Loss: 0.8887 | Train Acc: 0.6913 | Val Loss: 0.4181 | Val Acc: 0.8753
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.01.pth
Saved new best checkpoint (epoch 31, val_acc=0.8753)
------------------------------------------------------------------------------------------
Epoch 32 | Train Loss: 0.8699 | Train Acc: 0.6980 | Val Loss: 0.4095 | Val Acc: 0.8747
------------------------------------------------------------------------------------------
Epoch 33 | Train Loss: 0.8546 | Train Acc: 0.7057 | Val Loss: 0.4081 | Val Acc: 0.8700
------------------------------------------------------------------------------------------
Epoch 34 | Train Loss: 0.8220 | Train Acc: 0.7155 | Val Loss: 0.4021 | Val Acc: 0.8740
------------------------------------------------------------------------------------------
Epoch 35 | Train Loss: 0.8380 | Train Acc: 0.7094 | Val Loss: 0.3873 | Val Acc: 0.8753
------------------------------------------------------------------------------------------
Epoch 36 | Train Loss: 0.8318 | Train Acc: 0.7124 | Val Loss: 0.3902 | Val Acc: 0.8700
------------------------------------------------------------------------------------------
Early stopping at epoch 36. Best val acc: 0.8753 (epoch 31)
Training finished. Best val acc: 0.8753 (epoch 31)
==========================================================================================
Metrics persisted for mlp_3x512-256-128_bs256_sgd_lr0_01 -> metrics\mlp\mlp_3x512-256-128_bs256_sgd_lr0_01.json
  Run 2: Adam optimizer with lr=0.001
Epoch 01 | Train Loss: 2.4014 | Train Acc: 0.2218 | Val Loss: 2.0952 | Val Acc: 0.4167
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 1, val_acc=0.4167)
------------------------------------------------------------------------------------------
Epoch 02 | Train Loss: 1.9583 | Train Acc: 0.3706 | Val Loss: 1.5083 | Val Acc: 0.5440
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 2, val_acc=0.5440)
------------------------------------------------------------------------------------------
Epoch 03 | Train Loss: 1.6769 | Train Acc: 0.4598 | Val Loss: 1.1906 | Val Acc: 0.6407
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 3, val_acc=0.6407)
------------------------------------------------------------------------------------------
Epoch 04 | Train Loss: 1.4837 | Train Acc: 0.5132 | Val Loss: 0.9940 | Val Acc: 0.6867
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 4, val_acc=0.6867)
------------------------------------------------------------------------------------------
Epoch 05 | Train Loss: 1.3391 | Train Acc: 0.5552 | Val Loss: 0.8557 | Val Acc: 0.7500
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 5, val_acc=0.7500)
------------------------------------------------------------------------------------------
Epoch 06 | Train Loss: 1.2388 | Train Acc: 0.5830 | Val Loss: 0.7525 | Val Acc: 0.7700
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 6, val_acc=0.7700)
------------------------------------------------------------------------------------------
Epoch 07 | Train Loss: 1.1781 | Train Acc: 0.6076 | Val Loss: 0.6716 | Val Acc: 0.7907
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 7, val_acc=0.7907)
------------------------------------------------------------------------------------------
Epoch 08 | Train Loss: 1.1170 | Train Acc: 0.6303 | Val Loss: 0.6218 | Val Acc: 0.8167
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 8, val_acc=0.8167)
------------------------------------------------------------------------------------------
Epoch 09 | Train Loss: 1.0693 | Train Acc: 0.6335 | Val Loss: 0.5792 | Val Acc: 0.8260
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 9, val_acc=0.8260)
------------------------------------------------------------------------------------------
Epoch 10 | Train Loss: 1.0228 | Train Acc: 0.6498 | Val Loss: 0.5649 | Val Acc: 0.8267
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 10, val_acc=0.8267)
------------------------------------------------------------------------------------------
Epoch 11 | Train Loss: 0.9815 | Train Acc: 0.6664 | Val Loss: 0.5196 | Val Acc: 0.8387
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 11, val_acc=0.8387)
------------------------------------------------------------------------------------------
Epoch 12 | Train Loss: 0.9606 | Train Acc: 0.6738 | Val Loss: 0.5076 | Val Acc: 0.8407
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 12, val_acc=0.8407)
------------------------------------------------------------------------------------------
Epoch 13 | Train Loss: 0.9418 | Train Acc: 0.6745 | Val Loss: 0.4982 | Val Acc: 0.8427
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 13, val_acc=0.8427)
------------------------------------------------------------------------------------------
Epoch 14 | Train Loss: 0.8896 | Train Acc: 0.6963 | Val Loss: 0.4607 | Val Acc: 0.8513
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 14, val_acc=0.8513)
------------------------------------------------------------------------------------------
Epoch 15 | Train Loss: 0.8839 | Train Acc: 0.6987 | Val Loss: 0.4580 | Val Acc: 0.8547
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 15, val_acc=0.8547)
------------------------------------------------------------------------------------------
Epoch 16 | Train Loss: 0.8647 | Train Acc: 0.7041 | Val Loss: 0.4385 | Val Acc: 0.8627
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 16, val_acc=0.8627)
------------------------------------------------------------------------------------------
Epoch 17 | Train Loss: 0.8537 | Train Acc: 0.7104 | Val Loss: 0.4341 | Val Acc: 0.8680
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 17, val_acc=0.8680)
------------------------------------------------------------------------------------------
Epoch 18 | Train Loss: 0.8244 | Train Acc: 0.7160 | Val Loss: 0.4180 | Val Acc: 0.8707
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 18, val_acc=0.8707)
------------------------------------------------------------------------------------------
Epoch 19 | Train Loss: 0.8214 | Train Acc: 0.7159 | Val Loss: 0.4078 | Val Acc: 0.8653
------------------------------------------------------------------------------------------
Epoch 20 | Train Loss: 0.7957 | Train Acc: 0.7285 | Val Loss: 0.3928 | Val Acc: 0.8773
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 20, val_acc=0.8773)
------------------------------------------------------------------------------------------
Epoch 21 | Train Loss: 0.7737 | Train Acc: 0.7342 | Val Loss: 0.3782 | Val Acc: 0.8867
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 21, val_acc=0.8867)
------------------------------------------------------------------------------------------
Epoch 22 | Train Loss: 0.7515 | Train Acc: 0.7381 | Val Loss: 0.3738 | Val Acc: 0.8807
------------------------------------------------------------------------------------------
Epoch 23 | Train Loss: 0.7527 | Train Acc: 0.7415 | Val Loss: 0.3646 | Val Acc: 0.8833
------------------------------------------------------------------------------------------
Epoch 24 | Train Loss: 0.7450 | Train Acc: 0.7428 | Val Loss: 0.3468 | Val Acc: 0.8867
------------------------------------------------------------------------------------------
Epoch 25 | Train Loss: 0.7174 | Train Acc: 0.7546 | Val Loss: 0.3397 | Val Acc: 0.8967
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 25, val_acc=0.8967)
------------------------------------------------------------------------------------------
Epoch 26 | Train Loss: 0.7169 | Train Acc: 0.7548 | Val Loss: 0.3338 | Val Acc: 0.8920
------------------------------------------------------------------------------------------
Epoch 27 | Train Loss: 0.7104 | Train Acc: 0.7534 | Val Loss: 0.3365 | Val Acc: 0.8933
------------------------------------------------------------------------------------------
Epoch 28 | Train Loss: 0.7031 | Train Acc: 0.7609 | Val Loss: 0.3185 | Val Acc: 0.9007
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 28, val_acc=0.9007)
------------------------------------------------------------------------------------------
Epoch 29 | Train Loss: 0.6893 | Train Acc: 0.7658 | Val Loss: 0.3186 | Val Acc: 0.9020
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 29, val_acc=0.9020)
------------------------------------------------------------------------------------------
Epoch 30 | Train Loss: 0.6790 | Train Acc: 0.7648 | Val Loss: 0.3077 | Val Acc: 0.9000
------------------------------------------------------------------------------------------
Epoch 31 | Train Loss: 0.6824 | Train Acc: 0.7660 | Val Loss: 0.3032 | Val Acc: 0.9013
------------------------------------------------------------------------------------------
Epoch 32 | Train Loss: 0.6622 | Train Acc: 0.7719 | Val Loss: 0.3020 | Val Acc: 0.9013
------------------------------------------------------------------------------------------
Epoch 33 | Train Loss: 0.6587 | Train Acc: 0.7752 | Val Loss: 0.2914 | Val Acc: 0.9080
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 33, val_acc=0.9080)
------------------------------------------------------------------------------------------
Epoch 34 | Train Loss: 0.6500 | Train Acc: 0.7725 | Val Loss: 0.2886 | Val Acc: 0.9133
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 34, val_acc=0.9133)
------------------------------------------------------------------------------------------
Epoch 35 | Train Loss: 0.6381 | Train Acc: 0.7795 | Val Loss: 0.2857 | Val Acc: 0.9127
------------------------------------------------------------------------------------------
Epoch 36 | Train Loss: 0.6369 | Train Acc: 0.7806 | Val Loss: 0.2779 | Val Acc: 0.9100
------------------------------------------------------------------------------------------
Epoch 37 | Train Loss: 0.6239 | Train Acc: 0.7863 | Val Loss: 0.2737 | Val Acc: 0.9180
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 37, val_acc=0.9180)
------------------------------------------------------------------------------------------
Epoch 38 | Train Loss: 0.6310 | Train Acc: 0.7826 | Val Loss: 0.2749 | Val Acc: 0.9093
------------------------------------------------------------------------------------------
Epoch 39 | Train Loss: 0.6266 | Train Acc: 0.7827 | Val Loss: 0.2748 | Val Acc: 0.9113
------------------------------------------------------------------------------------------
Epoch 40 | Train Loss: 0.5977 | Train Acc: 0.7931 | Val Loss: 0.2596 | Val Acc: 0.9260
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 40, val_acc=0.9260)
------------------------------------------------------------------------------------------
Epoch 41 | Train Loss: 0.6025 | Train Acc: 0.7949 | Val Loss: 0.2637 | Val Acc: 0.9187
------------------------------------------------------------------------------------------
Epoch 42 | Train Loss: 0.6030 | Train Acc: 0.7972 | Val Loss: 0.2530 | Val Acc: 0.9267
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 42, val_acc=0.9267)
------------------------------------------------------------------------------------------
Epoch 43 | Train Loss: 0.5958 | Train Acc: 0.7969 | Val Loss: 0.2500 | Val Acc: 0.9247
------------------------------------------------------------------------------------------
Epoch 44 | Train Loss: 0.5944 | Train Acc: 0.8017 | Val Loss: 0.2497 | Val Acc: 0.9273
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.001.pth
Saved new best checkpoint (epoch 44, val_acc=0.9273)
------------------------------------------------------------------------------------------
Epoch 45 | Train Loss: 0.5937 | Train Acc: 0.7993 | Val Loss: 0.2575 | Val Acc: 0.9133
------------------------------------------------------------------------------------------
Epoch 46 | Train Loss: 0.5881 | Train Acc: 0.8019 | Val Loss: 0.2483 | Val Acc: 0.9200
------------------------------------------------------------------------------------------
Epoch 47 | Train Loss: 0.5810 | Train Acc: 0.8037 | Val Loss: 0.2521 | Val Acc: 0.9213
------------------------------------------------------------------------------------------
Epoch 48 | Train Loss: 0.5715 | Train Acc: 0.8040 | Val Loss: 0.2414 | Val Acc: 0.9200
------------------------------------------------------------------------------------------
Epoch 49 | Train Loss: 0.5759 | Train Acc: 0.8053 | Val Loss: 0.2389 | Val Acc: 0.9240
------------------------------------------------------------------------------------------
Early stopping at epoch 49. Best val acc: 0.9273 (epoch 44)
Training finished. Best val acc: 0.9273 (epoch 44)
==========================================================================================
Metrics persisted for mlp_3x512-256-128_bs256_adam_lr0_001 -> metrics\mlp\mlp_3x512-256-128_bs256_adam_lr0_001.json
  Run 2: SGD optimizer with lr=0.001, momentum=0.9
Epoch 01 | Train Loss: 2.7941 | Train Acc: 0.0752 | Val Loss: 2.6940 | Val Acc: 0.0973
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 1, val_acc=0.0973)
------------------------------------------------------------------------------------------
Epoch 02 | Train Loss: 2.7003 | Train Acc: 0.1071 | Val Loss: 2.5375 | Val Acc: 0.2427
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 2, val_acc=0.2427)
------------------------------------------------------------------------------------------
Epoch 03 | Train Loss: 2.6153 | Train Acc: 0.1363 | Val Loss: 2.4462 | Val Acc: 0.2840
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 3, val_acc=0.2840)
------------------------------------------------------------------------------------------
Epoch 04 | Train Loss: 2.5414 | Train Acc: 0.1634 | Val Loss: 2.3716 | Val Acc: 0.2947
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 4, val_acc=0.2947)
------------------------------------------------------------------------------------------
Epoch 05 | Train Loss: 2.4832 | Train Acc: 0.1855 | Val Loss: 2.3109 | Val Acc: 0.3027
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 5, val_acc=0.3027)
------------------------------------------------------------------------------------------
Epoch 06 | Train Loss: 2.4287 | Train Acc: 0.2066 | Val Loss: 2.2572 | Val Acc: 0.3173
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 6, val_acc=0.3173)
------------------------------------------------------------------------------------------
Epoch 07 | Train Loss: 2.3788 | Train Acc: 0.2160 | Val Loss: 2.2095 | Val Acc: 0.3280
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 7, val_acc=0.3280)
------------------------------------------------------------------------------------------
Epoch 08 | Train Loss: 2.3438 | Train Acc: 0.2387 | Val Loss: 2.1605 | Val Acc: 0.3473
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 8, val_acc=0.3473)
------------------------------------------------------------------------------------------
Epoch 09 | Train Loss: 2.3116 | Train Acc: 0.2452 | Val Loss: 2.1156 | Val Acc: 0.3620
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 9, val_acc=0.3620)
------------------------------------------------------------------------------------------
Epoch 10 | Train Loss: 2.2646 | Train Acc: 0.2598 | Val Loss: 2.0691 | Val Acc: 0.3813
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 10, val_acc=0.3813)
------------------------------------------------------------------------------------------
Epoch 11 | Train Loss: 2.2342 | Train Acc: 0.2757 | Val Loss: 2.0281 | Val Acc: 0.3947
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 11, val_acc=0.3947)
------------------------------------------------------------------------------------------
Epoch 12 | Train Loss: 2.1980 | Train Acc: 0.2865 | Val Loss: 1.9811 | Val Acc: 0.4013
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 12, val_acc=0.4013)
------------------------------------------------------------------------------------------
Epoch 13 | Train Loss: 2.1733 | Train Acc: 0.2908 | Val Loss: 1.9393 | Val Acc: 0.4167
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 13, val_acc=0.4167)
------------------------------------------------------------------------------------------
Epoch 14 | Train Loss: 2.1449 | Train Acc: 0.2994 | Val Loss: 1.9078 | Val Acc: 0.4333
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 14, val_acc=0.4333)
------------------------------------------------------------------------------------------
Epoch 15 | Train Loss: 2.1118 | Train Acc: 0.3132 | Val Loss: 1.8657 | Val Acc: 0.4367
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 15, val_acc=0.4367)
------------------------------------------------------------------------------------------
Epoch 16 | Train Loss: 2.0750 | Train Acc: 0.3252 | Val Loss: 1.8204 | Val Acc: 0.4560
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 16, val_acc=0.4560)
------------------------------------------------------------------------------------------
Epoch 17 | Train Loss: 2.0384 | Train Acc: 0.3378 | Val Loss: 1.7884 | Val Acc: 0.4600
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 17, val_acc=0.4600)
------------------------------------------------------------------------------------------
Epoch 18 | Train Loss: 2.0098 | Train Acc: 0.3458 | Val Loss: 1.7448 | Val Acc: 0.4753
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 18, val_acc=0.4753)
------------------------------------------------------------------------------------------
Epoch 19 | Train Loss: 1.9972 | Train Acc: 0.3525 | Val Loss: 1.7092 | Val Acc: 0.4873
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 19, val_acc=0.4873)
------------------------------------------------------------------------------------------
Epoch 20 | Train Loss: 1.9745 | Train Acc: 0.3604 | Val Loss: 1.6725 | Val Acc: 0.4967
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 20, val_acc=0.4967)
------------------------------------------------------------------------------------------
Epoch 21 | Train Loss: 1.9350 | Train Acc: 0.3688 | Val Loss: 1.6452 | Val Acc: 0.5013
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 21, val_acc=0.5013)
------------------------------------------------------------------------------------------
Epoch 22 | Train Loss: 1.9100 | Train Acc: 0.3785 | Val Loss: 1.6071 | Val Acc: 0.5133
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 22, val_acc=0.5133)
------------------------------------------------------------------------------------------
Epoch 23 | Train Loss: 1.8957 | Train Acc: 0.3839 | Val Loss: 1.5729 | Val Acc: 0.5280
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 23, val_acc=0.5280)
------------------------------------------------------------------------------------------
Epoch 24 | Train Loss: 1.8612 | Train Acc: 0.3959 | Val Loss: 1.5468 | Val Acc: 0.5333
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 24, val_acc=0.5333)
------------------------------------------------------------------------------------------
Epoch 25 | Train Loss: 1.8503 | Train Acc: 0.3947 | Val Loss: 1.5142 | Val Acc: 0.5447
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 25, val_acc=0.5447)
------------------------------------------------------------------------------------------
Epoch 26 | Train Loss: 1.8031 | Train Acc: 0.4111 | Val Loss: 1.4853 | Val Acc: 0.5493
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 26, val_acc=0.5493)
------------------------------------------------------------------------------------------
Epoch 27 | Train Loss: 1.8025 | Train Acc: 0.4123 | Val Loss: 1.4507 | Val Acc: 0.5560
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 27, val_acc=0.5560)
------------------------------------------------------------------------------------------
Epoch 28 | Train Loss: 1.7729 | Train Acc: 0.4220 | Val Loss: 1.4260 | Val Acc: 0.5720
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 28, val_acc=0.5720)
------------------------------------------------------------------------------------------
Epoch 29 | Train Loss: 1.7574 | Train Acc: 0.4249 | Val Loss: 1.3943 | Val Acc: 0.5760
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 29, val_acc=0.5760)
------------------------------------------------------------------------------------------
Epoch 30 | Train Loss: 1.7296 | Train Acc: 0.4346 | Val Loss: 1.3683 | Val Acc: 0.5880
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 30, val_acc=0.5880)
------------------------------------------------------------------------------------------
Epoch 31 | Train Loss: 1.7130 | Train Acc: 0.4381 | Val Loss: 1.3305 | Val Acc: 0.6027
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 31, val_acc=0.6027)
------------------------------------------------------------------------------------------
Epoch 32 | Train Loss: 1.6948 | Train Acc: 0.4409 | Val Loss: 1.3036 | Val Acc: 0.6147
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 32, val_acc=0.6147)
------------------------------------------------------------------------------------------
Epoch 33 | Train Loss: 1.6631 | Train Acc: 0.4487 | Val Loss: 1.2874 | Val Acc: 0.6233
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 33, val_acc=0.6233)
------------------------------------------------------------------------------------------
Epoch 34 | Train Loss: 1.6537 | Train Acc: 0.4552 | Val Loss: 1.2562 | Val Acc: 0.6280
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 34, val_acc=0.6280)
------------------------------------------------------------------------------------------
Epoch 35 | Train Loss: 1.6380 | Train Acc: 0.4622 | Val Loss: 1.2417 | Val Acc: 0.6300
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 35, val_acc=0.6300)
------------------------------------------------------------------------------------------
Epoch 36 | Train Loss: 1.6201 | Train Acc: 0.4679 | Val Loss: 1.2153 | Val Acc: 0.6393
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 36, val_acc=0.6393)
------------------------------------------------------------------------------------------
Epoch 37 | Train Loss: 1.5925 | Train Acc: 0.4778 | Val Loss: 1.1887 | Val Acc: 0.6553
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 37, val_acc=0.6553)
------------------------------------------------------------------------------------------
Epoch 38 | Train Loss: 1.5820 | Train Acc: 0.4765 | Val Loss: 1.1753 | Val Acc: 0.6627
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 38, val_acc=0.6627)
------------------------------------------------------------------------------------------
Epoch 39 | Train Loss: 1.5715 | Train Acc: 0.4816 | Val Loss: 1.1535 | Val Acc: 0.6667
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 39, val_acc=0.6667)
------------------------------------------------------------------------------------------
Epoch 40 | Train Loss: 1.5466 | Train Acc: 0.4855 | Val Loss: 1.1281 | Val Acc: 0.6653
------------------------------------------------------------------------------------------
Epoch 41 | Train Loss: 1.5445 | Train Acc: 0.4869 | Val Loss: 1.1103 | Val Acc: 0.6700
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 41, val_acc=0.6700)
------------------------------------------------------------------------------------------
Epoch 42 | Train Loss: 1.5089 | Train Acc: 0.5008 | Val Loss: 1.0927 | Val Acc: 0.6793
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 42, val_acc=0.6793)
------------------------------------------------------------------------------------------
Epoch 43 | Train Loss: 1.4989 | Train Acc: 0.5062 | Val Loss: 1.0733 | Val Acc: 0.6840
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 43, val_acc=0.6840)
------------------------------------------------------------------------------------------
Epoch 44 | Train Loss: 1.4926 | Train Acc: 0.5075 | Val Loss: 1.0535 | Val Acc: 0.6907
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 44, val_acc=0.6907)
------------------------------------------------------------------------------------------
Epoch 45 | Train Loss: 1.4728 | Train Acc: 0.5143 | Val Loss: 1.0401 | Val Acc: 0.7007
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 45, val_acc=0.7007)
------------------------------------------------------------------------------------------
Epoch 46 | Train Loss: 1.4584 | Train Acc: 0.5177 | Val Loss: 1.0188 | Val Acc: 0.7067
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 46, val_acc=0.7067)
------------------------------------------------------------------------------------------
Epoch 47 | Train Loss: 1.4400 | Train Acc: 0.5200 | Val Loss: 1.0059 | Val Acc: 0.7073
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 47, val_acc=0.7073)
------------------------------------------------------------------------------------------
Epoch 48 | Train Loss: 1.4385 | Train Acc: 0.5281 | Val Loss: 0.9913 | Val Acc: 0.7127
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 48, val_acc=0.7127)
------------------------------------------------------------------------------------------
Epoch 49 | Train Loss: 1.4165 | Train Acc: 0.5244 | Val Loss: 0.9727 | Val Acc: 0.7127
------------------------------------------------------------------------------------------
Epoch 50 | Train Loss: 1.4005 | Train Acc: 0.5322 | Val Loss: 0.9636 | Val Acc: 0.7233
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.001.pth
Saved new best checkpoint (epoch 50, val_acc=0.7233)
------------------------------------------------------------------------------------------
Training finished. Best val acc: 0.7233 (epoch 50)
==========================================================================================
Metrics persisted for mlp_3x512-256-128_bs256_sgd_lr0_001 -> metrics\mlp\mlp_3x512-256-128_bs256_sgd_lr0_001.json
  Run 3: Adam optimizer with lr=0.0005
Epoch 01 | Train Loss: 2.5757 | Train Acc: 0.1575 | Val Loss: 2.4076 | Val Acc: 0.2860
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 1, val_acc=0.2860)
------------------------------------------------------------------------------------------
Epoch 02 | Train Loss: 2.2411 | Train Acc: 0.2827 | Val Loss: 1.8876 | Val Acc: 0.4427
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 2, val_acc=0.4427)
------------------------------------------------------------------------------------------
Epoch 03 | Train Loss: 2.0103 | Train Acc: 0.3657 | Val Loss: 1.6191 | Val Acc: 0.5280
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 3, val_acc=0.5280)
------------------------------------------------------------------------------------------
Epoch 04 | Train Loss: 1.7871 | Train Acc: 0.4317 | Val Loss: 1.3520 | Val Acc: 0.6067
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 4, val_acc=0.6067)
------------------------------------------------------------------------------------------
Epoch 05 | Train Loss: 1.6157 | Train Acc: 0.4858 | Val Loss: 1.1678 | Val Acc: 0.6613
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 5, val_acc=0.6613)
------------------------------------------------------------------------------------------
Epoch 06 | Train Loss: 1.4772 | Train Acc: 0.5197 | Val Loss: 1.0078 | Val Acc: 0.7033
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 6, val_acc=0.7033)
------------------------------------------------------------------------------------------
Epoch 07 | Train Loss: 1.3789 | Train Acc: 0.5463 | Val Loss: 0.8991 | Val Acc: 0.7440
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 7, val_acc=0.7440)
------------------------------------------------------------------------------------------
Epoch 08 | Train Loss: 1.2980 | Train Acc: 0.5706 | Val Loss: 0.8043 | Val Acc: 0.7693
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 8, val_acc=0.7693)
------------------------------------------------------------------------------------------
Epoch 09 | Train Loss: 1.2210 | Train Acc: 0.5931 | Val Loss: 0.7454 | Val Acc: 0.7867
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 9, val_acc=0.7867)
------------------------------------------------------------------------------------------
Epoch 10 | Train Loss: 1.1763 | Train Acc: 0.6128 | Val Loss: 0.7010 | Val Acc: 0.7860
------------------------------------------------------------------------------------------
Epoch 11 | Train Loss: 1.1253 | Train Acc: 0.6162 | Val Loss: 0.6602 | Val Acc: 0.8020
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 11, val_acc=0.8020)
------------------------------------------------------------------------------------------
Epoch 12 | Train Loss: 1.0798 | Train Acc: 0.6387 | Val Loss: 0.6162 | Val Acc: 0.8147
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 12, val_acc=0.8147)
------------------------------------------------------------------------------------------
Epoch 13 | Train Loss: 1.0550 | Train Acc: 0.6415 | Val Loss: 0.5826 | Val Acc: 0.8300
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 13, val_acc=0.8300)
------------------------------------------------------------------------------------------
Epoch 14 | Train Loss: 1.0300 | Train Acc: 0.6523 | Val Loss: 0.5551 | Val Acc: 0.8340
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 14, val_acc=0.8340)
------------------------------------------------------------------------------------------
Epoch 15 | Train Loss: 0.9970 | Train Acc: 0.6638 | Val Loss: 0.5265 | Val Acc: 0.8373
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 15, val_acc=0.8373)
------------------------------------------------------------------------------------------
Epoch 16 | Train Loss: 0.9584 | Train Acc: 0.6744 | Val Loss: 0.5074 | Val Acc: 0.8433
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 16, val_acc=0.8433)
------------------------------------------------------------------------------------------
Epoch 17 | Train Loss: 0.9500 | Train Acc: 0.6757 | Val Loss: 0.4904 | Val Acc: 0.8547
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 17, val_acc=0.8547)
------------------------------------------------------------------------------------------
Epoch 18 | Train Loss: 0.9207 | Train Acc: 0.6818 | Val Loss: 0.4678 | Val Acc: 0.8587
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 18, val_acc=0.8587)
------------------------------------------------------------------------------------------
Epoch 19 | Train Loss: 0.9133 | Train Acc: 0.6893 | Val Loss: 0.4599 | Val Acc: 0.8620
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 19, val_acc=0.8620)
------------------------------------------------------------------------------------------
Epoch 20 | Train Loss: 0.8771 | Train Acc: 0.7052 | Val Loss: 0.4578 | Val Acc: 0.8587
------------------------------------------------------------------------------------------
Epoch 21 | Train Loss: 0.8765 | Train Acc: 0.7003 | Val Loss: 0.4377 | Val Acc: 0.8667
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 21, val_acc=0.8667)
------------------------------------------------------------------------------------------
Epoch 22 | Train Loss: 0.8532 | Train Acc: 0.7052 | Val Loss: 0.4287 | Val Acc: 0.8707
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 22, val_acc=0.8707)
------------------------------------------------------------------------------------------
Epoch 23 | Train Loss: 0.8401 | Train Acc: 0.7109 | Val Loss: 0.4095 | Val Acc: 0.8853
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 23, val_acc=0.8853)
------------------------------------------------------------------------------------------
Epoch 24 | Train Loss: 0.8219 | Train Acc: 0.7174 | Val Loss: 0.4036 | Val Acc: 0.8840
------------------------------------------------------------------------------------------
Epoch 25 | Train Loss: 0.8106 | Train Acc: 0.7242 | Val Loss: 0.3863 | Val Acc: 0.8867
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 25, val_acc=0.8867)
------------------------------------------------------------------------------------------
Epoch 26 | Train Loss: 0.7848 | Train Acc: 0.7296 | Val Loss: 0.3868 | Val Acc: 0.8780
------------------------------------------------------------------------------------------
Epoch 27 | Train Loss: 0.7761 | Train Acc: 0.7322 | Val Loss: 0.3815 | Val Acc: 0.8753
------------------------------------------------------------------------------------------
Epoch 28 | Train Loss: 0.7645 | Train Acc: 0.7366 | Val Loss: 0.3567 | Val Acc: 0.8887
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 28, val_acc=0.8887)
------------------------------------------------------------------------------------------
Epoch 29 | Train Loss: 0.7606 | Train Acc: 0.7410 | Val Loss: 0.3620 | Val Acc: 0.8913
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 29, val_acc=0.8913)
------------------------------------------------------------------------------------------
Epoch 30 | Train Loss: 0.7603 | Train Acc: 0.7429 | Val Loss: 0.3440 | Val Acc: 0.8953
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 30, val_acc=0.8953)
------------------------------------------------------------------------------------------
Epoch 31 | Train Loss: 0.7385 | Train Acc: 0.7495 | Val Loss: 0.3404 | Val Acc: 0.8960
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 31, val_acc=0.8960)
------------------------------------------------------------------------------------------
Epoch 32 | Train Loss: 0.7342 | Train Acc: 0.7530 | Val Loss: 0.3313 | Val Acc: 0.9093
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 32, val_acc=0.9093)
------------------------------------------------------------------------------------------
Epoch 33 | Train Loss: 0.7462 | Train Acc: 0.7518 | Val Loss: 0.3218 | Val Acc: 0.9020
------------------------------------------------------------------------------------------
Epoch 34 | Train Loss: 0.7086 | Train Acc: 0.7547 | Val Loss: 0.3232 | Val Acc: 0.8967
------------------------------------------------------------------------------------------
Epoch 35 | Train Loss: 0.7068 | Train Acc: 0.7610 | Val Loss: 0.3178 | Val Acc: 0.9000
------------------------------------------------------------------------------------------
Epoch 36 | Train Loss: 0.6895 | Train Acc: 0.7658 | Val Loss: 0.3029 | Val Acc: 0.9160
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 36, val_acc=0.9160)
------------------------------------------------------------------------------------------
Epoch 37 | Train Loss: 0.6715 | Train Acc: 0.7705 | Val Loss: 0.3132 | Val Acc: 0.9067
------------------------------------------------------------------------------------------
Epoch 38 | Train Loss: 0.6985 | Train Acc: 0.7618 | Val Loss: 0.3010 | Val Acc: 0.9100
------------------------------------------------------------------------------------------
Epoch 39 | Train Loss: 0.6938 | Train Acc: 0.7556 | Val Loss: 0.2960 | Val Acc: 0.9180
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 39, val_acc=0.9180)
------------------------------------------------------------------------------------------
Epoch 40 | Train Loss: 0.6707 | Train Acc: 0.7689 | Val Loss: 0.2915 | Val Acc: 0.9153
------------------------------------------------------------------------------------------
Epoch 41 | Train Loss: 0.6702 | Train Acc: 0.7722 | Val Loss: 0.2860 | Val Acc: 0.9120
------------------------------------------------------------------------------------------
Epoch 42 | Train Loss: 0.6555 | Train Acc: 0.7760 | Val Loss: 0.2788 | Val Acc: 0.9167
------------------------------------------------------------------------------------------
Epoch 43 | Train Loss: 0.6549 | Train Acc: 0.7768 | Val Loss: 0.2778 | Val Acc: 0.9160
------------------------------------------------------------------------------------------
Epoch 44 | Train Loss: 0.6535 | Train Acc: 0.7780 | Val Loss: 0.2790 | Val Acc: 0.9193
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 44, val_acc=0.9193)
------------------------------------------------------------------------------------------
Epoch 45 | Train Loss: 0.6296 | Train Acc: 0.7853 | Val Loss: 0.2693 | Val Acc: 0.9193
------------------------------------------------------------------------------------------
Epoch 46 | Train Loss: 0.6340 | Train Acc: 0.7832 | Val Loss: 0.2700 | Val Acc: 0.9220
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 46, val_acc=0.9220)
------------------------------------------------------------------------------------------
Epoch 47 | Train Loss: 0.6342 | Train Acc: 0.7802 | Val Loss: 0.2694 | Val Acc: 0.9220
------------------------------------------------------------------------------------------
Epoch 48 | Train Loss: 0.6176 | Train Acc: 0.7911 | Val Loss: 0.2609 | Val Acc: 0.9227
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_adam_0.0005.pth
Saved new best checkpoint (epoch 48, val_acc=0.9227)
------------------------------------------------------------------------------------------
Epoch 49 | Train Loss: 0.6253 | Train Acc: 0.7851 | Val Loss: 0.2654 | Val Acc: 0.9193
------------------------------------------------------------------------------------------
Epoch 50 | Train Loss: 0.6071 | Train Acc: 0.7919 | Val Loss: 0.2623 | Val Acc: 0.9227
------------------------------------------------------------------------------------------
Training finished. Best val acc: 0.9227 (epoch 48)
==========================================================================================
Metrics persisted for mlp_3x512-256-128_bs256_adam_lr0_0005 -> metrics\mlp\mlp_3x512-256-128_bs256_adam_lr0_0005.json
  Run 3: SGD optimizer with lr=0.0005, momentum=0.9
Epoch 01 | Train Loss: 2.8199 | Train Acc: 0.0700 | Val Loss: 2.7017 | Val Acc: 0.0693
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 1, val_acc=0.0693)
------------------------------------------------------------------------------------------
Epoch 02 | Train Loss: 2.7600 | Train Acc: 0.0905 | Val Loss: 2.6085 | Val Acc: 0.1720
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 2, val_acc=0.1720)
------------------------------------------------------------------------------------------
Epoch 03 | Train Loss: 2.7047 | Train Acc: 0.1051 | Val Loss: 2.5499 | Val Acc: 0.2153
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 3, val_acc=0.2153)
------------------------------------------------------------------------------------------
Epoch 04 | Train Loss: 2.6530 | Train Acc: 0.1255 | Val Loss: 2.5016 | Val Acc: 0.2440
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 4, val_acc=0.2440)
------------------------------------------------------------------------------------------
Epoch 05 | Train Loss: 2.6215 | Train Acc: 0.1380 | Val Loss: 2.4589 | Val Acc: 0.2660
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 5, val_acc=0.2660)
------------------------------------------------------------------------------------------
Epoch 06 | Train Loss: 2.5722 | Train Acc: 0.1597 | Val Loss: 2.4189 | Val Acc: 0.2820
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 6, val_acc=0.2820)
------------------------------------------------------------------------------------------
Epoch 07 | Train Loss: 2.5486 | Train Acc: 0.1619 | Val Loss: 2.3846 | Val Acc: 0.2887
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 7, val_acc=0.2887)
------------------------------------------------------------------------------------------
Epoch 08 | Train Loss: 2.5129 | Train Acc: 0.1727 | Val Loss: 2.3501 | Val Acc: 0.2953
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 8, val_acc=0.2953)
------------------------------------------------------------------------------------------
Epoch 09 | Train Loss: 2.4868 | Train Acc: 0.1874 | Val Loss: 2.3227 | Val Acc: 0.2993
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 9, val_acc=0.2993)
------------------------------------------------------------------------------------------
Epoch 10 | Train Loss: 2.4663 | Train Acc: 0.1962 | Val Loss: 2.2998 | Val Acc: 0.3073
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 10, val_acc=0.3073)
------------------------------------------------------------------------------------------
Epoch 11 | Train Loss: 2.4323 | Train Acc: 0.2039 | Val Loss: 2.2704 | Val Acc: 0.3160
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 11, val_acc=0.3160)
------------------------------------------------------------------------------------------
Epoch 12 | Train Loss: 2.4178 | Train Acc: 0.2052 | Val Loss: 2.2424 | Val Acc: 0.3293
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 12, val_acc=0.3293)
------------------------------------------------------------------------------------------
Epoch 13 | Train Loss: 2.3932 | Train Acc: 0.2187 | Val Loss: 2.2177 | Val Acc: 0.3400
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 13, val_acc=0.3400)
------------------------------------------------------------------------------------------
Epoch 14 | Train Loss: 2.3604 | Train Acc: 0.2277 | Val Loss: 2.1982 | Val Acc: 0.3473
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 14, val_acc=0.3473)
------------------------------------------------------------------------------------------
Epoch 15 | Train Loss: 2.3568 | Train Acc: 0.2334 | Val Loss: 2.1724 | Val Acc: 0.3600
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 15, val_acc=0.3600)
------------------------------------------------------------------------------------------
Epoch 16 | Train Loss: 2.3217 | Train Acc: 0.2444 | Val Loss: 2.1493 | Val Acc: 0.3713
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 16, val_acc=0.3713)
------------------------------------------------------------------------------------------
Epoch 17 | Train Loss: 2.3127 | Train Acc: 0.2466 | Val Loss: 2.1227 | Val Acc: 0.3800
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 17, val_acc=0.3800)
------------------------------------------------------------------------------------------
Epoch 18 | Train Loss: 2.2928 | Train Acc: 0.2563 | Val Loss: 2.0994 | Val Acc: 0.3900
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 18, val_acc=0.3900)
------------------------------------------------------------------------------------------
Epoch 19 | Train Loss: 2.2749 | Train Acc: 0.2576 | Val Loss: 2.0734 | Val Acc: 0.3927
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 19, val_acc=0.3927)
------------------------------------------------------------------------------------------
Epoch 20 | Train Loss: 2.2568 | Train Acc: 0.2698 | Val Loss: 2.0528 | Val Acc: 0.4013
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 20, val_acc=0.4013)
------------------------------------------------------------------------------------------
Epoch 21 | Train Loss: 2.2416 | Train Acc: 0.2757 | Val Loss: 2.0322 | Val Acc: 0.4027
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 21, val_acc=0.4027)
------------------------------------------------------------------------------------------
Epoch 22 | Train Loss: 2.2152 | Train Acc: 0.2819 | Val Loss: 2.0104 | Val Acc: 0.4113
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 22, val_acc=0.4113)
------------------------------------------------------------------------------------------
Epoch 23 | Train Loss: 2.2065 | Train Acc: 0.2810 | Val Loss: 1.9944 | Val Acc: 0.4187
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 23, val_acc=0.4187)
------------------------------------------------------------------------------------------
Epoch 24 | Train Loss: 2.1923 | Train Acc: 0.2922 | Val Loss: 1.9666 | Val Acc: 0.4267
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 24, val_acc=0.4267)
------------------------------------------------------------------------------------------
Epoch 25 | Train Loss: 2.1762 | Train Acc: 0.2986 | Val Loss: 1.9471 | Val Acc: 0.4333
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 25, val_acc=0.4333)
------------------------------------------------------------------------------------------
Epoch 26 | Train Loss: 2.1690 | Train Acc: 0.2941 | Val Loss: 1.9261 | Val Acc: 0.4393
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 26, val_acc=0.4393)
------------------------------------------------------------------------------------------
Epoch 27 | Train Loss: 2.1347 | Train Acc: 0.3099 | Val Loss: 1.9048 | Val Acc: 0.4460
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 27, val_acc=0.4460)
------------------------------------------------------------------------------------------
Epoch 28 | Train Loss: 2.1303 | Train Acc: 0.3085 | Val Loss: 1.8826 | Val Acc: 0.4540
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 28, val_acc=0.4540)
------------------------------------------------------------------------------------------
Epoch 29 | Train Loss: 2.1032 | Train Acc: 0.3203 | Val Loss: 1.8685 | Val Acc: 0.4560
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 29, val_acc=0.4560)
------------------------------------------------------------------------------------------
Epoch 30 | Train Loss: 2.1007 | Train Acc: 0.3120 | Val Loss: 1.8447 | Val Acc: 0.4600
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 30, val_acc=0.4600)
------------------------------------------------------------------------------------------
Epoch 31 | Train Loss: 2.0773 | Train Acc: 0.3261 | Val Loss: 1.8212 | Val Acc: 0.4700
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 31, val_acc=0.4700)
------------------------------------------------------------------------------------------
Epoch 32 | Train Loss: 2.0667 | Train Acc: 0.3370 | Val Loss: 1.8045 | Val Acc: 0.4727
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 32, val_acc=0.4727)
------------------------------------------------------------------------------------------
Epoch 33 | Train Loss: 2.0569 | Train Acc: 0.3316 | Val Loss: 1.7797 | Val Acc: 0.4727
------------------------------------------------------------------------------------------
Epoch 34 | Train Loss: 2.0340 | Train Acc: 0.3515 | Val Loss: 1.7686 | Val Acc: 0.4787
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 34, val_acc=0.4787)
------------------------------------------------------------------------------------------
Epoch 35 | Train Loss: 2.0198 | Train Acc: 0.3502 | Val Loss: 1.7392 | Val Acc: 0.4833
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 35, val_acc=0.4833)
------------------------------------------------------------------------------------------
Epoch 36 | Train Loss: 2.0151 | Train Acc: 0.3473 | Val Loss: 1.7231 | Val Acc: 0.4840
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 36, val_acc=0.4840)
------------------------------------------------------------------------------------------
Epoch 37 | Train Loss: 1.9916 | Train Acc: 0.3559 | Val Loss: 1.7007 | Val Acc: 0.4933
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 37, val_acc=0.4933)
------------------------------------------------------------------------------------------
Epoch 38 | Train Loss: 1.9769 | Train Acc: 0.3611 | Val Loss: 1.6910 | Val Acc: 0.4907
------------------------------------------------------------------------------------------
Epoch 39 | Train Loss: 1.9676 | Train Acc: 0.3563 | Val Loss: 1.6701 | Val Acc: 0.4933
------------------------------------------------------------------------------------------
Epoch 40 | Train Loss: 1.9521 | Train Acc: 0.3675 | Val Loss: 1.6529 | Val Acc: 0.5027
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 40, val_acc=0.5027)
------------------------------------------------------------------------------------------
Epoch 41 | Train Loss: 1.9399 | Train Acc: 0.3730 | Val Loss: 1.6390 | Val Acc: 0.5060
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 41, val_acc=0.5060)
------------------------------------------------------------------------------------------
Epoch 42 | Train Loss: 1.9278 | Train Acc: 0.3763 | Val Loss: 1.6149 | Val Acc: 0.5093
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 42, val_acc=0.5093)
------------------------------------------------------------------------------------------
Epoch 43 | Train Loss: 1.9170 | Train Acc: 0.3772 | Val Loss: 1.6036 | Val Acc: 0.5187
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 43, val_acc=0.5187)
------------------------------------------------------------------------------------------
Epoch 44 | Train Loss: 1.9054 | Train Acc: 0.3811 | Val Loss: 1.5801 | Val Acc: 0.5193
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 44, val_acc=0.5193)
------------------------------------------------------------------------------------------
Epoch 45 | Train Loss: 1.8968 | Train Acc: 0.3860 | Val Loss: 1.5641 | Val Acc: 0.5280
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 45, val_acc=0.5280)
------------------------------------------------------------------------------------------
Epoch 46 | Train Loss: 1.8866 | Train Acc: 0.3880 | Val Loss: 1.5456 | Val Acc: 0.5353
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 46, val_acc=0.5353)
------------------------------------------------------------------------------------------
Epoch 47 | Train Loss: 1.8648 | Train Acc: 0.3963 | Val Loss: 1.5349 | Val Acc: 0.5353
------------------------------------------------------------------------------------------
Epoch 48 | Train Loss: 1.8527 | Train Acc: 0.3923 | Val Loss: 1.5184 | Val Acc: 0.5427
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 48, val_acc=0.5427)
------------------------------------------------------------------------------------------
Epoch 49 | Train Loss: 1.8351 | Train Acc: 0.4094 | Val Loss: 1.5082 | Val Acc: 0.5480
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 49, val_acc=0.5480)
------------------------------------------------------------------------------------------
Epoch 50 | Train Loss: 1.8350 | Train Acc: 0.3980 | Val Loss: 1.4857 | Val Acc: 0.5507
------------------------------------------------------------------------------------------
Model saved to checkpoints\mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
Saved new best checkpoint (epoch 50, val_acc=0.5507)
------------------------------------------------------------------------------------------
Training finished. Best val acc: 0.5507 (epoch 50)
==========================================================================================
Metrics persisted for mlp_3x512-256-128_bs256_sgd_lr0_0005 -> metrics\mlp\mlp_3x512-256-128_bs256_sgd_lr0_0005.json

## 8. Saving Training Progress Plots

Code 
ARCHITECTURE_IDX = 2

for bs in BATCH_SIZE:
    for lr in LEARNING_RATE:
        print(f"\nPlotting training process for batch size {bs} and learning rate {lr} (SGD)")
        plot_training_process(
            total_acc_train_sgd[(bs, lr)],
            total_loss_train_sgd[(bs, lr)],
            total_acc_validation_sgd[(bs, lr)],
            total_loss_validation_sgd[(bs, lr)],
            f"plots/mlp/{architectures[ARCHITECTURE_IDX]['name']}_{bs}_{lr}_sgd.png"
        )

        print(f"\nPlotting training process for batch size {bs} and learning rate {lr} (Adam)")
        plot_training_process(
            total_acc_train_adam[(bs, lr)],
            total_loss_train_adam[(bs, lr)],
            total_acc_validation_adam[(bs, lr)],
            total_loss_validation_adam[(bs, lr)],
            f"plots/mlp/{architectures[ARCHITECTURE_IDX]['name']}_{bs}_{lr}_adam.png"
        )

Plotting training process for batch size 32 and learning rate 0.01 (SGD)
Training process plot saved to plots/mlp/mlp_3x512-256-128_32_0.01_sgd.png

Plotting training process for batch size 32 and learning rate 0.01 (Adam)
Training process plot saved to plots/mlp/mlp_3x512-256-128_32_0.01_adam.png

Plotting training process for batch size 32 and learning rate 0.001 (SGD)
Training process plot saved to plots/mlp/mlp_3x512-256-128_32_0.001_sgd.png

Plotting training process for batch size 32 and learning rate 0.001 (Adam)
Training process plot saved to plots/mlp/mlp_3x512-256-128_32_0.001_adam.png

Plotting training process for batch size 32 and learning rate 0.0005 (SGD)
Training process plot saved to plots/mlp/mlp_3x512-256-128_32_0.0005_sgd.png

Plotting training process for batch size 32 and learning rate 0.0005 (Adam)
Training process plot saved to plots/mlp/mlp_3x512-256-128_32_0.0005_adam.png

Plotting training process for batch size 256 and learning rate 0.01 (SGD)
Training process plot saved to plots/mlp/mlp_3x512-256-128_256_0.01_sgd.png

Plotting training process for batch size 256 and learning rate 0.01 (Adam)
Training process plot saved to plots/mlp/mlp_3x512-256-128_256_0.01_adam.png

Plotting training process for batch size 256 and learning rate 0.001 (SGD)
Training process plot saved to plots/mlp/mlp_3x512-256-128_256_0.001_sgd.png

Plotting training process for batch size 256 and learning rate 0.001 (Adam)
Training process plot saved to plots/mlp/mlp_3x512-256-128_256_0.001_adam.png

Plotting training process for batch size 256 and learning rate 0.0005 (SGD)
Training process plot saved to plots/mlp/mlp_3x512-256-128_256_0.0005_sgd.png

Plotting training process for batch size 256 and learning rate 0.0005 (Adam)
Training process plot saved to plots/mlp/mlp_3x512-256-128_256_0.0005_adam.png

9. Testing

During evaluation we scan checkpoints/mlp/, parse filenames using regex to recover:
depth, hidden_units, batch, optimizer, learning_rate. We then:

  1. Reconstruct the exact MLP architecture.
  2. Evaluate on the test set.
  3. Track the best model by test accuracy (ties can be extended to break on F1).
  4. Plot confusion matrix + training curves for the corresponding run.
  5. Use the same parsed metadata to standardize plot filenames:
plots/mlp/best_model_training_process_{model_name}_{batch}_{lr}_{optimizer}.png
Code 
pattern_with_opt = re.compile(r"^mlp_(\d+)x([0-9\-]+)_(\d+)_(adam|sgd)_([0-9.]+)\.pth$")
pattern_no_opt = re.compile(r"^mlp_(\d+)x([0-9\-]+)_(\d+)_([0-9.]+)\.pth$")

def parse_checkpoint_filename(fname: str):
    """Parse new MLP checkpoint filename format.

    Returns dict with: depth, units(list[int] or int), batch, optimizer (str|None), lr (float) or None if not parseable.
    """
    m = pattern_with_opt.match(fname)
    if m:
        depth = int(m.group(1))
        units_raw = m.group(2)
        batch = int(m.group(3))
        optimizer = m.group(4)
        lr = float(m.group(5))
    else:
        m2 = pattern_no_opt.match(fname)
        if not m2:
            return None
        depth = int(m2.group(1))
        units_raw = m2.group(2)
        batch = int(m2.group(3))
        optimizer = None
        lr = float(m2.group(4))
    # units pattern may be single int or hyphen separated
    if '-' in units_raw:
        units_list = [int(u) for u in units_raw.split('-') if u]
        hidden_units = units_list
    else:
        hidden_units = int(units_raw)
    return {
        "depth": depth,
        "hidden_units": hidden_units,
        "batch": batch,
        "optimizer": optimizer,
        "lr": lr,
    }
Code 
MODELS_DIR = "checkpoints/mlp"

best = {
    "path": None,
    "acc": 0.0,
    "f1": 0.0,
    "cm": None,
    "hidden_layers": None,
    "hidden_units": None,
    "batch_size": None,
    "optimizer": None,
    "learning_rate": None,
}


# Iterate over all .pth files in directory
if os.path.isdir(MODELS_DIR):
    for model_file in os.listdir(MODELS_DIR):
        if not model_file.endswith(".pth"):
            continue
        parsed = parse_checkpoint_filename(model_file)
        if not parsed:
            print(f"Skipping unrecognized filename pattern: {model_file}")
            continue
        model_path = os.path.join(MODELS_DIR, model_file)
        print(f"\nTesting model: {model_path}")
        print(f"  Parsed => depth: {parsed['depth']} hidden_units: {parsed['hidden_units']} batch: {parsed['batch']} optimizer: {parsed['optimizer']} lr: {parsed['lr']}")

        batch_size = parsed['batch']
        # hidden_units may be int or list
        model_hidden_units = parsed['hidden_units']
        model_mlp = MLPBaseline(
            input_dim=INPUT_DIM,
            num_classes=NUM_CLASSES,
            hidden_layers=parsed['depth'],
            hidden_units=model_hidden_units,
        ).to(device)
        model_mlp.load_state_dict(torch.load(model_path, map_location=device))
        model_mlp.eval()

        # Evaluate with correct dataloader
        test_loader = dataloaders_dict.get(batch_size, dataloaders_dict[BATCH_SIZE[0]])["test"]
        test_acc, f1, cm = test_model(model_mlp, test_loader, criterion)
        print(f"Accuracy: {test_acc:.4f} | F1: {f1:.4f}")

        # Update best model
        if test_acc > best["acc"]:
            best.update({
                "path": model_path,
                "acc": test_acc,
                "f1": f1,
                "cm": cm,
                "hidden_layers": parsed['depth'],
                "hidden_units": parsed['hidden_units'],
                "batch_size": parsed['batch'],
                "optimizer": parsed['optimizer'],
                "learning_rate": parsed['lr'],
            })
            print(f"  -> New best model: {model_path} (Acc: {test_acc:.4f}, F1: {f1:.4f})")
else:
    print(f"Models directory not found: {MODELS_DIR}")

Testing model: checkpoints/mlp\mlp_1x512_256_adam_0.0005.pth
  Parsed => depth: 1 hidden_units: 512 batch: 256 optimizer: adam lr: 0.0005
Test Loss: 0.3945 | Test Accuracy: 0.8960 | Macro F1: 0.8959
Accuracy: 0.8960 | F1: 0.8959
  -> New best model: checkpoints/mlp\mlp_1x512_256_adam_0.0005.pth (Acc: 0.8960, F1: 0.8959)

Testing model: checkpoints/mlp\mlp_1x512_256_adam_0.001.pth
  Parsed => depth: 1 hidden_units: 512 batch: 256 optimizer: adam lr: 0.001
Test Loss: 0.4004 | Test Accuracy: 0.8933 | Macro F1: 0.8938
Accuracy: 0.8933 | F1: 0.8938

Testing model: checkpoints/mlp\mlp_1x512_256_adam_0.01.pth
  Parsed => depth: 1 hidden_units: 512 batch: 256 optimizer: adam lr: 0.01
Test Loss: 0.5964 | Test Accuracy: 0.8193 | Macro F1: 0.8184
Accuracy: 0.8193 | F1: 0.8184

Testing model: checkpoints/mlp\mlp_1x512_256_sgd_0.0005.pth
  Parsed => depth: 1 hidden_units: 512 batch: 256 optimizer: sgd lr: 0.0005
Test Loss: 1.0823 | Test Accuracy: 0.7213 | Macro F1: 0.7208
Accuracy: 0.7213 | F1: 0.7208

Testing model: checkpoints/mlp\mlp_1x512_256_sgd_0.001.pth
  Parsed => depth: 1 hidden_units: 512 batch: 256 optimizer: sgd lr: 0.001
Test Loss: 0.7773 | Test Accuracy: 0.8073 | Macro F1: 0.8070
Accuracy: 0.8073 | F1: 0.8070

Testing model: checkpoints/mlp\mlp_1x512_256_sgd_0.01.pth
  Parsed => depth: 1 hidden_units: 512 batch: 256 optimizer: sgd lr: 0.01
Test Loss: 0.5651 | Test Accuracy: 0.8407 | Macro F1: 0.8415
Accuracy: 0.8407 | F1: 0.8415

Testing model: checkpoints/mlp\mlp_1x512_32_adam_0.0005.pth
  Parsed => depth: 1 hidden_units: 512 batch: 32 optimizer: adam lr: 0.0005
Test Loss: 0.4149 | Test Accuracy: 0.8927 | Macro F1: 0.8927
Accuracy: 0.8927 | F1: 0.8927

Testing model: checkpoints/mlp\mlp_1x512_32_adam_0.001.pth
  Parsed => depth: 1 hidden_units: 512 batch: 32 optimizer: adam lr: 0.001
Test Loss: 0.4488 | Test Accuracy: 0.8707 | Macro F1: 0.8703
Accuracy: 0.8707 | F1: 0.8703

Testing model: checkpoints/mlp\mlp_1x512_32_adam_0.01.pth
  Parsed => depth: 1 hidden_units: 512 batch: 32 optimizer: adam lr: 0.01
Test Loss: 0.9853 | Test Accuracy: 0.6727 | Macro F1: 0.6652
Accuracy: 0.6727 | F1: 0.6652

Testing model: checkpoints/mlp\mlp_1x512_32_sgd_0.0005.pth
  Parsed => depth: 1 hidden_units: 512 batch: 32 optimizer: sgd lr: 0.0005
Test Loss: 0.4753 | Test Accuracy: 0.8713 | Macro F1: 0.8714
Accuracy: 0.8713 | F1: 0.8714

Testing model: checkpoints/mlp\mlp_1x512_32_sgd_0.001.pth
  Parsed => depth: 1 hidden_units: 512 batch: 32 optimizer: sgd lr: 0.001
Test Loss: 0.5046 | Test Accuracy: 0.8627 | Macro F1: 0.8639
Accuracy: 0.8627 | F1: 0.8639

Testing model: checkpoints/mlp\mlp_1x512_32_sgd_0.01.pth
  Parsed => depth: 1 hidden_units: 512 batch: 32 optimizer: sgd lr: 0.01
Test Loss: 0.4036 | Test Accuracy: 0.8867 | Macro F1: 0.8873
Accuracy: 0.8867 | F1: 0.8873

Testing model: checkpoints/mlp\mlp_2x256_256_adam_0.0005.pth
  Parsed => depth: 2 hidden_units: 256 batch: 256 optimizer: adam lr: 0.0005
Test Loss: 0.4382 | Test Accuracy: 0.8760 | Macro F1: 0.8759
Accuracy: 0.8760 | F1: 0.8759

Testing model: checkpoints/mlp\mlp_2x256_256_adam_0.001.pth
  Parsed => depth: 2 hidden_units: 256 batch: 256 optimizer: adam lr: 0.001
Test Loss: 0.3188 | Test Accuracy: 0.9047 | Macro F1: 0.9045
Accuracy: 0.9047 | F1: 0.9045
  -> New best model: checkpoints/mlp\mlp_2x256_256_adam_0.001.pth (Acc: 0.9047, F1: 0.9045)

Testing model: checkpoints/mlp\mlp_2x256_256_adam_0.01.pth
  Parsed => depth: 2 hidden_units: 256 batch: 256 optimizer: adam lr: 0.01
Test Loss: 0.6511 | Test Accuracy: 0.8007 | Macro F1: 0.7989
Accuracy: 0.8007 | F1: 0.7989

Testing model: checkpoints/mlp\mlp_2x256_256_sgd_0.0005.pth
  Parsed => depth: 2 hidden_units: 256 batch: 256 optimizer: sgd lr: 0.0005
Test Loss: 1.2766 | Test Accuracy: 0.6373 | Macro F1: 0.6308
Accuracy: 0.6373 | F1: 0.6308

Testing model: checkpoints/mlp\mlp_2x256_256_sgd_0.001.pth
  Parsed => depth: 2 hidden_units: 256 batch: 256 optimizer: sgd lr: 0.001
Test Loss: 0.9235 | Test Accuracy: 0.7420 | Macro F1: 0.7401
Accuracy: 0.7420 | F1: 0.7401

Testing model: checkpoints/mlp\mlp_2x256_256_sgd_0.01.pth
  Parsed => depth: 2 hidden_units: 256 batch: 256 optimizer: sgd lr: 0.01
Test Loss: 0.3972 | Test Accuracy: 0.8820 | Macro F1: 0.8819
Accuracy: 0.8820 | F1: 0.8819

Testing model: checkpoints/mlp\mlp_2x256_32_adam_0.0005.pth
  Parsed => depth: 2 hidden_units: 256 batch: 32 optimizer: adam lr: 0.0005
Test Loss: 0.3817 | Test Accuracy: 0.8873 | Macro F1: 0.8867
Accuracy: 0.8873 | F1: 0.8867

Testing model: checkpoints/mlp\mlp_2x256_32_adam_0.001.pth
  Parsed => depth: 2 hidden_units: 256 batch: 32 optimizer: adam lr: 0.001
Test Loss: 0.4685 | Test Accuracy: 0.8587 | Macro F1: 0.8582
Accuracy: 0.8587 | F1: 0.8582

Testing model: checkpoints/mlp\mlp_2x256_32_adam_0.01.pth
  Parsed => depth: 2 hidden_units: 256 batch: 32 optimizer: adam lr: 0.01
Test Loss: 1.1275 | Test Accuracy: 0.6520 | Macro F1: 0.6501
Accuracy: 0.6520 | F1: 0.6501

Testing model: checkpoints/mlp\mlp_2x256_32_sgd_0.0005.pth
  Parsed => depth: 2 hidden_units: 256 batch: 32 optimizer: sgd lr: 0.0005
Test Loss: 0.6782 | Test Accuracy: 0.8073 | Macro F1: 0.8064
Accuracy: 0.8073 | F1: 0.8064

Testing model: checkpoints/mlp\mlp_2x256_32_sgd_0.001.pth
  Parsed => depth: 2 hidden_units: 256 batch: 32 optimizer: sgd lr: 0.001
Test Loss: 0.4540 | Test Accuracy: 0.8693 | Macro F1: 0.8681
Accuracy: 0.8693 | F1: 0.8681

Testing model: checkpoints/mlp\mlp_2x256_32_sgd_0.01.pth
  Parsed => depth: 2 hidden_units: 256 batch: 32 optimizer: sgd lr: 0.01
Test Loss: 0.4238 | Test Accuracy: 0.8733 | Macro F1: 0.8732
Accuracy: 0.8733 | F1: 0.8732

Testing model: checkpoints/mlp\mlp_3x512-256-128_256_adam_0.0005.pth
  Parsed => depth: 3 hidden_units: [512, 256, 128] batch: 256 optimizer: adam lr: 0.0005
Test Loss: 0.2695 | Test Accuracy: 0.9140 | Macro F1: 0.9139
Accuracy: 0.9140 | F1: 0.9139
  -> New best model: checkpoints/mlp\mlp_3x512-256-128_256_adam_0.0005.pth (Acc: 0.9140, F1: 0.9139)

Testing model: checkpoints/mlp\mlp_3x512-256-128_256_adam_0.001.pth
  Parsed => depth: 3 hidden_units: [512, 256, 128] batch: 256 optimizer: adam lr: 0.001
Test Loss: 0.2611 | Test Accuracy: 0.9173 | Macro F1: 0.9171
Accuracy: 0.9173 | F1: 0.9171
  -> New best model: checkpoints/mlp\mlp_3x512-256-128_256_adam_0.001.pth (Acc: 0.9173, F1: 0.9171)

Testing model: checkpoints/mlp\mlp_3x512-256-128_256_adam_0.01.pth
  Parsed => depth: 3 hidden_units: [512, 256, 128] batch: 256 optimizer: adam lr: 0.01
Test Loss: 0.6350 | Test Accuracy: 0.7960 | Macro F1: 0.7941
Accuracy: 0.7960 | F1: 0.7941

Testing model: checkpoints/mlp\mlp_3x512-256-128_256_sgd_0.0005.pth
  Parsed => depth: 3 hidden_units: [512, 256, 128] batch: 256 optimizer: sgd lr: 0.0005
Test Loss: 1.5322 | Test Accuracy: 0.5567 | Macro F1: 0.5468
Accuracy: 0.5567 | F1: 0.5468

Testing model: checkpoints/mlp\mlp_3x512-256-128_256_sgd_0.001.pth
  Parsed => depth: 3 hidden_units: [512, 256, 128] batch: 256 optimizer: sgd lr: 0.001
Test Loss: 1.0112 | Test Accuracy: 0.7120 | Macro F1: 0.7109
Accuracy: 0.7120 | F1: 0.7109

Testing model: checkpoints/mlp\mlp_3x512-256-128_256_sgd_0.01.pth
  Parsed => depth: 3 hidden_units: [512, 256, 128] batch: 256 optimizer: sgd lr: 0.01
Test Loss: 0.4240 | Test Accuracy: 0.8693 | Macro F1: 0.8678
Accuracy: 0.8693 | F1: 0.8678

Testing model: checkpoints/mlp\mlp_3x512-256-128_32_adam_0.0005.pth
  Parsed => depth: 3 hidden_units: [512, 256, 128] batch: 32 optimizer: adam lr: 0.0005
Test Loss: 0.3128 | Test Accuracy: 0.9113 | Macro F1: 0.9110
Accuracy: 0.9113 | F1: 0.9110

Testing model: checkpoints/mlp\mlp_3x512-256-128_32_adam_0.001.pth
  Parsed => depth: 3 hidden_units: [512, 256, 128] batch: 32 optimizer: adam lr: 0.001
Test Loss: 0.4147 | Test Accuracy: 0.8747 | Macro F1: 0.8747
Accuracy: 0.8747 | F1: 0.8747

Testing model: checkpoints/mlp\mlp_3x512-256-128_32_adam_0.01.pth
  Parsed => depth: 3 hidden_units: [512, 256, 128] batch: 32 optimizer: adam lr: 0.01
Test Loss: 1.3245 | Test Accuracy: 0.5713 | Macro F1: 0.5620
Accuracy: 0.5713 | F1: 0.5620

Testing model: checkpoints/mlp\mlp_3x512-256-128_32_sgd_0.0005.pth
  Parsed => depth: 3 hidden_units: [512, 256, 128] batch: 32 optimizer: sgd lr: 0.0005
Test Loss: 0.5245 | Test Accuracy: 0.8473 | Macro F1: 0.8463
Accuracy: 0.8473 | F1: 0.8463

Testing model: checkpoints/mlp\mlp_3x512-256-128_32_sgd_0.001.pth
  Parsed => depth: 3 hidden_units: [512, 256, 128] batch: 32 optimizer: sgd lr: 0.001
Test Loss: 0.4121 | Test Accuracy: 0.8773 | Macro F1: 0.8768
Accuracy: 0.8773 | F1: 0.8768

Testing model: checkpoints/mlp\mlp_3x512-256-128_32_sgd_0.01.pth
  Parsed => depth: 3 hidden_units: [512, 256, 128] batch: 32 optimizer: sgd lr: 0.01
Test Loss: 0.3635 | Test Accuracy: 0.8940 | Macro F1: 0.8937
Accuracy: 0.8940 | F1: 0.8937

Plot training process and confusion matrix of the best model

Code 
if best["cm"] is not None:
    print(f"\nBest model: {best['path']} | Acc: {best['acc']:.4f} | F1: {best['f1']:.4f}")
    plot_confusion_matrix(best["cm"], datasets["train"].classes)

Best model: checkpoints/mlp\mlp_3x512-256-128_256_adam_0.001.pth | Acc: 0.9173 | F1: 0.9171

Code 
if best["path"] is not None:
    hidden_units = best['hidden_units']
    depth = best['hidden_layers']
    learning_rate = best['learning_rate']
    optimizer = best["optimizer"]
    if isinstance(hidden_units, list):
        if len(set(hidden_units)) == 1:
            units_part = str(hidden_units[0])
        else:
            units_part = '-'.join(str(u) for u in hidden_units)
    else:
        units_part = str(hidden_units)
    model_name = f"mlp_{depth}x{units_part}"

    print(f"Model_name: {model_name} depth: {depth} hidden_units: {hidden_units} batch: {batch_size} optimizer: {optimizer} lr: {learning_rate}")
    key = (batch_size, learning_rate)
    plot_dir = "plots/mlp"
    os.makedirs(plot_dir, exist_ok=True)
    out_path = f"{plot_dir}/best_model_{model_name}_{batch_size}_{learning_rate}_{optimizer}.png"
    if optimizer == "sgd":
        plot_training_process(
            total_acc_train_sgd[key],
            total_loss_train_sgd[key],
            total_acc_validation_sgd[key],
            total_loss_validation_sgd[key],
            out_path
        )
    else:
        plot_training_process(
            total_acc_train_adam[key],
            total_loss_train_adam[key],
            total_acc_validation_adam[key],
            total_loss_validation_adam[key],
            out_path
        )
else:
    print("No best model path available.")
Model_name: mlp_3x512-256-128 depth: 3 hidden_units: [512, 256, 128] batch: 32 optimizer: adam lr: 0.001
Training process plot saved to plots/mlp/best_model_mlp_3x512-256-128_32_0.001_adam.png

## 10. Inference

  1. A single random image is sampled from each class folder under data/test/ (one representative per class).
  2. The previously selected best model (determined during the testing phase by highest test accuracy, with F1 as an auxiliary metric) is reconstructed with the exact depth and hidden unit pattern parsed from its checkpoint filename.
  3. The model weights are loaded and the model is switched to evaluation mode (model.eval()), disabling dropout and other training‑only behaviors.
  4. Each selected image is passed to a predict_image helper which:
    • Loads the image from disk.
    • Applies the same preprocessing pipeline used for training (grayscale → resize → tensor → optional flatten).
    • Moves the tensor to the configured device (CPU or GPU).
    • Performs a forward pass to obtain raw logits.
    • Applies softmax to convert logits into class probabilities.
    • Returns: (predicted_class_label, probability_vector).
  5. The script prints the file path, predicted class, and full probability distribution for inspection.

Sampling one image per class yields a concise qualitative sanity check—verifying the model is not collapsing onto a small subset of labels and that probabilities are reasonably calibrated across categories.

  • A confident prediction typically shows one probability ≫ others (e.g., >0.85).
  • Diffuse probability mass may indicate class ambiguity or insufficient training for that pattern.
  • You can aggregate probabilities across many samples to study calibration or create reliability diagrams.

Reproducibility Notes - Because selection uses random.choice, results vary run to run unless a random seed is fixed beforehand. - If the best model path is missing (e.g., you skipped the training/testing cells), inference is skipped gracefully.

Code 
# Collect one random image per class folder in data/test

TEST_ROOT = "data/test"
selected_images = []
for cls in sorted(os.listdir(TEST_ROOT)):
    cls_dir = os.path.join(TEST_ROOT, cls)
    if not os.path.isdir(cls_dir):
        continue
    imgs = [f for f in os.listdir(cls_dir) if f.lower().endswith((".jpg",))]
    if not imgs:
        continue
    selected = os.path.join(cls_dir, random.choice(imgs))
    selected_images.append(selected)

if best["path"]:
    mlp_model = MLPBaseline(INPUT_DIM, NUM_CLASSES, hidden_layers=best['hidden_layers'], hidden_units=best['hidden_units']).to(device)
    mlp_model.load_state_dict(torch.load(best["path"], map_location=device))
    mlp_model.eval()

    mlp_model.load_state_dict(torch.load(best["path"]))

    for img_path in selected_images:
        print(f"Selected image for prediction: {img_path}")
        prediction, probs = predict_image(
            model=mlp_model,
            image_path=img_path,
            class_names=datasets['train'].classes,
            device=device,
        )
        print(f"Predicted class: {prediction} | Probabilities: {probs}")
        print("-" * 40)
else:
    print("No best model available; skipping inference.")
Selected image for prediction: data/test\0\input_75_8_1.jpg

Predicted class: 0 | Probabilities: tensor([[ 11.3904, -12.2261,  -2.3158,   3.8903,   0.3439,   1.8015,  -6.5832,
         -16.5941, -11.3394,   0.5255,  -3.3463,  -6.8872,  -8.2695,  -7.1423,
          -1.7092]])
----------------------------------------
Selected image for prediction: data/test\1\input_20_2_2.jpg

Predicted class: 1 | Probabilities: tensor([[-9.1706,  9.6848,  1.4202, -9.2204, -1.2867, -8.0299, -7.3803,  5.0229,
          0.1950, -2.9024, -7.4999,  0.1121, -0.4695, -0.8693, -8.1282]])
----------------------------------------
Selected image for prediction: data/test\10\input_84_6_11.jpg

Predicted class: 10 | Probabilities: tensor([[-3.2899, -3.9746,  7.0414, -3.2823,  5.6937, -0.9156, -8.8936, -4.4274,
         -5.3487, -3.0470, -5.2519, -1.2145, -1.0656, -4.3607, -5.3224]])
----------------------------------------
Selected image for prediction: data/test\100\input_43_6_12.jpg

Predicted class: 100 | Probabilities: tensor([[  1.1718, -11.6968,  -2.7218,   5.7633,   1.2186,   2.1666,  -3.9680,
          -9.0907,  -4.8230,  -0.2899,   0.7404,  -3.9622,  -3.5210,  -6.5571,
          -2.1264]])
----------------------------------------
Selected image for prediction: data/test\1000\input_37_4_13.jpg

Predicted class: 1000 | Probabilities: tensor([[-4.1152, -7.2432,  5.4166, -2.3016,  7.6224, -2.5369, -8.6737, -3.6526,
         -1.4864, -4.6155, -2.8225, -0.9238, -1.7768, -7.9694, -5.9671]])
----------------------------------------
Selected image for prediction: data/test\10000\input_95_8_14.jpg

Predicted class: 10000 | Probabilities: tensor([[-2.4220, -8.9127, -2.3464, -0.2305, -2.0238,  4.6374, -2.2130, -7.1580,
         -5.5547, -1.9363, -2.7200,  1.9036, -2.5352, -0.5148,  1.2866]])
----------------------------------------
Selected image for prediction: data/test\100000000\input_87_4_15.jpg

Predicted class: 100000000 | Probabilities: tensor([[-5.9294, -9.2025, -7.3634, -2.4977, -5.7419, -1.5768,  6.9346, -6.0697,
         -6.2999, -0.4386, -3.2526,  0.0448,  1.3977,  1.6831,  4.2096]])
----------------------------------------
Selected image for prediction: data/test\2\input_18_6_3.jpg

Predicted class: 2 | Probabilities: tensor([[-11.9768,   2.9587,  -5.1875,  -8.6680,  -4.5262,  -9.0045,  -5.2278,
           8.4477,   5.6607,  -4.4871,  -2.1239,   0.9703,   1.0224,  -3.5862,
          -5.6118]])
----------------------------------------
Selected image for prediction: data/test\3\input_60_6_4.jpg

Predicted class: 3 | Probabilities: tensor([[-11.4116,  -5.2593,  -7.3727,  -4.2574,  -0.6236,  -6.5885,  -7.4024,
           5.7431,  10.4569,  -6.3150,   2.4253,   1.7995,  -2.4591, -10.7197,
          -6.7950]])
----------------------------------------
Selected image for prediction: data/test\4\input_24_6_5.jpg

Predicted class: 4 | Probabilities: tensor([[-2.3199, -8.5628, -6.8482,  2.7207, -3.2714, -1.0685,  1.0094, -7.9601,
         -7.9154,  9.5457, -1.4743, -4.9515, -4.3279, -6.5351, -3.9005]])
----------------------------------------
Selected image for prediction: data/test\5\input_24_5_6.jpg

Predicted class: 5 | Probabilities: tensor([[ -5.8042, -11.2937,  -7.5235,   1.9233,  -2.0272,  -0.9582,  -2.6332,
          -4.2351,   0.8506,  -1.8879,   6.5357,  -2.8166,  -0.3333,  -9.2336,
           0.6918]])
----------------------------------------
Selected image for prediction: data/test\6\input_8_10_7.jpg

Predicted class: 6 | Probabilities: tensor([[-6.9944,  1.8791, -1.0389, -5.8421, -0.9480, -3.2694, -3.5747,  1.8478,
          0.1345, -2.5900, -5.3777,  3.3296, -1.1088, -1.0717, -3.8910]])
----------------------------------------
Selected image for prediction: data/test\7\input_16_1_8.jpg

Predicted class: 7 | Probabilities: tensor([[ -9.7747, -12.2382,  -6.6143,  -5.4310,  -5.6335,  -7.2178,   2.9036,
          -5.4629,  -4.6623,  -5.8294,  -2.4503,  -2.4188,  10.2430,  -7.1214,
           5.3602]])
----------------------------------------
Selected image for prediction: data/test\8\input_21_1_9.jpg

Predicted class: 8 | Probabilities: tensor([[-6.5955, -7.2172, -6.5703, -4.9139, -7.7215, -0.6308,  2.3653, -4.6833,
         -7.0078, -5.8925, -6.5288,  2.2233, -3.5481, 11.5187,  0.9459]])
----------------------------------------
Selected image for prediction: data/test\9\input_78_1_10.jpg

Predicted class: 9 | Probabilities: tensor([[ -5.0279, -13.0262,  -8.3055,  -1.1183,  -5.9557,   0.9460,   4.5270,
          -8.8749,  -6.1113,  -3.1296,  -1.2629,   0.8325,   1.5230,  -1.5704,
           6.6165]])
----------------------------------------