Multiclass Classification of Chinese Handwriting Characters

Machine Learning Assignment 2

Carlos Emiliano Mendoza Hernandez

2025-10-16

Inputs and Outputs

• Definition of Task: Supervised multiclass image classification.
• Feature Extraction: Each image is flattened into a 4096-dimensional vector.
• Input: A flattened 64x64 grayscale image (4096 numerical features) representing a single handwritten character.
• Output: A single integer in {0, 1, 2, …, 100000000} indicating the predicted numeral.

Raw Inputs

Preprocessing Steps

1. The Dataset Loader (preprocessing_utils.py):
   • This module handles loading the images from the CSV file, applying data augmentation (rotation and affine transforms), and preparing them for the model (resizing, converting to tensors, and flattening).
   • It defines a standard PyTorch Dataset class to facilitate easy batching and shuffling during training.

Training Inputs

Model Components

• Model Choice:
  • Logistic Regression: Linear separability baseline.
  • MLP: Deeper architecture: repeated Linear -> BatchNorm -> ReLU -> Dropout blocks.
• Loss Function: Cross-entropy loss for multiclass classification.
• Optimization Algorithm: Adam and SGD.
• Evaluation Metric: Accuracy on validation and test sets, confusion matrix analysis.

Model Implementation

2. The MLP Model (mlp.py):
   • The core of our model is a configurable MLP.
   • It’s implemented as a PyTorch nn.Module, allowing for easy integration with PyTorch’s training loop.

Key components

• nn.Linear: Fully connected layers.
• nn.BatchNorm1d: Batch normalization layers to stabilize training.
• nn.ReLU: Activation function to introduce non-linearity.
• nn.Dropout: Regularization to prevent overfitting.

Model Summary

MLP summary (input_dim=4096, num_classes=15)

===================================================================================================================
Layer (type:depth-idx)                   Input Shape               Output Shape              Param #
===================================================================================================================
MLPBaseline                              [1, 4096]                 [1, 15]                   --
├─Sequential: 1-1                        [1, 4096]                 [1, 15]                   --
│    └─Linear: 2-1                       [1, 4096]                 [1, 512]                  2,097,664
│    └─BatchNorm1d: 2-2                  [1, 512]                  [1, 512]                  1,024
│    └─ReLU: 2-3                         [1, 512]                  [1, 512]                  --
│    └─Dropout: 2-4                      [1, 512]                  [1, 512]                  --
│    └─Linear: 2-5                       [1, 512]                  [1, 256]                  131,328
│    └─BatchNorm1d: 2-6                  [1, 256]                  [1, 256]                  512
│    └─ReLU: 2-7                         [1, 256]                  [1, 256]                  --
│    └─Dropout: 2-8                      [1, 256]                  [1, 256]                  --
│    └─Linear: 2-9                       [1, 256]                  [1, 128]                  32,896
│    └─BatchNorm1d: 2-10                 [1, 128]                  [1, 128]                  256
│    └─ReLU: 2-11                        [1, 128]                  [1, 128]                  --
│    └─Dropout: 2-12                     [1, 128]                  [1, 128]                  --
│    └─Linear: 2-13                      [1, 128]                  [1, 15]                   1,935
===================================================================================================================
Total params: 2,265,615
Trainable params: 2,265,615
Non-trainable params: 0
Total mult-adds (Units.MEGABYTES): 2.27
===================================================================================================================
Input size (MB): 0.02
Forward/backward pass size (MB): 0.01
Params size (MB): 9.06
Estimated Total Size (MB): 9.09
===================================================================================================================

Training Implementation

3. The training and Evaluation Loop (models_utils.py):

• Iterate over the training data in batches.

• Calculate the loss (Cross-Entropy Loss).

• Update model weights using backpropagation and the chosen optimizer (Adam or SGD).

• Evaluate the model on the validation set after each epoch to monitor performance and prevent overfitting.

• Early Stopping: Monitors validation loss to halt training if no improvement is observed over a set number of epochs.

Main Technical Challenges

1. Ensuring Reproducibility:

   • Challenge: With so many experiments (different learning rates, batch sizes, architectures) it’s hard to keep track of what works.
   • Solution: Logged all hyperparameters and results systematically.

Main Technical Challenges

2. Systematic Hyperparameter Sweeps:

   • Challenge: Manually running every combination of hyperparameters is time-consuming and error-prone.
   • Solution: Developed a script to automate hyperparameter sweeps, allowing for efficient exploration of the parameter space. We used nested loops in our main notebook (implementation.ipynb) to iterate over combinations of learning rates, batch sizes, and architectures.

Main Technical Challenges

3. Automated Model Selection:

   • Challenge: With dozens of saved models, how do we find the best one?
   • Solution: Writing a script that automatically loads each checkpoint, evaluates it on the test set, and ranks them by acuracy to find the champion model.

Best Model Training Curves

Model name: MLP (3 hidden layers, 512-256-128 units, dropout=0.5), Optimizer: Adam, Learning Rate: 0.001, Batch Size: 64

Training Curves

Best Model Confusion Matrix

• Accuracy: 0.9173
• F1-Score: 0.9171

Confusion Matrix

Benefits and Limitations

Benefits:

• Good for structured data: MLPs are a great general-purpose tool.
• Interpretable (to a degree): It’s easier to understand the impact of hyperparameters (like depth and width) on an MLP than on more complex models.
• Fast to train: Compared to larger models, these MLPs are computationally efficient.

Benefits and Limitations

Limitations:

• Ignores Spatial Information: By flattening the image into a 1D vector, we lose all the 2D spatial relationships between pixels. This is a major disadvantage for image data.
• Prone to Overfitting: On smaller datasets, MLPs can easily memorize the training data, which is why we need techniques like Dropout and early stopping.
• Outperformed by CNNs for Images: For almost any image classification task, a well-designed CNN will outperform an MLP.

Conclusions and Future Work

• Conclusions:

  • Successfully built a pipeline to train and evaluate MLP models for Chinese numeral classification, and systematically tested the impact of different hyperparameters.
  • Our framework demonstrated that, a moderately deep, regularized MLP is essential: an adaptive optimizer (Adam) to navigate the loss landscape efficiently, a well-chosen learning rate (0.001) to ensure stable steps, and a medium-sized batch (64) to balance training speed with generalization.

Conclusions and Future Work

• Future Work:

  • Implement a CNN baseline to leverage the spatial structure of the images.
  • Use automated hyperparameter tuning tools like Optuna or Ray Tune.
  • Ensemble the top-performing models to potentially boost accuracy further.