Model Distillation: Kidney SOFA Example

author:

date: 30. April 2021

download: https://github.com/StatNLP/empirical_methods


Learner: Feed forward neural network with seven layers (number of neurons per layer: 128, 161, 203 , 256, 203 , 161, 128) and relu activation function. The network was trained with SGD (batch size 64) and learning rate .01 for 5 epochs and a dropoutrate of .2 for inter hidden layer weights. All other optimizer settings are default values of pyTorch's SGD optimizer. The objective function was MSE.

In a second step thresholds to turn the network output (score) into class predictions were learned . For this purpose a ordinal regression model was trained (R package "ordinal").

Train set size: nrow(train_data) measurement points. Test set size: nrow(test_data) measurement points.

Performance Evaluation

Circular Model - Train set accuracy (overall and per class)

Confusion Matrix:

Circular Model - Test set accuracy (overall and per class)

Confusion Matrix:

Non-circular Model - Train set accuracy (overall and per class)

Confusion Matrix:

Non-circular Model - Test set accuracy (overall and per class)

Confusion Matrix:

Correlation pattern

Nullification

TEACHER: circular, STUDENT: all features

TEACHER: circular, STUDENT: no bili

TEACHER: non-circular, STUDENT: all features

TEACHER: non-circular, STUDENT: no bili