Thanks for the detailed set of posts. I must say that I'm somehow disappointed from the results. First, your training schedule is very specific and contains many hyperpaprameters, what implies overfitting, or at least very high specificity for this data sets. Aditonaly you've applied a lot of research effort in your task, and stated that "deeper dive is required" I don't hink this is the way to go. I think the way to go would be trying simpler approaches on more datasets, tryin to genelize.

Cheers :)