Your track down is very correct. Amongst all Script Control tasks, CheckField/SetField is the slowest as it is using pure reflection and no delegation (fieldInfo.GetValue…), while all other script control tasks (eg CheckProperty) are faster since a delegate is created instead of using pure reflection.
The reason CheckField/GetField does not, is that from a field you can’t create a delegate somehow (that I know of), while from a property you can, since properties do have a method representation for getter/setter.
So with that said, there is no further optimization that can be done in CheckField & SetField tasks, unless a delegate can be created *somehow*
So the actual solution if you definetely need to, is to use properties instead of fields 🙂