I just re-checked this and works as expected. Hm…
I know you said that the game objecty you reference is not a prefab, but is the GraphOwner a prefab, or the Graph a non-bound graph reference (an Asset graph)?
This is the only thing I can think of going on here. Can you confirm?
Unity does not allow having a scene reference IN a prefab or asset reference. You can have an asset reference IN a scene object.