If I read your post correctly, when dist wire removed while cranking, stater motor speeds up? When re-connected, slows way down?
This is a symptom of too much inital timing lead. Back off distributor a few degrees, and try again.
However, if timing is set correctly, and it still bucks against the starter, and starter drags, especially when starter is heat-soaked, then I'd think of rebuilding the starter or get a rebuilt one. Just be sure to exchange the nose piece from the old to the new to keep the backlash setting. Also, place a heat shield over it, it'll be the best $20 spent.
I'm sure others here will have some good suggestions.