Infaspeed Throttle Repair

I'm trying to complete a repair to my Duffy Instaspeed throttle, and my design is somewhat different than shown in this rebuild link: Duffy InfaSpeed Controller Throttle Rebuild (dougpile.com)  

For some background, the problem I'm experiencing is that there is a "dead spot" in the forward throttle position. In other words, when I move the handle forward about 3/4 of the way, the motor "stalls". If I move the handle back or forward beyond this dead spot, the motor resumes and everything is fine. I've confirmed with an ohmmeter that the dead spot is in the potentiometer (aka "pot" or rheostat), and I've obtained the replacement pot.

The problem is that instead of gears, my throttle has a one-piece black collar and handle shaft that is attached directly to the pot shaft, as shown in the picture, and I can't seem to get the black collar/shaft off of the shaft of the pot. There is one set screw, which I've removed, and what appears to be a turn limit pin for the throttle handle. I'm thinking the limit pin has nothing to do with securing the collar to the shaft, but the collar won't budge, and I don't want to get too aggressive for fear of destroying something.

Am I missing something here for removing the collar? Any suggestions about how to proceed? I contacted the Duffy parts department to see if I could get a new collar (if that's what it's called), but they only sell the entire throttle assembly for about $400.  I want to avoid that, if possible, since the throttle just isn't that complicated.  

Thanks for any advice or suggestions!

NOTE:  I can't seem to attach the picture of the guts of my throttle!