You can only drive the pin out in one direction. It has a bulbous head on one side. That one simply limits the handle travel. The other pin has a hair pin clip,which pushes out, then the pin and the cable idler will drop out.
If the shaft is in good condition, you can clean it up with 0000 steel wool,and replace the handle. It also has a pin that has to be drive out. The housing is semi-gloss black. Be sure to lubricate the idler pin when you re-install it.
-Keith