Hey Pillage
Check that there is not a large buildup of chain lube "mung" all over the shift shaft where it runs through the left side case.
Regardless, I would remove the left side sprocket cover, clean everything and check the shift shaft for bending or damage. If this doesn't happen every time, it is likely the shift shaft is binding in the left side case, either because it is bent. damaged or full of crap.