I do the same, retrieve with the left cast with the right, and I am right handed. I think my sensitivity and faster reflexes are in my dominate hand.
What happens if as you cast with your left and as the lure hits the water a fish takes it (not likely to happen, but you never know) but you now have to switch back to holding in your left and retrieveing with your right, and you have to take up the slack during which the fish may let go. Valuable seconds and maybe a good fish lost in changing your rod hand, don't you think? :blink:....lol.