After a whole bunch of calls I have it sorted out.

Problem was I was confusing a long button push on the headset for Voice Command activation with what's need to reconnect headset.

A quick Tap for out going or incoming works (whether it is in a disconnected state or not). If it's disconnected it just takes longer.

On out going I was pushing and holding listening for the Voice Command beep prompt thinking that was a reconnect headset beep. So when doing that on an incoming call it got all mixed up.