Driving above MAX DUTY CYCLE, braking without any demand

Hi all, 

I have been using a VESC to control a motor which I am using as a (sort of) generator. The motor in question is coupled to a larger AC motor which drives it, and I am investigating how well the motor/VESC setup can follow brake demands. 

The motor is powered by a bench power supply, and has circuitry built in so that when the VESC is braking, the power is sent through to a dump resistor. This has been working perfectly so far however I have now found a few problems I'm hoping someone can help me with. 

Essentially, when motor is being driven at a high enough speed so that the duty cycle rises above 95% (to say 120%), the motor begins braking without any brake demand from me, VESC tool seems to think that the motor current and battery current are both 0, however with a torque transducer I have set up I can see that the motor is providing torque.

On top of this, if I then demand a braking (or driving) current, instead of listening to what I tell it, the VESC provides whatever braking current it needs to return to the 95% duty cycle. For example I demand 2A of braking and it hits me with 17A. I understand this isn't what VESC'S are usually used for, but in my head someone with a low KV motor riding down a hill could essentially get a very large braking force without asking for it, has anyone experienced something like this before?

From my (perhaps limited) understanding, it seems that when the back EMF of the motor coils rises above the VESC supply voltage, the motor will provide braking torque without any demand from the VESC, but when I have the VESC disconnected from the motor so the coils are floating, the motor can be driven at any speed without any torque generated, so I'm not sure why the VESC achieve this?

I would really appreciate anyone's help/thoughts/advice on this!

Thanks!