I am using VESC tool 6.05. It has the same behaviour with two different ESC-s (flipky 75100 and makerbase MKS84100) and motors (medium 16pole outrunner and large 14 pole outrunners with hall sensors):
No problems in FOC mode, motor and hall detection is OK. Motor is running normal.
If i change to BLDC mode, the detection is sometimes running well, and applying (saving) the parameters is succesful. My motors are running at higher rpms in BLDC mode and this at around 10% lower battery and motor amps. So i like it.
The problem ist, that switching between FOC and BLDC mode and saving the motor parameters seems to corrupt the parameter structure. Loading default parameters or a saved .xml config does not help. Then the only way to get a functioning BLDC mode again is to reflash the frimware and start a new clean setup.
Typically there are no error messages, but sometimes i get this by pressing the 'appy' button after a succesful BLDC motor/hall identification:
Or another error message (swapped phase lines, so different hall table):