It sounds like you need to pick between how real-time versus smooth you would like the behavior to be.

Genuinely real-time movement does not let you plan any motion, since you don’t know what the future will bring. That leads to several potential issues:
(1) Potentially moving too fast to stop or turn quickly
(2) Potential “gaps” in movement. If there is a time gap between subsequent motion commands, or not a smooth transition between subsequent motion commands, the motion will be jerky.
(3) Jerky movement for other reasons, for example sending movements in too short increments, such that the USB has trouble keeping up.

A few things to consider: Adding an intentional delay of (say) 100 ms between real time and intended movement can help a lot. You could make it such that motion commands are sent every 20 ms on a schedule (so that there are no gaps), such that each movement is aware of the present position and speed, and forms a bridge between where you are now (in position and speed) and where you want to be in 100 ms.