Street photographers...

... have to know when it is wise to feel afraid.

At that point, you fiddle with your camera, as if there is a fault, and walk calmly away.

What else can I say?