The instantaneous velocity of the stone going around the circular path is always along that tangent to the circle at the instant. When the string breaks, the centripetal force ceases. Due to inertia, the stone continuous its motion along tangent to the circular path, that is why it flies away tangentially.
