The Governor, despite being aware of the fact that Evans was very clever, did not make any fool proof arrangements to bring him back. He could have called the prison force instead of police to take Evans back to the prison house. He should have made more elaborate security arrangements. He could have cross-checked the identity of the officers who took Evans away or he would have personally escorted Evans back to jail.
