Alert! If the purpose is being used as a default purpose for a donor or the purpose of a donation you will not be able to delete it as this forms part of your audit trail.
An alternative to deleting the purpose would be to click "Edit" and mark the purpose as inactive by unchecking the "Active" checkbox.