To remove a contact in the Web Portal go to CRM > All Contacts. Check the box next to the Contact you want to delete and then select Set Inactive from the pop up menu at the bottom of the page. Multiple contacts can be made inactive at the same time.
Note: A contact cannot be made inactive in the App, only via the Web Portal.