You can delete a contact or company directly from their record. After navigating to the record, on the left side near the top of the page is a section “Actions.” You can select “Delete” there, and then confirm in the dialog that appears to remove the record.