There’s no need to fill in all details. 

You can have a 'job' as annual leave or a staff meeting with not CRM contact selected. This allows you to 'block out' time for this user to avoid double booking.