Go to Options in the top right of the diary
Toggle on 'Hide unavailable team members' -> Update.
If a column has bookings in it, it will still show.
Team member column showing when not scheduled
How to remove a team member from the diary who shouldn't be showing