If the member wished to no longer have that person as a director then he could (in theory) rally support not to reelect the director when the resolution to reelect is proposed at the AGM
If the director is proposed for removal between AGMs then special notice and all associated rules will apply for an “other general meeting” to be convened by the directors in the mid-term