I think the best system would be to make it so people that don't meet the rank restriction can still play on the server and they just can't vote, unless they are the only person on the server, that way higher ranked people can't be voted off by noobs, but if the server is empty they can still play whatever they want.