Does it keep you in the queue or does it just kick you and reinvite you automatically when you log on/off?
If it keeps offline people in the queue then I found a possible explanation to the 20-minute "Waiting for last DPS to accept" crap I went through yesterday. Every time that damn check box appeared, one DPS would not accept.