"Green" means something that creates as little of a carbon footprint as possible, and is usually in relation to something else, ie x is greener than y.
I don't know if cotton could be considered "green" to any other cloth fiber plant. Things like hemp, bamboo, etc, are easier to grow and much more sustainable (require less water, pesticides, fuel, etc). Polyester is "green", because instead of putting plastic into landfills we are reusing it in a creative and effective way.
Silk and wool, well, these come from live animals and with silk many die in the process. I supposed that would make wool more sustainable, but I'd bet herds of alpaca create a bigger carbon footprint than silkworm larvae.
Without doing any research at all, I would guess that most polyester is still made with non-recycled petro-chemicals.
Cotton doesn't have to be planted yearly in all cases. It can be harvested as greenly and cleanly as any other plant. It probably doesn't yield as much fiber as hemp would with the same amount of resources but hemp, while strong, isn't as versatile as cotton for making cloth.
There is some real stretching going on here to vilify cotton simply because gbaji mentioned it as being a 'green' plant. lol.