If you read the HBO GoT site, the actor playing The Mountain is actually really 7 foot tall in real life. He just doens't come across that way in episode 4 because he isn't standing still next to any of the shorter actors. The actor playing the Hound is only a few inches off 7 foot himself (like in the books), so again, The Mountain doesn't look that big next to him.
They've given him a big horse to carry the weight, and a sword that matches as well, so you have to pay attention to actually get the scale of the actor. Try watching Loras on the ground near where The Mountain and The Hound fight, and later watch The Mountain push through the crowd when King Robert halts the fight.
The actor playing Hodor is halfway between 6 and 7 feet, and the one playing Drogo is the fourth tallest actor at about 6"4'