Almost every day someone somewhere makes some attempt to hold a door open for me, and almost every day I get irritated about it. Why, you ask - they're just trying to be helpful. Well, I've found that most of the time when people hold a door open, they get in my way more than anything. Let me give you a few examples of when not to hold a door open, and I wish I had visual aids for this but I'm too underpaid at this writing job to go through that trouble. This will probably seem like I'm an ungrateful bastard who dislikes helping people but really I just want to make things more efficient.
1) When I'm still about 20 feet away, someone holds a door open and waits for me to get there. This bothers me because I feel rushed to get there so he doesn't have to stand there as long. I want to take my sweet time walking in the building. Sometimes I will go so far as to slow down or even stop, pretending to tie my shoe or something to avoid this.
2) Standing in the doorway while holding it open for someone. I'm not a very big guy but even this sometimes requires getting a bit too close for comfort. If you're going to hold the door open, get out of the door way so people have plenty of room to get through.
3) Holding the first door open when there's a second door immediately behind. Really, it's just more efficient to hold the second one, and much like #2, you often are clogging up the space between the doors when you hold door #1 while door #2 is still closed. I recommend going through both doors and then maybe you can hold door #2 as long as you can do this without being in the way.
4) Expecting me to say thanks for holding the door open and slowing me down by half a second. It's one thing if my hands are full with lunch for the office or with boxes, or if I'm elderly, but I rarely am any of those. Obviously, if any of those conditions apply, you should hold the door open for someone as long as you don't violate #2.
5) This one is a lot like #3. Holding a door open when you have to unlock the second door. Don't bother holding door #1 for me just so I can get in the cramped space sooner and wait for you to open the second door. Where I work, There's an unlocked door followed immediately by a door that we have to enter one at a time with our personal ID code. Then we go up the stairs where there is another door. And sometimes the person in front of me holds door #1, thereby making me wait to get through door #2, then holds door #3 for me when I'm still halfway up the stairs. grr.
6) Of course, this might all go out the window if you're on a date. As a guy, you generally should go out of the way to make sure you are in position to make a good holding of the door, and not in a good position to make it difficult for her to get by without rubbing against you.