Ill draw in coffee shops quite often. I think I've gotten used to it since I go to the local drink and draw, thou it is much more comfortable to draw with a group in public than alone.
I am definitely more uncomfortable when I am out in public by myself in a crowded area.
I have always wanted to life draw buildings since architecture and street design is probably one of my biggest faults as an artist, and trying to capture it in photography is not my strong suit. Its never exactly what I wanted to draw (and taking photos in public also makes me self conscious). But doing it in public makes me uncomfortable. If i could find a nice non crowded coffee shop or a quite place in a park, that would be my ideal outside drawing space.
I also think just a mixture of the area I live in, sort of looking like an intimidating person, and putting on headphones, people are less likely to mess with me when I am drawing in public. Most of the time if someone goes out of their way to comment, its just to say "thats really cool" and then they let me get back to my work.