As I understand it, it depends on several issues. One is your personal "fear factor". If you're comfy with your tap water, then it's probably okay to use it straight out of the tap.
Another factor is how the tap water is purified. While Chlorine can be evaporated out of water fairly easily, many cities use another chemical (I don't remember the name at the moment) which does not dissipate that way.
So, you would need to contact the group that handles your city's water to find out what they use.
I don't know how it is in the rest of the world, but here, we have "water stores" that sell water which has been put through a reverse osmosis system to purify it. I think it works out to something like 15 cents a gallon for us.
You can also get one of those charcoal filter systems that either hook to the tap or are in a pitcher.