If cracks or gaps are present in the building envelope then warm air leaks out and cold air can infiltrate into the space. Leaks can add 10% to your energy costs and are a major cause of heating energy waste in brownstones.

Below is a Department of Energy breakdown of where most leaks occur in a home.

Sources of Air Leaks in Your Home:
 1 Dropped ceiling  7 Door frames
 2 Recessed light  8  Chimney flashing
 3 Attic or crawl space entrance  9 Window frames
 4 Sill plates  10 Electrical outlets and switches
 5 Water and furnace flues  11 Plumbing and utility access
 6 All ducts  

You can find leaks by holding a lit incense/smoke stick around windows, doors, electrical outlet and switch boxes, plumbing stacks and spots where pipes enter the walls, ceiling fixtures, vent holes air conditioning or exhaust vents, and other areas where it seems there might be a seam. Leaks will be obvious as the smoke will flatten and move with the breeze. To increase the effectiveness turn on all exhaust fans which will have the effect of pulling air in through any cracks – kind of like a mini blower door test. The most accurate results can be obtained from a full scale blower door test.

You can seal leaks by caulking or inserting expanding foam (for cracks larger than ΒΌ-inch) or cotton wool around things that don't move (outlet and air conditioner covers and window frames, plumbing chases and attic hatchway for example), or by weatherstripping things that do move (like windows and doors).   See our Article  Window, Skylight and Door Basics -- Increasing Energy Efficiency for a specific discussion of window-related issues.