Could be a few reasons. If you are using an existing chimney then you need a register plate to block it off at the base. If this doesn't work well then you also need a flue liner. The top of the chimney should have a tin hat to keep rain and hail out, but if you live in a home close to a cliff or steep hill then you can get downdraught problems at night. Fitting a different chimney will help but you need advice on which type.

All fires will have a downdraught to start with when lit as the hot smoke cools in the cold chimney. The trick is to get a fast burning fire first to warm the chimney and get the smoke going up.

Is it fitted with a chimney cowel?