Chances are, we are probably not even celebrating the correct day anyway.
In the third century, efforts were made to find out the date of the Nativity, but only in the year 336 was the date of the December 25 festival set in commemoration of Jesus' birth. Pope Julius formally selected December 25 as the day of Christmas in 349 A.D.
Part of the reason for placing Jesus' birth on the 25th (of December), was because there was an ancient Jewish belief that the great prophets were to have an "intergral age" (where you die on the same day as either your birth or your conception). It was believed that Jesus was conceived on March 25 (according to some historians, such as Sextus Julius Africanus), so a nine month count would place his birth on December 25.
There is no real basis to the belief that the December date was chosen by the church to "muscle-out" the existing Roman Saturnalia festival or other Pagan winterfests but, the myths persist.