First of all, we could have extravagantly better Election Day parties if we had the whole day off. Second, I hypothesize that it would increase the percentage of the eligible public who vote. With the whole day off, knowing why you have the day off, and not having to take off from work, the impetus to vote would be greater.
Of course, some would argue that increasing the percentage of people who vote is a bad idea, but we have a name for those people: Republicans, as poor people vote far less than do wealthier people. (Actually, 86% of registered voters voted in the 2000 elections; we should really keep focusing on increasing voter registration. But I still want a day off.)