There's a famous quote by H.L. Menken that Puritanism is "the haunting fear that someone, somewhere may be happy". While their doctrines and daily lives did emphasize work over play, Puritans were not the dour fun-haters they have been portrayed as recently. In fact, they engaged in a lot of recreational activities including sports, visual arts, literature, and music, and saw these pursuits as necessary for reinvigorating both body and spirit.
Puritans particularly enjoyed spending time outdoors. Some of the more popular diversions were hiking, picnicking, and fishing. Hunting was seen as necessary for sustenance, but discouraged as a recreational activity. This was an aspect of their general discouragement of any sort of excess.
Like many people today, they condemned violent sports such as boxing and cockfighting, primarily due to objections over causing harm to God's creatures. Some disparaged bowling due to the gambling that frequently happened on the side, and tennis because of a perceived association with Catholic monks, but others said the games were just fine in and of themselves. The main concern among Puritan leaders was that these pastimes not interfere with people's work or prayer, which should always come first.
New England Puritans were a remarkably literate group for their time. They strongly encouraged congregants to read and write, particularly religious materials. There is a wealth of sermons, personal diaries, and even poetry that has survived to this day, not only because they wrote so prolifically, but also because these materials were often published for others to study in their leisure time.
There is considerably less visual art, since their pragmatic ways made for simple decorations. Homes had little in terms of embellishments, but some Puritans did paint or draw. Women's handicrafts mostly produced practical items like clothing and quilts, but there was also the occasional embroidery project.
Although Puritans objected to the use of music in church as distracting and "Popish", they frequently enjoyed singing and playing instruments in the home. Similarly dancing was all about context; many denounced "promiscuous dancing" (i.e. both sexes dancing together) as they felt it could lead to fornication, but folk dancing that avoided physical contact between men and women was generally permitted.
If you'd like to learn more about how early Puritans truly lived, check out Puritanism: a Very Short Introduction by Francis J. Bremer. It's a quick, entertaining read, and contains a great list of suggestions for further study if you want to get more in-depth on any of the subjects it covers.