A woman in a century-old straw hat, sitting with a needle and thread, warns me “the sun can crisp you up.” Nearby another woman, speaking at first in Hebrew, says, “My husband came from Ukraine in 1903. I moved here in 1904.” Afterward I chat with a cooper with sharply parted white hair. He’s proud of a recent deal he made: a barrel for two brooms.

No one ever tells you how to respond to this stuff. You know, these sorts of costumed role-players or living historians. Here at Strawbery Banke—a charming collection of a few dozen colonial-era buildings in Portsmouth, New Hampshire—I’m finding it’s rewarding to just play along.

Settled in the early 1600s, Portsmouth might be America’s greatest small town. The port town was built on the Piscataqua River for exclusively the king’s commerce. On the wharves, workers built British ships—ships later used against the colonies in the Revolutionary War. In the years since, it simply never expanded like Boston, or even Portland, Maine, just north. (The tidal river isn’t ideal for constant comings and goings, one local tells me.) Still, there’s lobster and sand, history and theater, New Hampshire’s best restaurants and lots of beer. And walking by 18th- and 19th-century town houses along its lively cobbled center, it’s hard to believe only 21,000 people call it home.

Opened in 1958, the Strawbery Banke Museum is considered the first urban renewal project in the United States to favor preservation over demolition. (“Its impact ripped through the preservation movement like a sound wave,” Linda Landry writes in her 2003 book, Classic New Hampshire.)

Frankly, I enjoy its colonial-era punch of the few dozen buildings more than I expected. The Shapley-Drisco House is a time-warp peek split evenly between its different incarnations: One side of the house is made up like a home in the 1790s, the other like the 1950s.

And in the Pitt Tavern here I learn about Portsmouth’s other big passion. A gray-haired man with white loafers and a crimson shirt laments to me how New Hampshire used to be “number one in beer consumption” (42.2 gallons per capita in 2013) but recently fell behind North Dakota.

“That’s only because of all the petroleum going on there,” he says with some bitterness.

Yes, beer is big in Portsmouth. A handful of breweries line the winding central streets (my hotel, the Ale House Inn, is a former brewery), while the bigger-name Smuttynose and Redhook breweries are within 20 minutes’ drive. One of my favorite spots for a pint in the center is Book and Bar, a used bookstore in the former U.S. Custom House that’s now licensed to sell microbrews.

On a back alley, Earth Eagle Brewings is a compact bar-eatery with playful artwork and nine tables that quickly fill with locals. The focus is experimentation by beer, mostly local microbrews. The Spruce Bringsteen is a sweet pale ale, while the Beltane, an amber gruit, has “a real barnyard quality, think blue cheese,” says my waitress.

I get her barnyard description once I try it, and their “figgy pig” special (a flatbread sandwich with Gruyère, bacon, porchetta, and fig jam) is my favorite meal I’ve had in New Hampshire so far.

Portsmouth’s a great walking town. On my first day, I walk down quiet streets with Georgian homes (five windows on the second floor, two on either side of the front door) and stop in Sheafe Street Books, complete with a “pirates” section. The owner, an upstate New York transplant, sends me to nearby Peirce Island, where I see people wandering the riverside trails.

In the morning I join a walking tour led by Jeff Thomson, a local from just across the river in Kittery, Maine. One stop leads us to something I missed on my first day’s ramble. A short walk from John Paul Jones’s old home, the African Burying Ground is the site of a slave cemetery only discovered during road construction in 2003, and it is now a moving memorial. Part of it features eight faceless figures. “I stand for those who feel anger,” one reads. “I stand for those who find dignity in these bones,” reads another.

Thomson notes, “We don’t always like talking about it, but we must remember slavery existed here too.”

Another big draw, of course, is the coastline. New Hampshire’s doesn’t last long—only 13 miles (21 kilometers)—but is lined with sand and rocky beaches that draw sunbathers and surfers. Quieter Rye, just outside Portsmouth, is home to mansions you can visit. For me, the best part of the coast is the lobster. You find it everywhere, at fish shacks on the beach and restaurants in town.

A friend of a friend, Robert Nudd, invites me for a lobster dinner at home with his wife, Sheila. His family has been here since the 1600s, and he’s been fishing, sometimes for 60 days straight, for four decades. His crew sometimes ribs him for holding onto pre-digital technology, he says, and he can’t imagine doing anything else. Today he’s caught dozens of lobster, and is cooking up way too many for our dinner.

Lobster is big up and down New Hampshire’s wee coastline. Shacks and sit-down restaurants boil up fresh catches.

Get that? Portsmouth is a lively colonial town that’s escaped most traveler radars. It has beaches and beer and theater, and its lobster is delicious.

HOW TO DO THIS TRIP

Stay: Ale House Inn has compact boutique rooms within walking distance of all the center shops, restaurants, theaters, and sites.

See: Strawbery Banke is a district of a few dozen homes with costumed role-players. The $19.50 admission lasts for two consecutive days. The bulk of the sandy beaches are in Hampton, within a 20-minute drive. They get busy.

Do: Discover Portsmouth hosts several themed walking tours. For $12, I got a lot out of the 60-minute daily walking tour. You can kayak by islands in the Piscataqua; Portsmouth Kayak Adventures rents kayaks and offers tours.

Drink: Book and Bar is a used bookstore with beer, open late. Earth Eagle Brewings on a back alley is worth the chase.