Across a career that began in a rundown West Street building thick with dirt, cigarette smoke, and broken dreams, Pete Hamill has written a million or so words in newspapers and books.

His crisp, yet poetic style has inspired generations into journalism, and has drawn rebukes from offended politicians like Spiro Agnew, a disgraced vice president who called his columns “irrational ravings.”

Through it all – the bars, the wagon, the high-profile women – Mr. Hamill, 69, has been chasing ghosts.

They are not just the young ghosts from his Brooklyn childhood – his parents, a first love, or the Dodgers – but also the shimmering, barely visible ghosts of the men and women who landed on an island called Manna-hata from the early 1600s.

Many were Dutch or English, more than a few were slave traders, and the first colonial governors were stupid, corrupt, or both. But they laid the foundation for what would become the ever-changing landscape of New York City – specifically the lower part of Manhattan. Waves of immigrants would change it, destroy it, and rebuild it.

Mr. Hamill’s ghosts, from Peter Minuit to the waves of European immigrants in the late 1800s and early 1900s, walked the streets of Five Points, fashioned a wide boulevard that would become Broadway, and built a vibrant financial center near the old Wall Street.

Many came ashore at the Battery and Mr. Hamill’s 20th – and latest – book, “Downtown: My Manhattan,” starts us there on a magical tour of the beginnings of our city, good and bad.

“In some ways, it took me all my life” to write this book, Mr. Hamill, thin, bearded and graying, says from his writing hideaway in Cuernavaca, Mexico.

“This is not a book I could have written when young. I had to live long enough to let the city seep into me, to reveal itself in small ways, and to see the cycle of decay and revival working their mysterious ways through our lives.”

It is a book that chronicles generations of brave, visionary, nasty, and colorful rogues, merchants, gangsters, showmen, and builders.

There’s Henry Hudson, who sailed his Halve Maen (Half Moon) up the North River, and the flamboyant British governor, Lord Cornbury, who pranced around the ramparts of an old fort in drag in the early 1700s and had himself painted as Queen Anne.

There’s Captain William Kidd of Pearl Street, who provided material for the original Trinity Church and who would be hanged in London in 1701 for piracy, and John Jacob Astor III, fur trader, opium trafficker, and the first real estate king of New York.

There’s Five Points gang leader Paolo Vaccarelli, better known as Paul Kelly, whose legendary two-hour fight with Jewish gangster Edward Osterman – Monk Eastman, to you – ended in a draw in a Bronx barn.

There’s August Belmont, a 19th-century German Jewish banker who raised his children as Episcopalians and would have a racetrack named after him.

And there’s Alexander T. Stewart, a 5-foot-tall Irish immigrant who opened the first department store in the city on Broadway between Chambers and Reade streets. First called the Marble Palace, it became known as The Sun Building in 1917.

“My favorite place is the Battery,” Mr. Hamill says. “When the light trembles on the waves of the great harbor, I can think of those mad Dutchmen coming into the harbor after 4 1/2 months at sea…

“I also can see that French woman with her hand raised high, holding a torch, and what it meant to all those who sailed past….All those Irishmen and Jews and Italians and Germans who joined with the Africans and the Knickerbockers to create the New York tribe. My tribe. Your tribe. Invincible.”

It is the place he returns to most often to chase his ghosts.

“I keep going back there because my mother and father were among the many millions who sailed that bay and found safe harbor,” he says. “They all went through hard times, discriminations of various kinds, poverty, danger. But they gave much more to America than they took from America.”

His ghosts stretched into his lifetime and, like all ghosts, they have vanished. “I miss the old Penn Station. I miss Ebbets Field and the Polo Grounds,” he says. “I miss the Singer Building, and the old Italian guys who sold ices on the streets in August.”

All are gone and much of what we know now will be gone when our grandchildren have grown. Read the book, take walks, chase some ghosts, and pass them on.