Ask anyone for an example of an urban Black community in America and the first name that’ll probably come to their minds is “Harlem.” The area of Manhattan north of Central Park has been arguably the best-known Black neighborhood in the U.S. for about a century now, dating as far back as the Harlem Renaissance of the 1920s and ‘30s.
What’s much less known about the neighborhood, however, is the story of how it became a Black cultural mecca in the first place. It’s a story of entrepreneurship, vision, and grassroots solidarity in the face of racial oppression — and some cut-throat business tactics and possible fraud.
Originally a Dutch colonial settlement named after a city in the Netherlands, Harlem was largely populated by working-class Jewish and Italian immigrants throughout the late 19th century. Yet a crash in the real estate market and racial violence elsewhere in the city—such as the Tenderloin District race riot of 1900—prompted the beginning of a mass migration of Black tenants uptown.
Handyman-turned-entrepreneur
Enter Massachusetts-born Philip A. Payton, Jr., arguably the founding father of Harlem as we know it. Payton was a college-educated, self-made man who worked his way up from jobs such as handyman and barber to become a successful real estate entrepreneur. At first, Payton worked for white landlords in Harlem, offering to fill their buildings with Black tenants and then manage the buildings for them.
A series of incidents—from business disputes between rival white landlords to a racial brawl between Black and white residents on West 130th St. on Christmas Day 1901—showed Payton that the real estate industry cried out for someone who could navigate tensions.
Determined to create a safe space in the city for Black residents, Payton founded the Afro-American Realty Company in 1904. He raised the investment capital for the venture by lobbying the Black elite from that era, including the Rev. William H. Brooks and newspaper publisher Fred R. Moore. In today’s parlance, Afro-American Realty was arguably a “social enterprise”: a business that works not only to make money, but to tackle community and societal problems as well. The company’s prospectus proclaimed, “The very prejudice that has heretofore worked against us can be turned and used to our profit.”
The rise of Afro-American Realty
Afro-American Realty started leasing homes in Harlem to black tenants, using billboards and subway ads to promote its housing offerings. By the end of 1904, Payton managed 12 buildings and owned 4 tenement buildings lock, stock, and barrel, even though the economy was going through a slump at the time.
By enticing Black tenants from other New York neighborhoods into Harlem, Payton began to attract negative attention from the city’s dismayed real estate establishment. One newspaper, the New York Herald, called it “an untoward circumstance.” A real estate publication, the New York Indicator, bluntly declared, “Their presence is undesirable among us. [...] They should not only be disenfranchised, but also segregated in some colony in the outskirts of the city, where their transportation and other problems will not inflict injustice and disgust on worthy citizens.”
So the establishment struck back. A white-owned firm called the Hudson Realty Company set about buying up nearby land for residential development. Hudson Realty purchased three of Afro-American Realty’s buildings—40, 42, and 44 West 135th Street—and immediately evicted all of the buildings’ tenants, all of whom were Black. It then replaced them with white tenants and instituted racially restrictive housing covenants that required willing developers to rent their properties exclusively to whites.
But Payton didn’t take this provocation lying down. In a ballsy counter-move that The New York Times called a “real estate race war,” Afro-American Realty bought two nearby apartment houses, promptly evicted their white tenants, and opened their doors to the very same Black tenants Hudson had kicked out. In the end, Hudson Realty had to sell the original three buildings back to Payton’s company at huge losses.
This “Negro invasion” (also in the Times’ words) also caused an outmigration of white Harlem residents, in a pattern that would repeat itself in cities across America for the rest of the century.
Payton’s undoing — and legacy
Of course, this was no philanthropic endeavor. Afro-American Realty Company swiftly accumulated $1 million in assets ($34 million in today’s dollars, not taking into account property value appreciation), with rent revenue of $114,000 per year ($3.8 million in today’s dollars).
Business grew at a healthy clip, partly because Payton followed the common industry practice of charging Black tenants more than white ones — as much as $30 a month for a 4-room tenement apartment, which was $5 per month more than white tenants had been paying. Payton defended this practice on the grounds that banks charged him higher interest rates than white landlords and developers had to pay because he rented entirely to Black tenants.
There were also accusations of shady practices at the company. In October 1906, 35 shareholders brought a lawsuit, accusing Payton of fabricating the company’s prospectus, exaggerating the company’s actual assets at the time it was issued. Bizarrely, the business associates had Payton arrested as part of the case (in the U.S. you are arrested for criminal charges, not as part of civil lawsuits). The court ruled in favor of the plaintiffs later that year.
As it turned out, Afro-American Realty’s first dividend, issued to shareholders in June 1907, would turn out to be its last. The company never bounced back from the bad press that stemmed from the lawsuit, or from an economic crisis of 1907. It closed shop in 1908. Payton himself never entirely recovered from this fiasco, either, but he managed to keep working as a real estate broker until he died of liver cancer in 1917.
Nonetheless, Payton’s efforts kick-started the process that made Harlem the Black cultural capital that it has been for almost a century. In the ensuing decades, the neighborhood would give rise to a flowering of renown Black writers, musicians, political activists and business leaders — which ultimately helped fuel the Civil Rights movement.
Today, Harlem is undergoing another transformation. After struggles with urban decay and high crime in the 1980s and 1990s, now gentrification and skyrocketing housing prices threaten to undermine the cultural identity of the neighborhood. According to newly released data from last year's census, for instance, Harlem gained more than 18,000 white residents and lost more than 10,805 African-Americans and more than 2,000 Latinos between 2010 and 2020. But remembering figures like Payton helps to preserve the neighborhood’s heritage.
