The Glyph Code: When Cities Use AI to See Their Scars

Introduction: Languages the City Writes in Secret

Before the sun rises in São Paulo or Lisbon, the walls already speak. In alleyways, underpasses, and above street lamps, cryptic messages crawl across the concrete—names, codes, affiliations. São Paulo has pixação—tall, black, spiked glyphs etched in impossible places, a vertical scream of the invisible classes. Lisbon has sprawling tags and pastel throw-ups, painted in quick movements behind corners and bus stops, defying the city's self-image as postcard-perfect.

But now, both cities are trying to read what was never meant for them.

Across the Atlantic, engineers and municipalities are building tools that use artificial intelligence to scan, classify, and geolocate street markings in real time. It’s a leap in urban management—and a potential blow to cultural resistance.

Because the real question isn’t whether machines can recognize graffiti.
It’s what happens once they do.

São Paulo: Learning to Detect Dissent

In 2018, a team of researchers in São Paulo developed a pioneering AI system capable of detecting pixação in Google Street View images. Using a dataset of just 632 annotated examples, they trained a Mask R-CNN model with a ResNet-101 backbone—a setup more commonly seen in autonomous vehicles or facial recognition systems. Despite the limited training material, the AI reached a mean average precision of 0.57.

Technically, it worked. Sociologically, it was loaded.

The goal? To automate the creation of pixação maps for urban planners, city governments, and clean-up crews. The logic? Manual inspection was expensive and time-consuming. The consequence? A tagging culture born out of marginalization was now being reduced to data points in a bureaucratic dashboard.

Pixação isn’t just graffiti. It’s protest in code. It’s territorial, performative, cryptic—illegible by design. To detect it is to surveil it. To map it is to prepare to erase it.

Lisbon: Real-Time Recognition, Real-Time Risk

Five years later, Lisbon raised the stakes. In 2023, a team of researchers, supported by the Lisbon City Council, developed an AI pipeline that goes beyond detection: it can classify images into three categories—legal street art, illegal graffiti, and clean walls—then geolocate the result in real time using video feeds from municipal cars.

According to their report, the system achieved 81.4% overall classification accuracy, with an F1-score of 86% for street art and 81% for illegal graffiti. It’s a major leap in functionality—and a direct application of AI into daily governance. When a tag is flagged, it doesn’t end in a spreadsheet. It ends with a clean-up order.

The project also includes the development of a “city graffiti database,” which could be used by researchers, policymakers, or—hypothetically—artists themselves. But the line between civic innovation and cultural erasure is razor-thin.

Lisbon, already obsessed with aesthetic cleanliness in public space, is now algorithmically sanitizing its walls.

Two Cities, One Anxiety

São Paulo’s system was academic, slow, and partial. Lisbon’s is fast, operational, and embedded into city infrastructure. But the underlying anxiety is the same: who gets to write on the walls, and who gets to decide what stays?

There is an eerie symmetry here. Both cities deploy AI to handle the messy, layered presence of popular expression. Both train machines to "understand" what urban elites often treat as visual pollution. And in both cases, the cultural stakes are high: graffiti, pixação, and tagging are not random acts of vandalism. They are manifestations of exclusion, inequality, and voice.

Yet while São Paulo’s model struggles with accuracy and nuance—especially given the stylized, spiky aesthetic of pixação—Lisbon’s system brings automation into full motion. The risk is no longer misclassification. It’s overreach.

Archives or Arrests?

Not all outcomes are dystopian. There’s a flicker of potential—if the tech is used to document, not erase.

Lisbon’s open dataset offers a strange new possibility: that AI could help preserve a disappearing visual culture. That it could identify the evolution of tagging styles, track the emergence of new street calligraphies, or even help taggers archive their legacy before gentrification wipes it clean. São Paulo’s earlier efforts, too, could evolve into tools for understanding the geography of discontent—where the glyphs gather, and why.

But that would require a shift in perspective. A move away from graffiti as nuisance and toward graffiti as social mirror. From punishment to preservation.

And most importantly: it would require putting these tools into the hands of those who write the city, not just those who govern it.

Who’s Reading Whom?

Urban AI does not exist in a vacuum. It is built from data labeled by humans with agendas, assumptions, and institutional biases. Every image flagged as “illegal graffiti” is a judgment call. Every cleanup triggered by a pixel threshold is an act of violence against someone’s expression.

São Paulo and Lisbon are writing the future of graffiti policy with neural networks. But they are also revealing a deeper truth: that technology doesn’t see. It categorizes. And what it cannot understand, it may delete.

The artists of the street have always known this. That’s why pixação looks the way it does. That’s why taggers use codes and aliases and paint unreachable places. Not to decorate, but to defy.

If AI is to see them, it must first be taught to look—not just for lines on a wall, but for meaning in resistance.

Postscript: In both cities, walls speak. But now, the machines are learning to listen. The question is whether they will hear a cry for presence—or issue an automated silence.