Adversarial Style Modification: Probing the Human–AI Perception Gap

We often recognize a writer's style before we can explain what makes it distinctive. A characteristic rhythm, choice of words, or way of constructing sentences can be enough for us to recognize an author's voice. Computational models can also identify authors from patterns in their writing, often with remarkable accuracy. Yet human perception of authorial style and computational authorship attribution have largely been studied separately.

But how closely are the two connected? If a text is modified until a model can no longer identify its author, will human readers lose recognition at the same point?

Our shared task, Adversarial Style Modification: Probing the Human–AI Perception Gap, investigates this question experimentally. Participants modify texts to reduce automatic author recognition while preserving the author's style for human readers. Rather than treating adversarial modification solely as a way of fooling a classifier, we use it to probe human and machine sensitivity to stylistic change.

Understanding this relationship matters in at least two contexts. For authorship obfuscation and privacy, transformations that successfully conceal identity from automatic systems may still leave users identifiable to human readers. Conversely, creative protection may require the opposite: preserving an author's distinctive style for human readers while making it harder for models to learn, imitate, or reproduce that style.

To the best of our knowledge, Adversarial Style Modification is the first benchmark specifically designed to quantify the relationship between automatic authorship recognition and human perception of authorial style under adversarial modification.