Plato

What does it mean to know something?

2,400 years ago, Plato worried about a new information technology: writing.

In the Phaedrus, Socrates tells of the god Theuth, who presents writing as a way to make people wiser and improve their memory.

But King Thamus replies: On the contrary. Writing will make people worse at remembering. And it will give them the appearance of knowledge without real knowledge.

They will be able to repeat things without knowing them themselves.

It sounds strangely contemporary.

Because what happens when the written word can not only be stored outside of us, but also be produced by a machine?

Knowledge as practice

What does it mean to know something?

For a bioscientist, the written word has never been the knowledge itself.

To know something about a bacterium or a virus is to be able to work with instruments, samples and data. To know when an experiment has gone wrong. To see that something strange has happened. To be able to tell what is noise and what is interesting.

The text is a way of reporting what you have found.

But what happens when the same starts to hold for the journalist? The programmer? The researcher? The student?

When the text is no longer proof that someone knows something?

The industrialization of text

What does it mean to know something?

We have already started to mass-produce text.

In the 19th century, textile production was industrialized. Production moved from the craftsman to the factory.

Now text production is being industrialized.

But the factory is not in Göteborg or Norrköping. It is in the cloud.

Text that used to be produced one piece at a time—by journalists, communicators, programmers, researchers—can now be produced on an industrial scale.

What happens then to the relation between the worker, the knowledge and the text that is produced?

The knowledge of the body

What does it mean to know something?

At the same time, something strange is happening.

It seems easier to automate much of what we have called intellectual work than what we have called practical work.

A machine can write an essay on Kant in a few seconds.

But folding a T-shirt is still a hard problem in robotics.

And to teach robots such things, we now put cameras on people who work with their hands, collect their movements, and try to teach the machines to imitate them.

What looked simple turns out to be hard.

What looked advanced sometimes turns out to be easier to imitate.

Hierarchies of knowledge

What does it mean to know something?

We have long had a hierarchy of knowledge.

Thinking has ranked above doing. Theoretical knowledge above practical knowledge. Writing a text above folding a shirt.

But AI does something strange to that hierarchy.

What we have regarded as advanced human knowledge work—writing, summarizing, programming, analyzing—turns out to be surprisingly easy to imitate.

At the same time, it is still incredibly hard to build a machine that can walk into an unfamiliar kitchen and empty the dishwasher.

So perhaps AI does not only tell us something about what machines can do.

What does it mean to know something?

It forces us to ask what we meant by human knowing in the first place.

Plato worried that writing would give us the appearance of knowledge without the knowledge itself.

2,400 years later, we have built machines that are fantastically good at producing exactly this: the appearance of knowing.

And so perhaps the really interesting question is not:

What can AI do?

But:

What does it mean to know something?

And perhaps we have valued the wrong things.

What does a method reveal and hide?

STS was built on a reaction against an overly stable account of science and technology. Our project was to show that choices were made, and that things could have been otherwise.

So how does STS react when the field tells us that AI is a black box? How do we disentangle the thingness of AI? When we look closely, we see how people struggle to put these technologies together. And we see how people struggle to integrate them into their work practices.

Where do we enter?

What is the entry point for studying AI systems? Lotta Björklund Larsen and I asked a version of this question in How should we theorize algorithms? Each entry point shows something different:

• Going under the hood—algorithmic ethics, auditing—shows us injustices, but not how they came about.
• Looking at the interactions around AI shows the invisible work of making AI work with us: perhaps all the human labor it takes to make these “autonomous” machines work.
• Following the actors—the tinkering and work of constructing AI—lets us show that people work to make these machines.

Tinkering, and the moments of choice

What kinds of choices are made when AI is built? In The practices and politics of machine learning I tried to write a field guide to the moments of choice and conflict in building AI systems.

It is not an innocent choice to vectorize. Or to choose features. Or to decide on temporalities. Or to assume a Gaussian curve. When we follow the construction of AI, we can show just how many choices are made along the way. The papers in the panel gave lovely examples of exactly this.

The politics of absence

But where is the politics of choice? One answer is to ask what is made absent. In Ontological overflows and the politics of absence I traced how absences are made—and layered—in the systems we use to construct pandemics. The same thing happens when synthetic datasets are constructed: the tails of the distribution are cut off. Saghi Hajisharif, Ericka Johnson and I show how in The ontological politics of synthetic data, where the outliers disappear and intersectional hallucinations appear.

The epistemic trap of the vernacular

Our own concepts can be traps too. With David Ribes, in Computational universalism, and in conversation with Lucy Suchman’s piece on the uncontroversial “thingness” of AI, I have been thinking about concepts as epistemic traps. In Enacting the pandemic the trap was “the algorithm.” Today it might be “ground truthing” or “datafication.”

When we take up the vernacular of the field, do we risk stabilizing the very things we want to show could be otherwise? Talking about ground truths stabilizes the notion that there are ground truths. Naming invisible work, or a creole or pidgin, is also a way of seeing—emic and etic categories do different work.

What do we win and lose?

So what do we win and lose when we choose a particular method, or a particular theory? How can we stay with the original political project of showing that things could be otherwise?

Is it worth staying with? I think so! The point is that we have a choice—to make other choices—when AI is made and stabilized. Then we have to reflect on what our methods allow us to do. So that we can show that it could be otherwise. And make a better AI society.

What is the relationship between the world, data and representation? That question has followed me through two projects on synthetic data: Social Complexity and Fairness in Synthetic Medical Data, with Ericka Johnson, Gabriel Eilertsen, Saghi Hajisharif and Tahereh Dehdarirad, and now the WASP-HS research group Synthetic Data: Facts, Representations, and Transparency, with Ylva Söderfeldt, Ericka Johnson and our team.

Wait… what?

In a recent paper, Dietmar Offenhuber writes that with synthetic data “we are confronted with data that does not involve any representational relationships.”

Wait… what?

Researchers have shown that diffusion models can reproduce images from their training data—almost pixel for pixel. Put the original next to the generated image, and it is hard to say that there is no representational relationship. So what is the relationship between the world, data and representation? And what does it mean to represent something?

Original training images and images generated by a diffusion model. From Carlini et al. 2023, “Extracting Training Data from Diffusion Models.”

Philosophers talk about truthlikeness, or verisimilitude: how close to the truth something is. (Thanks to Tanya Osborne at Chalmers for this line of thinking.) But STS has a different starting point.

Data are made in practice

STS has long shown that data are fundamentally made in practice. Or, as Lisa Gitelman put it, “raw data” is an oxymoron. A few classics show how:

• Seeing. John Law and Michael Lynch showed how bird watchers learn to see the world and tell objects apart with lists and field guides. Which representations of the world are best to see with?
• Data and evidence. Karin Knorr Cetina and Klaus Amann showed how scientists fix visual traces as evidence. In the lab, autoradiographs are cut, pieced together, labeled and pointed at before they become evidence.
• Sublata. Bruno Latour followed soil scientists in the Amazon, from world to samples, traces, measurements, inscriptions, figures and arguments. Representation is made through chains of transformations.
• Capta. Johanna Drucker reminds us that data contain epistemologies: data are given, capta are taken.
• Crafting. Janet Vertesi showed how images are crafted to see specific things.

So instead of asking whether synthetic data represents the world, we can ask: how is synthetic data made to represent the world in practice? We can sketch its chain of transformations—and ask where the human is in the loop.

Without pretraining, there is only noise

Try it yourself. Prompt GPT-2 without pretraining with “The meaning of life is:” and you get a random string of tokens: “sinners blows1500 ancestry rarelections WinnerLEY wreckage Salary…” The pretrained GPT-2 answers with a sentence: “to make a distinction between things we want to be, and people we cannot make real…” The same goes for image models. Stable Diffusion without pretraining gives you noise. Pretrained, it gives you a futuristic city with neon lights.

The world is in the data. Pretraining is a step in the chain of transformations—not a break in it.

Model cultures and architectural assumptions

The chain is also shaped by the models themselves. As one computer science professor told me in an interview: “Certain groups specialize in particular architectures, become proficient in them, and then continue working with those.” And: “Different architectures are suited to different types of data… convolutional neural networks… [are] developed on the basis of certain a priori assumptions that images have a specific structure.”

Model cultures and architectures carry assumptions about what the world looks like.

Ground truthing

And then there is ground truth. Florian Jaton has shown that ground truthing is a politically and technically organized agreement about what will count as reality. Synthetic data does not escape this. It is one more place where such agreements are made.

Two words and some questions

So: synthetic data. Two words, and some questions to move forward with:

• How do we teach generative AI to see phenomena?
• What chains of transformations does synthetic data involve?
• How is synthetic data stabilized as evidence?
• How is synthetic data crafted to show certain things?
• How do model cultures and architectures shape synthetic data?
• How does synthetic data circulate and gain authority?

I take up some of these questions in The ontological politics of synthetic data with Saghi Hajisharif and Ericka Johnson, and in The practices and politics of machine learning. I have also argued with Offenhuber at more length in another post.

References

Amann, K., and K. Knorr Cetina. 1988. “The Fixation of (Visual) Evidence.” Human Studies 11 (2): 133–69.

Carlini, Nicholas, Jamie Hayes, Milad Nasr, et al. 2023. “Extracting Training Data from Diffusion Models.” arXiv. https://doi.org/10.48550/arXiv.2301.13188

Drucker, Johanna. 2011. “Humanities Approaches to Graphical Display.” Digital Humanities Quarterly 5 (1).

Gitelman, Lisa, ed. 2013. “Raw Data” Is an Oxymoron. Cambridge, MA: MIT Press.

Jaton, Florian. 2021. The Constitution of Algorithms: Ground-Truthing, Programming, Formulating. Cambridge, MA: MIT Press.

Latour, Bruno. 1995. “The ‘Pédofil’ of Boa Vista.” Common Knowledge 4 (1): 144–87.

Law, John, and Michael Lynch. 1988. “Lists, Field Guides, and the Descriptive Organization of Seeing.” Human Studies 11 (2): 271–303.

Offenhuber, Dietmar. 2025. “Synthetic Data and the Shifting Ground of Truth.” arXiv:2509.13355. https://doi.org/10.48550/arXiv.2509.13355

Vertesi, Janet. 2014. “Drawing as: Distinctions and Disambiguation in Digital Images of Mars.” In Representation in Scientific Practice Revisited, edited by Catelijne Coopmans, Janet Vertesi, Michael Lynch and Steve Woolgar. Cambridge, MA: MIT Press. https://doi.org/10.7551/mitpress/9780262525381.003.0002

ground truth becomes a self-referential affair, in which the labels used as a ground truth repository are themselves synthetic products of a generative model and as such not connected to real-world observations.

In other words: synthetic data has no real-world referent. I disagree, strongly.

Synthetic data without a referent is noise

If there were no referent, why are there women, doctors and educational levels in synthetic datasets? Without a referent, they would not be there. Synthetic data without training data—without a referent—would become what? Noise. That is precisely what a randomly initialized transformer outputs: noise, with the biases of the transformer architecture.

Saying that synthetic data has no referent ignores the theoretical insights and methods that STS has developed over the past decades. What we are dealing with is not a break in the chain of reference, to borrow Bruno Latour’s notion of circulating reference. It is one more step in the many transformations from world to data. So instead of making weirdly realist points about synthetic data, STS should ask how these chains of transformation work: real → data → synthetic data.

Breaking the world, and mimicking it

We need to back up a few steps and ask what computer scientists are doing with synthetic data. Synthetic data is not always made to be faithful to the world. Sometimes it is made to make a model work better—by throwing weird data at it. That is not a representational practice. It is a design practice, where synthetic data is used to shape the performance of a model.

At one point, computer scientists are testing and developing a model: testing edge cases, outliers, weird data. The version of the world they construct is one where extremes are what is sought after. Let’s break the world to test the features of the model! But even this is not without referent or representation. It is not cosmic noise they throw at the model. It is data purposefully designed to push the limits of what a machine learner can predict, generate or steer.

At other points, synthetic data is made to represent the world. But there are purposes here too. We want to mimic things: a text by a specific author, an X-ray image, a population. Here we can discuss what kinds of fidelity are sought after. But saying that the data is disconnected from ground truth is not very analytical. It pays no attention to all the work that goes into making a machine learner output data that actually looks and feels like the world.

Translations and treasons

We can absolutely discuss translations and treasons. What is deemed worthy of representation in synthetic data? Which relationships are preserved? Which aspects of the world are made to travel? And conversely: what is not represented? What transformations of the real data are made? How is it tested?

Perhaps a better question is about mimesis—how the world is represented and imitated through computational transformations. That is something Offenhuber discusses, and should take seriously.

How is synthetic data valued?

There are many questions to ask about synthetic data that don’t fall into a realist trap. We can attend to the tinkering with synthetic data in practice: how it is constructed, and how fidelity is valued differently in different places. As Offenhuber observes, synthetic data is judged by

how well it preserves privacy in a particular application, or the quality of predictions of a model trained with it.

These are questions of valuation and evaluation, rich with social practices, metrics, yardsticks and actors that can be fruitfully untangled. And are they so different from the questions laboratory workers face every day? How well does this dataset represent a patient? What sources of noise are there? Is there enough metadata to describe the dataset?

Offenhuber also writes that “data quality can no longer be considered in relation to reality.” But why would the actors not consider synthetic data in relation to reality? Depending on what the dataset is for, much blood, sweat and tears are shed by the people who create it. Why would they not want it judged in relation to “reality”?

And when he concludes that “data quality is no longer about what the data represent, but rather, what the data do,” I would say: that depends on what the actors want to do. Data quality can be measured in so many ways. One of the most important is whether the data is useful for its purpose.

Ground like coffee

Then, finally, something I really like. Offenhuber borrows it from a data physicalization project at a complexity science workshop in Vienna:

the ground in ground truth no longer refers to a stable reference, but is ground like coffee, boiled and sent through a filter that selectively withholds components.

Yes! And this is exactly why “ground truth” is an epistemic trap. It makes us think like computer scientists. It traps us in a binary opposition between the real and the synthetic. Let’s not throw out the baby with the bathwater: STS has good tools for following chains of transformation. Synthetic data is one more link in the chain—and the interesting questions are about what each link adds, filters out and makes travel.

With Saghi Hajisharif and Ericka Johnson, I have explored some of these questions in The ontological politics of synthetic data.

Reference

Offenhuber, Dietmar. 2025. “Synthetic Data and the Shifting Ground of Truth.” Talk at the Society for Social Studies of Science (4S) 2025, Seattle. arXiv:2509.13355. https://doi.org/10.48550/arXiv.2509.13355

That is the starting point for Blind Spots in Pandemic Preparedness: Pathogen X and the Development of Epidemic Surveillance after Covid, a new project funded by the Swedish Research Council. I will do it together with Lisa Lindén at Chalmers.

New systems, old assumptions?

In the wake of Covid-19 there has been a major push to build new surveillance systems for pandemic preparedness. Sewage is analyzed to track outbreaks. Bioinformatics is used to identify antibiotic resistance genes. Metagenomic techniques are developed to find unknown pathogens.

These systems promise better preparedness. But studies of past pandemics point to a risk: that new systems reinforce existing biases in what data is collected, which pathogens are detected, and whose health counts. So the project asks:

• Which pathogens are prioritized in emerging surveillance systems, and how does this shape the detection of unknown threats—Pathogen X?
• Which people and places are represented in the systems, and which are left out?
• What do these exclusions mean for pandemic preparedness, for data bias, pathogen detection and health equity?

Where do pandemics come from?

Our preliminary interviews already point to the problem. Informants had strong ideas about where pandemic pathogens would emerge, focusing on China and Africa. Pathogen X, however, might just as well come from thawing “zombie viruses” in the permafrost, or from our own wastewater systems.

This matters because surveillance systems are built on such imaginaries. A focus on particular “virulent zones”—bats or migratory birds in China, for instance—risks erasing the complexity of how diseases emerge.

Ontological overflows

To analyze this, the project builds on a concept I have been developing: ontological overflows. The point is that certain pathogens, populations and places are not just overlooked by surveillance systems. They are excluded—they become overflows in the frame of the system—through its practices, infrastructures and assumptions.

Rather than treating exclusions as accidental, the concept draws attention to how they are assembled: through data scarcity, modeling, risk computation and system design. I first worked this out in a paper on Zika, disease surveillance and mosquitoes, and it builds on earlier work on detecting the unknown in a sea of knowns.

Following the work

Lisa and I will do ethnographic fieldwork at three laboratories and at the Public Health Agency, following three systems: wastewater epidemiology, antibiotic resistance tracking and novel pathogen identification. They correspond to three of the biggest pandemic threats—viruses, bacteria and Pathogen X.

Many of these systems are still being built. That gives us a rare chance to see how assumptions about future pandemics are written into their design, rather than studying systems that are already in place. And, hopefully, to help build systems with fewer blind spots.

More on the projects page.

Infrastructures for the next pandemic

After Covid-19, Sweden has made a big push to build infrastructures to organize, govern and coordinate pandemic preparedness. Sewer water is now analyzed for pathogens to trace epidemics. New bioinformatic techniques identify antibiotic resistance genes. Metagenomic techniques are developed to identify unknown pathogens.

These are systems that shape how we know about pathogens, and what we know about them. And they embody particular values and norms: about what a pandemic is, where it comes from and what preparedness means.

The last pandemic, or the next?

Of special importance is the risk that we are planning for the wrong pandemic, by building infrastructures based on the last one. As Andrew Lakoff puts it, we need to be aware of the assumptions that underlie the tools we have inherited.

So the project asks:

• In what ways do emerging pandemic infrastructures shape knowledge about pathogens and pandemics?
• How are they constructed and implemented—and based on what imagined future pathogens?
• What values and norms shape them?
• Do they lead to new understandings of what pathogens and pandemics are?

Following knowledge through the systems

Lisa and I will use multi-sited ethnography to follow three infrastructures from the laboratories where they are built to the Public Health Agency, where their results feed into decisions.

Every infrastructure shines light on some dynamics of pathogens and leaves others in the dark. Genetic tracking, for instance, emphasizes the evolution of bacterial strains over the pathways of transmission. That leads to particular ways of understanding, and handling, an outbreak.

Where it comes from

The project grows out of my earlier work on disease surveillance at the European Centre for Disease Prevention and Control, in Sensing Salmonella and Enacting the Pandemic. It will run alongside Blind Spots in Pandemic Preparedness: Blind Spots asks what the systems leave out, this project asks how they are built and what they do to our knowledge.

More on the projects page.

The group is funded by the Wallenberg Foundations for five years. Five years is a long time in research, and it gives us room to build something.

What is synthetic data?

Synthetic data is data made by AI models to stand in for real data. Patient records that belong to no patient. Faces that belong to no one. It is used where real data is scarce, private or expensive, and it is spreading fast.

But synthetic data is not just a technical fix. It raises questions about what counts as truth. What does a synthetic data set treat as normal? Who falls outside it? And why should anyone trust data that was, in a sense, made up?

What we will do

We will follow how synthetic data is made and used across fields, and what it does to the knowledge built on it. Bias, representation, legitimacy and trust are at the center. We combine critical algorithm studies, computational experiments and intersectional analysis, and try to build tools and vocabularies for transparency.

The group grows out of an earlier WASP-HS project, Social Complexity and Fairness in Synthetic Medical Data, with Ericka, Gabriel Eilertsen and Saghi Hajisharif. That is where we started thinking about what we call intersectional hallucinations, in The ontological politics of synthetic data.

The group is part of InterSynth Lab, and WASP-HS describes it on its site. More to come.

Beyond the technical fix

Much of the debate about AI and fairness is framed as a technical problem. Make the algorithm transparent. Make it explainable. Fix the math. This is a convenient solution, for ethicists and system builders alike. But it shrinks the problem to the computational realm.

And there are limits to what the math can do. As the mathematician David Sumpter shows in Outnumbered, models struggle with heterogeneous populations. Some problems cannot be fixed inside the model.

Choices built into infrastructure

AI systems are not built from scratch. They are layered on top of other systems, each with its own assumptions and constraints. Kristian Bondo Hansen and Nanna Thylstrup call this stack bricolage.

The “left-pad” incident of 2016 shows what this means. After a dispute with the messaging company Kik over a package name, the developer Azer Koçulu withdrew all his code from the npm software repository. Among it was “left-pad,” a package of eleven lines. Thousands of projects depended on it. When it disappeared, major platforms and applications broke around the world.

A single developer’s local choice rippled through the global software supply chain. That is our computational reality today.

Randall Munroe, xkcd 2347: Dependency, CC BY-NC 2.5.

Choices built into data

Data does the same thing across time. Katrin Amelang and Susanne Bauer have followed how epidemiological risk scores travel. The Framingham Heart Study began in 1948, in a predominantly white, middle-class town in Massachusetts. Today its data is built into German primary care software, where it calculates and displays cardiovascular risk scores for patients.

Seventy years of changes in populations, lifestyles and living conditions—and the data is treated as if it were just as relevant today. Choices made in 1948 shape decisions in a doctor’s office now. That is our data reality today.

So where is agency?

This is a wicked problem. When choices are distributed across global infrastructures and decades of data, where do we even look? What is the site of a decision?

For STS, this raises two questions. How do we study these distributed assemblages? And how might we design them so that the right people have a voice at the moments that matter?

I think the challenge goes beyond making algorithms “fair” or “ethical.” It is about how computational infrastructures reshape the possibilities for human choice. That means moving beyond neat technical solutions, and getting into the messy work of making machine learning work in practice. I have tried to do some of that in Enacting the pandemic, and with Lotta Björklund Larsen in How should we theorize algorithms?

References

Amelang, Katrin, and Susanne Bauer. 2019. “Following the Algorithm: How Epidemiological Risk-Scores Do Accountability.” Social Studies of Science 49 (4): 476–502. https://doi.org/10.1177/0306312719862049

Hansen, Kristian Bondo, and Nanna Thylstrup. 2024. “Stack Bricolage and Infrastructural Impermanence in Financial Machine-Learning Modelling.” Journal of Cultural Economy 17 (1): 20–38. https://doi.org/10.1080/17530350.2023.2229347

Sumpter, David. 2018. Outnumbered: From Facebook and Google to Fake News and Filter-Bubbles – The Algorithms That Control Our Lives. London: Bloomsbury Sigma.