“In the information environments that young people spend a lot of their time, they’re constantly engaging with AI and machine learning in ways that surpass the expertise of their parents and even their educators. They’re more in touch with these tools. They experience them directly much more often. It’s not sufficient anymore to focus only on technical computing education, or only on teaching students who want to become programmers. Instead, we need to give them the tools to critically evaluate these systems as users encountering them constantly in daily life.”
– CLS2025 Keynote Speaker Danaé Metaxa
The Connected Learning Summit is excited to welcome Danaé Metaxa as a keynote speaker at CLS2025. Metaxa is the Raj and Neera Singh Term Assistant Professor of Computer and Information Science at the University of Pennsylvania, with a secondary appointment in the Annenberg School for Communication; as well as co-founder of the Penn HCI group, and executive committee member for Penn’s Center for Media, Technology, and Democracy. Their research focuses on “bias and representation in sociotechnical systems, and the effects of such systems on users,” with an emphasis on users impacted in high-stakes social contexts like politics, as well as on marginalized groups like youth, LGBTQ+ populations, and people of color. It’s Metaxa’s belief that building systems that benefit those most marginalized also often have widespread and significant benefits for society as a whole.
Metaxa, a computer scientist, works at the intersection of multiple fields – HCI, computational social science, digital literacy, and CS education. For them, the unifying thread and core interest through these various research areas is the interactions people have with technology. For their CLS keynote, Metaxa will be discussing the education branch of their work, which involves AI auditing, digital literacy, and how educators and researchers can use computational tools and strategies to better educate and serve young people in cultivating their approach to technology. We spoke to Metaxa about their current work as it relates to educational research, how AI audits can empower students to form nuanced and thoughtful approaches to technology, and what they’re most looking forward to about this year’s Connected Learning Summit.
CLS: What’s grabbing your attention and interest right now as a researcher? What feels most urgent?
Danaé Metaxa: “I’m particularly interested in how different technologies impact people’s lives, how people think about and understand technology, and how we can try to build tools or systems to improve people’s wellbeing. I’m especially focused on the experiences of marginalized communities.
Youth is one big example – there’s a lot of work that can be done to try to support them in the interactions they are having with technology. For a long time, computing was this ‘nice to have’ add-on [in education], where the thought was, for the students who were going into a field like software engineering or programming, it would be great if they could learn some computing earlier on. But these days, in the information environments in which young people spend a lot of their time, they’re constantly engaging with AI and with machine learning in ways that surpass the expertise of their parents and even their educators. They’re more in touch with these tools. They experience them directly much more often. It’s not sufficient anymore to focus only on technical computing education, or only on teaching students who want to become programmers. Instead, we need to give them the tools to critically evaluate these systems as users encountering them constantly in daily life.”
CLS: You do a lot with algorithm auditing – can you share more about that work, and where you started to see its value in a connection to education?
Danaé Metaxa: “Algorithm audits, or AI audits, are a method for understanding the impacts, often specifically social impacts, of computational systems. The idea is, as an outsider who doesn’t have access to the code or the dataset, how can you try to understand what that system is doing? Why it does what it does, and what effect that’s having on its users and society?
This human-centered focus is how I got interested [in auditing]. AI auditing is one way that we can try to understand these systems that are really opaque to outsiders, but have an outsized social impact. This focus includes search engines, which are really big mediators of our information ecosystems, and social media; these days it also includes generative AI, which has taken over a lot of the role of the search engine in terms of being an information intermediary, and getting increasingly integrated in social media and a lot of other tools that we interact with.
I conduct audits, and develop new techniques and frameworks for doing audits. I’ve done some work looking at gender and racial biases in image search results, in political leaning content online, and on social media and search engines. The way that this connects to education for me is that auditing is an informative, useful technique you can use as an outsider, for which you don’t even necessarily need a lot of technical expertise. And it can inform you about the system you’re using – how it works, what effect it’s having – even in the absence of internal access or transparency. That can be applied to education for young people – to learn, to engage with these systems critically and to develop that lens and the skills that they need to interrogate the systems that they’re interacting with every day.”
CLS: This year you won an NSF CAREER award. Congratulations, that’s huge! Can you share a bit about that work and the award?
Danaé Metaxa: The CAREER award that I wrote was about conducting sociotechnical audits of targeted advertising. Targeted ads are one really interesting ecosystem. A lot of the findings and methods can apply to other fields because ad targeting is AI-driven and heavily personalized. AI is heavily used in the creation of ads and in the matching process by which ad exchanges serve ads to people. It’s also an ecosystem in which end users have very little control. We are the product, and our attention is getting bought and sold by different companies. We have very little say in how that works, and even less power to change the kind of content we receive or to express our preferences in some way that’s respected.”
Metaxa’s CAREER-awarded project includes a proposal for two different systems for sociotechnical audits of targeted advertising – Intervenr and Admix. Metaxa describes Intervenr as a system for conducting audits that allows both a typical AI approach that reveals what content the system produces, but also an analysis of the impact the system has on users themselves.
Metaxa elaborated, “If I change the outputs of a system, does that change the user? People, users, are part of that sociotechnical loop — does it change them or their behavior, such that maybe they interact differently with the system over time? The normal AI auditing process asks, ‘why do different people see different ads?’ But [we also need to ask], ‘does that different content affect people differently?’ Users are really central to the ad ecosystem, and sociotechnical auditing is this way of measuring both how changes in the inputs will change what the system gives you, but also how that content affects you as a user.”
The second piece of the research is Admix, which Metaxa describes as “a system for end-user agency and collective action in the ads ecosystem.”
“At the moment,” they explained, “you have two options. One is to see ads all over the internet—and probably hate it. We’ve done some qualitative work to back up that statement; people broadly have negative experiences with targeted ads. The second option is to use an ad blocker. That’s not a bad option, but the ad blocker doesn’t let you reinsert yourself in that ecosystem, where your attention is being bought and sold. We wanted to build a system that could allow people individually, and even collectively, to gain agency in that adversarial environment.”
With Admix, users create “mixes” that designate what types of ads they’re okay with seeing, and what they don’t want to see. They can also connect with other users and curate mixes together, building on each other’s lists. For Metaxa and their colleagues, the immediate goal of the project is to give people interacting with ad-driven spaces a better understanding of what’s being shown to them, and more control over what they see—ultimately creating more user agency.
Ideally, enough people using these types of mixes together could engage in collective action, exerting meaningful pressure on advertisers to change their behavior. With enough people using a tool like Admix, Metaxa says, “instead of your attention getting passively bought and sold without your consent, these companies would feel some pressure to actually change their practices to respond to the desires and needs of the users.”
CLS: How did you get involved in the connected learning community?
Danaé Metaxa: “Through a very happy coincidence. I teach a seminar for graduate students called Algorithmic Justice, in which we study topics at the intersection of computing and social justice – looking to understand what types of biases these different systems display, and what can be done about it, what different community groups and scholars from different fields to say to speak to it. I was really lucky to have Luis Morales-Navarro as a student in my class in the spring of 2023. Luis is a Learning Sciences PhD student who works with [pioneer of computing, gaming, and learning research] Dr. Yasmin Kafai. Luis identified that the work he had been doing, on trying to help young people and or those with less expertise design and evaluate [learning] models, connected well to my work on AI auditing. In the last two years, Yasmin, Luis, and I have received several grants and written a number of papers together developing AI auditing as a digital literacy skill for youth, basically a specialized branch of child-computer interaction. I feel extremely grateful for this ongoing collaboration, and especially to Luis for kickstarting it. He’s a fantastic up and coming scholar in this space.”
CLS: Tell us about the Spencer grant you received on machine learning for algorithmic justice, involving AI auditing with local high school computer science students.
Danaé Metaxa: “This will be most of what I focus on in my keynote. There are a couple threads. The first is developing curriculum and educational modules for youth to learn about AI auditing as a digital literacy skill – working with local Philly youth, developing materials, developing frameworks. For instance, have one paper where we spell out five key components of AI auditing that are used by expert auditors conducting audits, but can also be taught to students.
The second thread is working with K-12 educators across the country to develop these educational materials as actual classroom modules and teach them in high school classrooms across the country. This work was originally an NSF grant that was terminated in the wave of politically-motivated grant terminations in April. Thankfully we have received some bridge funding from the Spencer Foundation and we continue that work.
Looking ahead, we plan to continue working with teachers through participatory design, not just to have them develop these materials, but also to prepare them to teach the materials and approaches to other teachers, spreading the curriculum through teacher networks.”
CLS: You’ve worked as an advisor outside academia in a number of different roles. Where do you see the value in taking scholarship into community-impacting work? Why is that important for scholars to do if they’re able to?
Danaé Metaxa: “I think it’s really important for scholars to have a theory of change, to understand how they want the world to look and how their work will help get them there. This is a common perspective among AI auditors – social justice has always been core to AI audits, because AI auditing grew out of tests for discrimination. Paid actors would go to apply for housing or apply for a loan, and see if they were treated differently based on race, for example. So auditing has always had this equity dimension to it. Some really impactful examples of AI audits include the Gender Shades audit, which resulted in researcher Joy Buolamwini testifying before Congress about racial and gender biases in facial detection algorithms, or researchers’ studies of discrimination in ad targeting leading to lawsuits against Meta.
So as a researcher, you can develop a body of work that makes you a useful expert for groups fighting the good fight, often through litigation and policy. Recently, for example, I published a paper in collaboration with the ACLU that provided suggestions for audit-mandating policy. In the past I’ve served as an advisor to the New York City Commission on Human Rights, concerning a large social media platform that rolled out some AI filters that were discriminatory to LGBT content. The way I think about the impact of my work and my theory of change is that I like to select audit targets that I hope will be high-impact on their own — that can achieve publicity or otherwise pressure companies to change practices — and that will make me useful to others engaged in the fight.
CLS: We talked about how you got involved in the connected learning community, but can you tell us what you’re most looking forward to about attending CLS this year?
Danaé Metaxa: “This will actually be my first time attending CLS. Much of what I do in my work is building bridges between different communities and different stakeholder groups, different disciplines. My sense is that CLS is a similar community of people that care about translating work across different spaces, so I’m really excited to be invited to participate this year — I feel like CLS is building a bridge to me, and I’m excited to experience this new space and get to know new people in this area.”
To learn more about Metaxa’s work, please visit their website.
We hope you’ll join us at CLS2025! The CLS2025 program runs from Sunday October 5 – Friday October 10. The event includes an online program of meetups, talks, and workshops as well as in person events organized by hosts and sponsors in campuses around the world. The in person events include keynotes and panels that will be streamed to online participants, as well as poster sessions, receptions and viewing parties. Register for CLS2025 today!
– Jenna Abrams
