Childnet have launched a brand new resource exploring generative AI and how it can be misused, with a specific focus on using generative AI to edit a photo of somebody to make them appear nude or naked, sometimes called ‘nudification’.
This resource is for use with learners aged 13-17, but may be suitable for use with 11 and 12 year olds with adaptation. The resource includes three lesson plans, which cover:
- What is generative AI and what can it be used for?
- Misuse of generative AI and the possible impacts
- Reporting nude images made using generative AI
Each lesson plan includes accompanying appendices and slides and is designed to be an hour long. These lessons are adaptable, and educators can divide the content up into smaller sections for shorter lesson lengths.
Previously, our Youth Advisory Board have discussed how difficult young people find it to determine when content, including images and video, is real or AI-generated. As an online safety charity part of the UK Safer Internet Centre, Childnet are hearing about the issue of AI-generated sexual imagery more from the young people we directly work with.
This is reinforced by research in this area, so we know it is an issue we need to both be aware of and help support young people with.
This resource has been created with kind support from Microsoft. Courtney Gregoire, Vice President, Chief Digital Safety Officer at Microsoft, said: “At Microsoft, we believe empowering teens and equipping parents and educators is key to tackling the risks posed by abusive AI-generated content. That’s why we’ve joined forces with Childnet to develop and launch impactful new resources on intimate image abuse. Together, we are providing practical support to help safeguard young people in schools across the UK—and around the world.”
Supporting documents for parents and carers
It is important to engage parents and carers with these topics to encourage consistent and cohesive messaging at home.
To help support educators to do this, we have created an accompanying information leaflet to help parents and carers understand this issue. The leaflet defines generative AI and nudification, explains the impact, and provides advice on how to support a young person with this issue.
The need for resources about nudification
In a focus group conducted before the creation of this resource, Childnet asked a selection of young people aged 11-17 a series of questions around the theme of generative AI and ‘nudification’. It was clear from those conversations and insights that this is a harm that young people are both aware of and experiencing, especially as a result of ongoing developments in generative AI technology. We found that some young people have been misusing generative AI as a joke, for revenge, to blackmail somebody, for catfishing, or as a form of online bullying.
The focus group also helped us identify some misconceptions around generative AI and nudification. For example, believing somebody would be more likely to have a nude image created of them if they are wearing a more revealing outfit. This may be a result of victim blaming culture (when a victim is blamed for abuse that has happened to them, which is explored in Lesson 3 of this resource) and a misunderstanding of how generative AI works – an image of someone doesn’t have to be ‘revealing’ for someone else to use generative AI to make them look naked in it.
Members of the focus group also shared how young people may struggle to report harmful AI-generated content that depicts them, or talk to somebody, as they might be worried about the reaction from others. Other possible impacts of this kind of image-based sexual abuse included mental health issues, vulnerability and feeling betrayed or humiliated. They stated that it is important for other young people to be made aware of this misuse of AI, as they felt it is becoming increasingly common among their peers.
There is a pressing need for an educational response to this topic, to ensure young people are aware of how to use generative AI safely and responsibly, the impact of abuse, and where they can go for support.
