Victoria Police said the boy has been released pending further enquiries and its investigation is ongoing.

The school's principal said it appeared images of about 50 girls from the school were taken from social media and manipulated with AI to create obscene images.

Victorian Premier Jacinta Allan said her thoughts were with the young women and their families and that there was 'no place for this disgraceful and misogynistic conduct in Victoria'.

'Women and girls deserve respect in class, online and everywhere else in our community, which is why we have made laws against this behaviour and we are teaching respectful relationships in schools to stop violence before it starts.

' She told ABC Radio Melbourne that her daughter was very upset and was throwing up in response to the images.

Emily said girls at the school were being interviewed by police after the images came to light and the incident had a significant impact on her daughter.

'It's just too much.

It's too much intensity with all the images and the AI is getting out of hand and we're obviously seeing the implications.

In a statement on Wednesday morning, the school said the students affected were being offered support.

'On behalf of the persons and families affected, Bacchus Marsh Grammar is taking this matter very seriously and has contacted Victoria Police.

' The wellbeing of Bacchus Marsh Grammar students and their families is of paramount importance to the school and is being addressed.

Digital safety expert Susan McLean from Cyber Safety Solutions said it was becoming increasingly easy to manipulate images online using AI technology.

'The people making them, the young people, they're getting more brazen and it's easy – if you go into the app store there are a number of apps you can download that create these very believable images almost instantly,' she said.

'These images are often undiscernible to the naked eye, they are very, very good and even if, in a case like this, there's a lot of publicity and everybody knows they're fake, that does not undo the harm.

' She called on schools to step up educational programs around cybersafety, but also said parents need to be vigilant about what their children are doing online.

Emily van der Nagel, who lectures in social media at Monash University, said the technology itself was not the primary area of concern.

'I think this is just an update of behaviour that we have long understood to be part of some really deeply troubling gender relationships involving adults and children,' she said.

'I feel like the education to properly address this kind of issue is not necessarily about the technology and about deepfake, it's about the way that boys and girls and men and women relate to each other.

' In 2022, the Victorian government passed new laws targeting sexual offending, including image-based sexual offences, to combat the growing issue of deepfake porn.

It is already an offence to create an intimate image that is contrary to community standards, and a separate offence to share or threaten to share the image.

The government has said all Victorian government schools are committed to the $82 million Respectful Relationships initiative, which supports schools to model respect, promote positive attitudes and behaviours, and help students build healthy relationships, resilience and confidence.

Intensive training and support for staff is also being rolled out across about 1,200 schools, with the government planning to invite all remaining non-government schools to join.

We acknowledge Aboriginal and Torres Strait Islander peoples as the First Australians and Traditional Custodians of the lands where we live, learn, and work.

This service may include material from Agence France-Presse (AFP), APTN, Reuters, AAP, CNN and the BBC World Service which is copyright and cannot be reproduced.