Will It Get Here? Privacy Out The Door?

[Corus Boy]

Apple will scan iPhone for nude photos, will report user to police if it finds child porn.

HIGHLIGHTS

    Apple announced on Thursday that it would launch new software to analyse user’s iCloud Photos for images of child sexual abuse.
    The new software will be used to check the photos stored in your iCloud and reaffirm whether it contains child sexual abuse material.
    Apple has also announced a feature to analyse each and every picture sent and received on the Messages app for child sexual abuse material.

External Link/Members Only

[daviemac]

Don't get involved with child porn and you have nothing to worry about.   

[PunterNumber69]

Don't get involved with child porn and you have nothing to worry about.   

+1

[Corus Boy]

 

Don't get involved with child porn and you have nothing to worry about.   

I quite agree but I'm sure that there are videos and photos of young looking girls out there, for example Hegre's models;

External Link/Members Only

Now they may not be illegal but will an 'algorithm' know that?  By the time it is poven, where may your life be?

And God forbid that you have photos of your kids at the beach or in a paddling pool in the garden?

[myothernameis]

Apple will scan iPhone for nude photos, will report user to police if it finds child porn.

In principle sounds a good idea, but cant help but think as time goes on, this software will be developed, and be applied to other area's.

I would have one concern, how does the software distinguish, between child porn, and normal porn, something like teen 18+.   If the software decides, that teen+ porn is classed as child porn, and alerts the police, and the police visit this person.   How do the police handle this, as possibility that person life might be ruined due do software deciding he had child porn on his phone

Now I have seen a porn video, of an Asian girl, and in the video, she could easily pass for 14, but is 30 years old

Weren't the police looking to do roughly the same thing, and wanting access to mobile phones, but companies like Iphone said no

[PunterNumber69]

I quite agree but I'm sure that there are videos and photos of young looking girls out there, for example Hegre's models;

External Link/Members Only

Now they may not be illegal but will an 'algorithm' know that?  By the time it is poven, where may your life be?

And God forbid that you have photos of your kids at the beach or in a paddling pool in the garden?

I'm assuming that the AI engine that's used for this has been trained by being shown images from a variety of sources and has been told this is a nude woman photo which is ok and this is a child abuse image which is not ok and it'll have been shown 100,000s or maybe 1,000,000s images as part of training of the AI. It won't be 100% accurate, which is why there's humans doing the final check before law enforcement is involved.  I think it's a positive step to help stamp out these crimes and put the sickos in jail.

There may be some photos of babies in baths, etc. that get alerts initially, but the humans will filter those out and then feed them into the AI engine training so it improves over time.

The child sex abusers will ditch Apple devices.

Other users may be concerned about Apple scanning all their photos though.

It's a brave move by Apple with a company stepping into law enforcement.

I don't envy the humans who will be checking the images that get flagged.

[Camargue]

I would have one concern, how does the software distinguish, between child porn, and normal porn, something like teen 18+.   If the software decides, that teen+ porn is classed as child porn, and alerts the police, and the police visit this person.   How do the police handle this, as possibility that person life might be ruined due do software deciding he had child porn on his phone

Doesn't it just check the image for a match against a database of known CSAM?
For now anyways.

[sparkus]

I'm very much in favour of anything which makes paedos unable to go about their sick activities but as with everything the devil will be in the trigger point for this.  Look at how easily people get banned or suspended from Facebook for breaching 'community guidelines' with innocuous content.

A lot of people's own family Instagram accounts have teens in quite risque poses (mine, for one).

[Blackpool Rock]

In principle sounds a good idea, but cant help but think as time goes on, this software will be developed, and be applied to other area's.

I would have one concern, how does the software distinguish, between child porn, and normal porn, something like teen 18+.   If the software decides, that teen+ porn is classed as child porn, and alerts the police, and the police visit this person.   How do the police handle this, as possibility that person life might be ruined due do software deciding he had child porn on his phone

Now I have seen a porn video, of an Asian girl, and in the video, she could easily pass for 14, but is 30 years old

Weren't the police looking to do roughly the same thing, and wanting access to mobile phones, but companies like Iphone said no

I was just thinking the same as there was a lot of coverage a few years ago about Apple refusing access to an Iphone of a terrorist that the FBI wanted to access
However if you believe Wiki then it appears Apple do sometimes allow access however it would have meant writing software to access that particular phone

External Link/Members Only

While I welcome anything that helps catch paedos I would question whether the police / authorities have enough resource to handle this, my understanding is that they are already struggling to deal with all the work they have so need to prioritise what to look at and what not to look at.

As already posted if this Tech starts picking up innocent images of say someone's 2 year old in the bath or making a sandcastle on the beach then it potentially overloads the system or rather it will overload the system.
It also raises concerns about privacy and what else some Govts start looking for, negative comments about the Govt / authorities etc, it's got real Big Brother potential

Truth be known I think people would be quite shocked at how many other people have dodgy images on computers etc, I knew a copper and he basically said exactly this as he knew of so many cases where police ceased computers / phones etc to look for evidence of certain crimes only to find child porn etc on the device and even on the devices of others in the household.
Effectively arrest a Teen for being involved in selling drugs and then arrest his dad or grandad for having indecent images on his laptop   

[Chorley]

I was just thinking the same as there was a lot of coverage a few years ago about Apple refusing access to an Iphone of a terrorist that the FBI wanted to access
However if you believe Wiki then it appears Apple do sometimes allow access however it would have meant writing software to access that particular phone

External Link/Members Only

While I welcome anything that helps catch paedos I would question whether the police / authorities have enough resource to handle this, my understanding is that they are already struggling to deal with all the work they have so need to prioritise what to look at and what not to look at.

As already posted if this Tech starts picking up innocent images of say someone's 2 year old in the bath or making a sandcastle on the beach then it potentially overloads the system or rather it will overload the system.
It also raises concerns about privacy and what else some Govts start looking for, negative comments about the Govt / authorities etc, it's got real Big Brother potential

Truth be known I think people would be quite shocked at how many other people have dodgy images on computers etc, I knew a copper and he basically said exactly this as he knew of so many cases where police ceased computers / phones etc to look for evidence of certain crimes only to find child porn etc on the device and even on the devices of others in the household.
Effectively arrest a Teen for being involved in selling drugs and then arrest his dad or grandad for having indecent images on his laptop 

It's been hapoenig in China for years. 

[PepeMAGA]

In principle sounds a good idea, but cant help but think as time goes on, this software will be developed, and be applied to other area's.

Agree, will start with legit targets, but is sure to to be applied elsewhere given the chance.
And in truth, if you were a pedo, you'd avoid getting an iPhone.
So then have it on android and Windows based devices.
After that, why not target terrorism. Again can't really complain about that... But define terrorism.
Hate speech could fall into that, if it suited the people in power at the time.

[myothernameis]

I don't envy the humans who will be checking the images that get flagged.

If the software flags up something, which requires further checks by human eyes, and they then scrutinize this persons phone, and he is innocent 100%, due to the software deciding something dodgy here

So nothing is found in relation to child porn, but have to think, is the GDPR being undermined, as they check this person phone, and maybe find something else

[lostandfound]

I heard this on Today R4 last week being reported as looking for matches against known child porn images, and seen other reports since saying the same thing using a "Neural Hash" generated using a neural network.

A cryptographic hash is a many to one transformation - you input a file / image and out comes a single value such as "ABCD12345", which makes it possible to do an efficient comparison.

Sometimes it may be possible for two different files / images to generate the same hash or "collision". Presumably Apple have a very good hashing algorithm / neural net which will not generate collisions!

And of course if supposed matches are then also checked by human eyes as a failsafe that sounds like a good system from the point of view of avoiding false positives - though some may still have privacy concerns.

[LLPunting]

I heard this on Today R4 last week being reported as looking for matches against known child porn images, and seen other reports since saying the same thing using a "Neural Hash" generated using a neural network.

A cryptographic hash is a many to one transformation - you input a file / image and out comes a single value such as "ABCD12345", which makes it possible to do an efficient comparison.

Sometimes it may be possible for two different files / images to generate the same hash or "collision". Presumably Apple have a very good hashing algorithm / neural net which will not generate collisions!

And of course if supposed matches are then also checked by human eyes as a failsafe that sounds like a good system from the point of view of avoiding false positives - though some may still have privacy concerns.

So will it be only exact matches to the database?  Hardly likely to be useful since those are all "known" images i.e. already found and incorporated into the database.  So "matching" is just presuming it's got some similarity to a digital measure of a small human showing too much flesh with possible "abuse" of certain identifiable body parts.
So look at what we get when we use the various image search engines and all the "near misses" for various ladies in different poses, wearing (or not) skimpy clothing...
How many images are going to get referred for human review, look at the problem Google et al are having with their content moderators. 

This all smacks of serious PR BS or an extremely intrusive breach of people's privacy.  Where's the opt-in for Apple customers to say I don't mind you scanning all my private photos or work photos?

They say the scanning will be done on the device and then "possibly offending" images posted up to the cloud for review.  Will unknown pedos just start working for Apple as content moderators? 

So every iPhone will be regularly updating its database of "neural hashes" to include new images that have been found that add to the "precision" of the AI scanning?  Will this memory be separate to what you pay for as usable by you the purchaser?  Will Apple have another cost to charge for in the increasingly bloated price of their devices?

The only agencies who should be scanning unpublished material are government authorities charged with upholding their jurisdictions law, how is it right for a greedy corporate to be intruding even further on the private materials of its customers?   No crime has been committed in taking a photo, only if that photo is known to be of illegal material.

[lostandfound]

So will it be only exact matches to the database?  Hardly likely to be useful since those are all "known" images i.e. already found and incorporated into the database.  So "matching" is just presuming it's got some similarity to a digital measure of a small human showing too much flesh with possible "abuse" of certain identifiable body parts.
So look at what we get when we use the various image search engines and all the "near misses" for various ladies in different poses, wearing (or not) skimpy clothing...
How many images are going to get referred for human review, look at the problem Google et al are having with their content moderators. 

This all smacks of serious PR BS or an extremely intrusive breach of people's privacy.  Where's the opt-in for Apple customers to say I don't mind you scanning all my private photos or work photos?

They say the scanning will be done on the device and then "possibly offending" images posted up to the cloud for review.  Will unknown pedos just start working for Apple as content moderators? 

So every iPhone will be regularly updating its database of "neural hashes" to include new images that have been found that add to the "precision" of the AI scanning?  Will this memory be separate to what you pay for as usable by you the purchaser?  Will Apple have another cost to charge for in the increasingly bloated price of their devices?

The only agencies who should be scanning unpublished material are government authorities charged with upholding their jurisdictions law, how is it right for a greedy corporate to be intruding even further on the private materials of its customers?   No crime has been committed in taking a photo, only if that photo is known to be of illegal material.

80 / 20 - most baddies will be swapping images as opposed to making their own. According to reports of previous cases they swap images around.

The database of hashes in going to be in the cloud. Most likely as you upload a hash is calculated and compared to a list of known hash values.

When I post on a subject like this I guarantee you will pop up with an aggressive post which is not thought through. 

[Marmalade]

This mostly seems a worry to tech-minded people although that should be a clue to everyone. I’ll list just one or two reviews since it’s come up.

External Link/Members Only

External Link/Members Only

External Link/Members Only

External Link/Members Only

External Link/Members Only

It sounds a bit like the story of catching terrorists. According to the techs, criminals will simply go to the trouble of working around it.

The bigger concern I think is this. Apple has built a reputation for security. It’s a reputation that is being eroded. Their software is not open source so it’s hard for outside experts to verify. Backdoors are a serious threat. Maybe our government wouldn’t misuse them but honestly, such software never remains secret that long. Israel — one of the leaders in the field — has been selling surveillance software mostly to the highest bidder. The latest report suggests 39% of U.K. businesses have already been hit by cyber attacks. For the individual user, what about when you travel abroad to a less tolerant country? Ones that make any sort of pornography or prostitution for that matter illegal. Or people who just like to think they can discuss things like politics and woke subjects privately?

You can try to avoid iMessage and Whatsapp. Or switch from iPhone. You know the saying, “Apple sells yesterday’s technology at tomorrow’s prices”.

[pantywetter]

Everyone supports the idea of stopping paedos and “think of the children.”

The arguments are:

- The consequences if the technology goes wrong.  Imagine if the police turn up at your house or workplace because you have a picture of a young looking but 18+ girl on your phone.  Indeed, my mates are sending Tik tok videos of hot girls on WhatsApp groups etc all the time.  This is high stakes stuff to put into an algorithm

- The slippery slope argument when they add new categories of material.  With how woke the big tech companies are you might soon lose your service for using gendered pronouns or questioning climate change

- The right to privacy.  Again, child porn is abhorrent, but 99.9999% of us would not have or want it.   Do we not have a right to have our photos and messages encrypted away from Algorithms and then human eyes?  Almost all of us have a very sensitive piece of information on our phones in that we are members and posting here on UKP.  I expect that to stay private and encrypted.

[lostandfound]

Describing a simple look up into a hash table as an "algorithm" is pushing it IMO - has to be one of the simpler ones! 

The other cloud providers are already scanning your uploaded images. Apple is just the first to go public about it in this way.

Apple emailed me as I have a new iThing - which is excellent as Apple have the best hardware in the industry - "Hey you have a new iThing - here's a year's subscription to TV+". From what I've watched so far their TV offerings are just as edgy and profane as the likes of Netflix and Amazon, so they are not, on that basis, some prudish outfit.

Amateur porn is I'm sure a large part of the appeal of iPhone, as I recall from the leak several years ago of many naked celeb selfies, so I don't think Apple will want to mess with that.

[petermisc]

The paedos who swap these photos typically have thousands of such images, and it is these people who are going to be top of the list for enforcement action.  A bit like if you have watched those programmes on fare dodgers, they target those with a history of dozens of dodgy-looking trips.

No need to worry if you have a few photos of your kids in the bath.  If you have thousands of photos of your kids in the bath, it might be a different matter.

[PunterNumber69]

To end the confusion.

BBC News - Apple regrets confusion over 'iPhone scanning'

External Link/Members Only