Chat Control and the New Panopticon
There will be surely Somebody Who Can Watch You
What is Chat Control?
Chat control is an issue that has been simmering in the EU for several years, as the EU continues to try to introduce measures aimed at real-time monitoring and censorship of all types of online communication, not just chat messaging, under the guise of preventing CSAM (child sexual abuse material) and sexual grooming (child sexual abusers’ attempts to trick and tame minors over the Internet, etc.). Although there was considerable concern that the bill might pass due to strong proponents such as Belgian and Swedish politicians, a coalition of people from diverse backgrounds, including cryptography and security experts, privacy and human rights activists, messaging service operators, and politicians, worked together to oppose the measure. The campaign was so successful that a vote scheduled for last week at the Council of the European Union had to be postponed because it was unlikely to get a majority. However, supporters are unlikely to give up, and there is a possibility that the proposal will spread beyond the EU, including to Japan.
How Chat Control works
Whether it is a text message or a video call, our online interactions are encrypted as they leave our computers and smartphones and are sent to the recipient. There are ways to deliberately introduce vulnerabilities into the encryption algorithm or the network infrastructure itself so that third parties, such as law enforcement, can eavesdrop, and it is not impossible to break encryption by brute force (a possibility that will increase as quantum computers and other technologies come into practical use), but at present it is basically impossible for a third party to decipher properly encrypted communications. Therefore, if censorship is desired, the main battleground must be on the device before encryption. This is because humans need to view the content, so the data must have been decrypted on the device at some point. The basic mechanism of chat control is to require the manufacturers of messaging apps and devices to use artificial intelligence to scan images and messages before they are encrypted, check them against government databases of known illegal images, etc., and report any suspicious ones to law enforcement. Since this would be done before encryption, end-to-end encryption would essentially be meaningless. The essentially same measures are sometimes rebranded as client-side scanning or upload moderation.
In the latest proposal (PDF) (called "Chat Control 2.0") to be voted on, text messages were removed from the censorship list, and only images, videos, and URLs were censored, and users had the right to opt out. However, if they did not opt in, they would not be able to share or upload images, etc. anyway, so there was little point. Law enforcement and intelligence agencies are vocal about their concerns about so-called "going dark" - criminals running rampant by becoming invisible to the outside world through encrypted communications - but the latest proposal exempted intelligence agencies, the military, police and some EU ministries from the chat controls. It was law enforcement that would go dark, hidden from the scrutiny of the real world.
The World Chat Control Will Bring
What chat control will bring is literally George Orwell's "1984"-style mass surveillance society, which will collapse trust by opening a huge hole in the crypto-communication ecosystem that can be exploited by criminals and rogue states alike. Aside from being a serious invasion of privacy, this type of censorship is notoriously ineffective at preventing child abuse in the first place. There are too many so-called false positives, which detect such as images that are neither child pornography nor anything else. Those trying to sell chat control technology claim that the hit rate is over 80% or 90%. However, according to the documents released in response to the FOI request, this is just an industry claim and cannot be taken at face value as they refuse to provide details of the technology or test data, let alone independent verification. Furthermore, it is not uncommon for large messaging services to exchange more than 100 billion messages per day. Even if the accuracy rate were truly 99%, a billion messages would still be false positives. That would leave law enforcement with a huge amount of misinformation and little time to follow up on real leads and save real children. And since it is only illegal to disobey chat controls, real sex offenders will use illegal encryption services, etc. with impunity. Only the law-abiding public will not be able to use them.
Moreover, once a mechanism such as chat control is in place, the target would not be limited to child pornography. For example, if URLs containing dissident documents are designated as "harmful content," dissidents who refer to them may be rounded up. Or, in reality, instead of a roundup, completely innocent people would be swept up in the surveillance by being sent "suspicious" encrypted images and URLs. Rather than reducing the danger to child abuse victims, the chat-control approach would endanger everyone's online safety.
The New Panopticon
By the way, it is not government censorship itself, such as chat control, that I personally fear. It is the self-censorship it causes. When people feel that they are being monitored, they begin to self-censor. Eventually, even truly private interactions, such as standing around talking, will come under the scrutiny of "self-surveillance," which will eventually lead to self-regulation of thought, let alone speech. To be deprived of private conversation is to be fundamentally deprived of freedom of speech, and ultimately to undermine the very foundation of democratic society.