Discord has admitted on X that a bug in its AI-assisted moderation system wrongly banned around 8,200 user accounts over the past two months after mistakenly flagging innocuous images as child sexual abuse material (CSAM). The disclosure followed widespread user reports that ordinary square grid images, including spreadsheets, chessboards, Minecraft inventory screens and transparent chequerboard backgrounds used in image editors, had triggered permanent account suspensions.Â
Discord said about 8,200 accounts were affected between May and early July, before another 200 users were banned over the weekend due to the same fault. According to the company, its systems use similarity matching against known CSAM to flag potentially illegal content for review by its Trust and Safety team. However, a software bug triggered permanent account bans instead of temporarily pausing uploads during that review.Â
Moreover, the company claimed the same bug prevented cleared accounts from being automatically reinstated, leaving affected users suspended until Discord manually restored their accounts.
How Discord’s CSAM detection works: Discord’s moderation system is designed to identify both known and previously unseen CSAM. For known material, the platform uses perceptual hashing technologies such as Microsoft’s PhotoDNA, which compare uploaded images against databases of digital fingerprints of confirmed illegal content rather than the images themselves. However, Discord has also developed machine learning models that analyse the visual content of images to identify potentially novel CSAM that has not yet been added to those databases. In both cases, the company says human Trust and Safety reviewers are expected to verify potential matches before any enforcement action is taken.
Also read:
- Discord rolls out End-to-End Encryption for all voice and video calls
- Discord Says Teen-By-Default Settings To Be Rolled Out Globally In March 2026
- OpenAI releases child safety policy framework recommendations to combat AI-enabled CSAM
The post AI moderation bug in Discord bans over 8000 accounts for child porn imagery appeared first on MEDIANAMA.
