Meta Ad System Fails to Block Child Sexual Abuse Promotions in India

Meta, the parent company of Facebook and Instagram, has come under scrutiny after allowing advertisements promoting child sexual abuse material to appear for users in India. Local authorities have demanded the social media giant explain how its ad-checking systems failed to detect these illegal promotions before they went live, as first reported by the BBC.

According to the BBC, Meta permitted several variations of such promotions to appear across its apps in recent weeks. The ads, observed by the BBC World Service, used explicit terms including “rape video” and “child video,” directing users to channels on the messaging platform Telegram, where the material could be purchased for as little as 99 rupees (approximately $1).

BBC reported that it flagged the offending promotions to Meta, and was initially told that the ads did not violate the company's guidelines. Meta removed the promotions shortly thereafter, but the incident has raised serious concerns about the effectiveness of its content moderation systems.

The revelation comes as Meta continues to advance its plans to replace most of its human moderation staff with artificial intelligence agents. These automated systems are empowered to assess and take action against potentially violating content across its platforms. Meta did not specify whether the offending ads slipped through due to its AI process, but told BBC that “no system is perfect, and our review process may not detect all policy violations.”

This incident represents another setback for Meta’s expanded AI push, which the company hopes will eventually provide a significant additional monetization pathway through AI automation as a service for third-party clients. Meta has already cut approximately 20% of its labor force this year, with a significant portion of those roles being replaced by AI agents. The company is seemingly using its own operations as an example of how its AI tools can reduce labor costs.

However, initial problems with its AI systems could become major impediments to its development. Last month, more than 20,000 Instagram accounts were breached after a group of users discovered they could convince Meta’s account support agent to grant them access without providing any evidence that they were the actual account holders. Meta said it has since patched the issue, but the scale and scope of how people pose queries to AI chatbots seems to lend itself to an almost infinite range of such exploits.

Meta is also facing challenges in AI development. Reuters reported last week that Meta CEO Mark Zuckerberg told employees that the development of its AI agents is not progressing as fast as anticipated, adding that recent staff cuts have not yet delivered the planned benefits. The Information reported that Meta has also implemented restrictions on AI token usage, as its costs skyrocket due to its mandated reliance on AI tools.

These costs are likely why Meta is now exploring new ways to monetize its massive AI investment, including the development of a cloud infrastructure offering that could maximize the usage of its expanding data center network. That increased data capacity had initially been earmarked for Meta’s own AI usage, but now the company seems to be reassessing its overall AI push and the true value of its possible AI offerings.

Issues with its internal use of AI could further compound concerns. Meanwhile, Meta is facing other public relations challenges on the AI front. Last week, Wired reported that Meta was using an army of contractors posing as minors online to test how competitor chatbots responded to prompts involving suicide, sex, eating disorders, and other high-risk subjects. Meta said this was standard research to better understand safeguards and mitigation measures, but perceptually, it is another knock against its AI push, which is already facing significant public backlash.

If Meta’s AI systems fail to protect children, that could be the last straw in terms of its approach. That could also have major implications for the future of its business, especially if Meta fails to build more advanced AI models via its much-lauded Superintelligence group.