Mandatory AI Filtering to Increase Platform Burden

Starting July 1, platform operators will be obligated to proactively detect and block not only illegally filmed videos but also images. Consequently, the introduction of artificial intelligence (AI) filtering is expected to become mandatory. The Korea Communications Commission plans to expand the scope of technical and managerial measures for preventing the distribution of illegally filmed content from videos to images, applying it to approximately 80 platforms.

While large platforms can easily respond with their own AI technology, small and medium platforms face a dual burden of technology and cost due to the obligation to adopt and operate new solutions, raising controversy over the system's effectiveness and fairness. The fact that only 0.9% of SMEs utilize AI, according to a survey by the Korea Technology Innovation Agency, underscores this burden.

There are concerns that users' posting activities may be restricted if legitimate images are mistakenly blocked by AI. The risk of user harm due to false positives, where legitimate photos are incorrectly blocked, is linked to the aforementioned burden on small and medium platforms. In particular, communities, second-hand marketplaces, and open chat services with high image usage could see user dissatisfaction spread if overly strict blocking standards are applied.

Naver has announced enhancements to its AI image recognition technology, applying identification and sanction measures across its services. Kakao has been increasing its response speed and intensity through its 'Fast Track' and 'Deep Research' systems.

The success of the system will not be measured by mere increases in detected cases, but by how accurately content is blocked and how swiftly false positives are rectified. The Korea Communications Commission emphasized the importance of responsible responses and the implementation of technical measures by operators to prevent the distribution of illegally filmed content. The commission stated it will support the system's implementation through cooperation with relevant agencies and continuously strengthen the framework for preventing the distribution of digital sex crimes.