MIT Technology Review: The Reality of AI Usage Remains a Labyrinth

It remains difficult to determine exactly how people are actually using AI. Unlike corporate reports, it appears that users utilize AI more for sensitive areas, such as personal information, than for work tasks. While major companies like Anthropic and OpenAI release data, researchers argue that this data is curated and point out the need for independent verification.

The AI Observatory is an open platform created by integrating and analyzing seven existing datasets collected with user consent. Its goal is to enable researchers and policymakers to evaluate the actual state of generative AI usage by analyzing real conversations with major models like Claude and Gemini. Researcher Ankur Ruel noted that major AI-related decisions are currently being made based on extremely limited data.

Anthropic's 'Economic Index,' a representative usage statistic, focuses on work and productivity-related conversations while filtering out the rest. When AI Observatory researchers applied Anthropic's analysis method to their own dataset, they found that 48% of conversations were excluded. The excluded conversations had a much higher proportion of health and relationships (44.2% vs 31.2%), adult and illegal topics (7.9% vs 2.1%), harassment and hate (27.5% vs 5.66%), and sexual content (16.7% vs 2.4%) compared to work-related conversations. OpenAI's 2025 ChatGPT report also counted that only 30% of consumer usage was related to work.

This holistic analysis shows that discussions centered on workplace efficiency fail to fully reflect the diversity of actual usage behavior, supporting the necessity for independent data monitoring.