OpenAI's Quest for Real-World Data: Unveiling the AI Training Process
In a bold move, OpenAI is reaching out to contractors, inviting them to share their past work experiences. The goal? To create a human baseline for evaluating the performance of its cutting-edge AI models. But here's the intriguing part: OpenAI wants contractors to upload real assignments and tasks from their current or previous jobs. This data will be crucial in measuring how well AI models can handle various tasks, a key step towards achieving AGI (Artificial General Intelligence).
The project, as detailed in confidential documents, involves contractors describing their work and providing real examples. These examples should be concrete outputs like Word docs, PDFs, or images, reflecting the actual work they've done. OpenAI even encourages the creation of fabricated work examples to demonstrate realistic responses in specific scenarios.
The process is intricate. Each task has two components: the task request (the instruction given by a manager or colleague) and the task deliverable (the actual work produced). OpenAI emphasizes that the examples must be real, on-the-job work, not summaries or summaries of summaries. This attention to detail ensures the data's authenticity and relevance.
A fascinating example is provided in the presentation: a 'Senior Lifestyle Manager' task at a luxury concierge company. The goal is to prepare a detailed itinerary for a 7-day yacht trip to the Bahamas, tailored to a family's interests. The 'experienced human deliverable' is a real itinerary created for a client, showcasing the contractor's expertise.
However, this initiative raises concerns about data privacy and security. OpenAI instructs contractors to delete corporate intellectual property and personally identifiable information from their uploaded files. This is a crucial step to protect sensitive data and prevent potential legal issues. One document even mentions a tool called 'Superstar Scrubbing' to assist in deleting confidential information.
Evan Brown, an intellectual property lawyer, warns that AI labs handling confidential information from contractors on this scale may face trade secret misappropriation claims. The risk lies in the trust placed in contractors to decide what is confidential. If personal information slips through, it could lead to legal complications.
This data collection strategy is part of a broader trend in the AI industry. Companies like OpenAI, Anthropic, and Google are hiring contractors to generate high-quality training data, enabling them to develop AI agents capable of automating complex tasks. This has led to the emergence of a lucrative sub-industry in AI training, with companies like Handshake and Surge AI valuing billions.
OpenAI's approach to sourcing real-world data is unique. While they explore various methods, including obtaining data from companies after business closures, the contractor program presents an opportunity to gather diverse and authentic data. However, the process must be carefully managed to ensure data privacy and security, addressing the concerns raised by legal experts.
