Role Overview
Mercor is partnering with a leading AI research lab on Project Vesuvius , an initiative designed to evaluate and enhance the ability of large language models (LLMs) to generate structured, high-quality research plans for open-ended machine learning problems.
We are seeking Machine Learning Researchers and PhDs to serve as annotators who will assess and provide structured feedback on AI-generated research plans. The goal is to improve how LLMs function as brainstorming partners for machine learning research.
This is a remote, short-term engagement with flexible hours and opportunities to contribute to frontier AI evaluation and research.
Key Responsibilities
Evaluate and compare AI-generated research plans for clarity, feasibility, and technical soundness.
Design and compile ML tasks based on real-world challenges and research competitions.
Draft detailed, executable natural language plans for machine learning workflows.
Implement and validate research plans in Python within a Docker environment.
Assess outputs against structured rubrics, provide usefulness scores, and deliver concise, objective feedback.
Ideal Qualifications
5+ years of experience in applied machine learning or a PhD in machine learning or related fields.
Strong understanding of ML research methodologies, experimental design, and evaluation practices.
Excellent analytical and technical writing skills.
Experience with reproducibility or benchmarking in ML research preferred.
Detail-oriented and able to deliver high-quality, structured feedback independently.
Engagement Details
Type: Independent contractor
Mode: Fully remote and asynchronous — work from anywhere, on your own schedule.
Commitment: Maximum 80 hours per week
Project Name: Vesuvius
This role is designed for researchers and engineers who value autonomy, precision, and meaningful contribution to frontier AI development.
Compensation & Contract Terms
Hourly Pay Rate: Up to $140/hour
Payment: Weekly via Stripe Connect
Contract Type: Independent contractor engagement
Structure: Remote, milestone-based evaluation with flexible scheduling
Application Process
Submit your resume or CV highlighting relevant ML research or engineering experience.
Complete a short AI-based interview and a brief questionnaire about your experience with reproducibility and model benchmarking.
Selected candidates will receive detailed onboarding materials and access to the project environment.
About Mercor
Mercor is a global talent marketplace connecting exceptional professionals with leading AI labs and research organizations. Our mission is to empower experts to contribute directly to the most influential and technically advanced AI projects worldwide. Mercor is backed by investors including Benchmark, General Catalyst, Adam D’Angelo, Larry Summers, and Jack Dorsey. Thousands of professionals across disciplines — from research and engineering to law and design — have joined Mercor to build the next generation of artificial intelligence systems.
...Hi, my name is Devin, and I LOVE making content. Seriously. And I want to find someone who loves it too! We are hiring an ambitious, in-house content creator and poster who understands that strong video is not just about filming and editing, but about performance. This...
...Legacy Towing Inc. is experiencing significant growth and is excited to expand our team with new Professional Tow Truck Drivers. We pride ourselves on fostering a fulfilling and competitive work culture, offering top pay and bonus incentives. Join us to be part of a dynamic...
...Welcome to Tower EV Car Wash Attendant We are delighted to have you show an interest in joining our team as an Car Wash Attendant... ...services to our customers. Our commitment to excellence, safety, and customer satisfaction sets us apart in the industry....
...you look stunningnever overdone. From facial rejuvenation and laser treatments to body contouring and skin care, we offer one of the... ...solutions in Southern Arizona. Job Overview: As a Laser Technician, you will perform safe and effective laser treatments for clients...
...collaboration, and accelerated intelligence. Learn about the Danaher Business System which makes everything possible. The Human Resources Manager is responsible for leading the HR function for three sites within our Diagnostics manufacturing organization. Two sites in...