This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for an Attorney, Evaluation Operations (Contract) based in the United States.
The Attorney, Evaluation Operations will play an important role in assessing and improving the quality, accuracy, and reliability of AI-powered legal solutions. In this position, you will leverage your legal expertise to evaluate AI-generated content, test new product features, and provide structured feedback before their release to customers. Working closely with legal product specialists, you will help shape the development of innovative tools designed to transform professional legal workflows. Your responsibilities will include reviewing legal research, contracts, memoranda, and other AI-generated outputs against established evaluation criteria. This is a flexible, project-based opportunity that allows you to contribute to cutting-edge legal technology while working remotely on your own schedule. The role is particularly suited to experienced attorneys interested in the intersection of legal practice, artificial intelligence, and product innovation.
Accountabilities
AI Output Evaluation: Compare multiple AI-generated responses to identical legal tasks, including legal memoranda, contract redlines, and research answers, assessing their accuracy, relevance, completeness, and overall quality against established evaluation rubrics.
Product Testing and Feedback: Test new legal technology features and workflows in pre-release environments, providing detailed qualitative feedback based on professional legal experience.
Evaluation Rubric Review: Review and assess grading rubrics to ensure they are accurate, comprehensive, consistent, and appropriate for evaluating legal AI outputs.
Dataset Development: Create and contribute to evaluation datasets containing realistic legal queries, contract markups, and synthetic legal documents to support the testing and improvement of AI systems.
Legal Research and Analysis: Apply practical legal knowledge and analytical judgment to assess AI-generated legal content and identify potential inaccuracies, gaps, or areas requiring improvement.
Cross-Functional Collaboration: Work closely with legal product specialists and evaluation teams to support research initiatives and provide insights that inform product development and key technical decisions.
Project-Based Delivery: Complete assigned evaluation projects within agreed deadlines, often involving short turnaround times of 24–48 hours. Projects generally require between 2 and 20 hours over several days, with the flexibility to accept or decline individual assignments.
Requirements
Juris Doctor (JD) degree and an active license to practice law in at least one U.S. jurisdiction.
At least 3 years of professional legal experience at an AmLaw 100 or comparable law firm, or within an in-house legal department.
Professional experience in either commercial litigation or transactional law.
Familiarity with artificial intelligence tools, including practical experience using AI applications in a professional or personal context.
Excellent legal writing skills, with the ability to communicate complex legal concepts and reasoning clearly, concisely, and accurately.
Strong analytical and critical-thinking abilities, particularly when evaluating the quality, reliability, and completeness of legal information.
Excellent attention to detail and the ability to apply structured evaluation criteria consistently.
Ability to work independently, manage project deadlines, and deliver high-quality feedback in a remote environment.
Availability to accept occasional short-notice assignments and work on weekends when project deadlines require it.
Flexibility to manage a variable workload, as projects are assigned on an as-needed basis without a minimum hours commitment.
Preferred Qualifications:
Previous experience using Harvey or other AI-powered legal research and productivity tools in professional practice.
Experience evaluating, grading, annotating, or reviewing AI-generated content.
For litigation attorneys: proficiency with LexisNexis or Westlaw and experience conducting legal research and cite-checking.
A combination of law firm and in-house legal experience.
Benefits
Competitive hourly compensation: $125 USD per hour, based on hours worked and invoiced.
Flexible, project-based workload with no minimum hours commitment.
Fully remote working arrangement within the United States.
Freedom to manage your own schedule within individual project deadlines.
Ability to accept or decline projects based on your availability.
Access to pre-release legal technology environments and innovative AI-powered tools.
A dedicated laptop provided for project work.
Six-month independent contractor engagement (1099).
Opportunity to contribute to the development and evaluation of emerging legal AI technologies.
Engagement Details:
Contract type: Independent contractor (1099), engaged directly by the partner company.
Duration: 6 months.
Working hours: Part-time and project-based, with no guaranteed minimum.
Payment: Contractors invoice for completed hours.
Work arrangement: Fully remote, United States-based.
Benefits: This independent contractor position does not include employee benefits.
Scope limitations: No client representation, court appearances, litigation work, or business development responsibilities.
How Jobgether works:
We use an
AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1