Language Expert Raters - US

Work Location: US Remote 

Engagement Model: Freelancer / Independent Contractor  
Start Date: ASAP 

Qualification Requirements: Successful completion of a role-specific written assessment and a background check. 

Main Responsibilities:

• Design and write effective prompts that elicit desired responses from LLMs, considering different tasks, contexts, and target audiences.

• Generate creative and informative text formats, including stories, articles, summaries, dialogues based on guidelines.

• Evaluate and compare the quality of LLM-generated responses based on criteria such as fluency, coherence, factuality, creativity, and bias.

• Analyze LLM outputs to identify strengths and weaknesses, and provide feedback to improve model performance.

• Collaborate with engineers, researchers, and product managers to refine prompting strategies and evaluation metrics.

• Contribute to the development of best practices and guidelines for prompt engineering and LLM evaluation.

Qualifications: 

• Bachelor's degree in English, Creative Writing, Linguistics, Computer Science, or a related field. 

• 3+ years of experience in content writing, copywriting, or a related field. 

• Strong understanding of LLM capabilities and limitations. 

• Excellent writing and editing skills, with the ability to adapt to different styles and tones. 

• Experience with prompt engineering and evaluating LLM outputs is a plus. 

• Understanding of statistics and data analysis is a plus. 

• Familiarity with data annotation and labeling tools is a plus. 

• Strong analytical and problem-solving skills. 

• Ability to work independently and as part of a team. 

• Can-do attitude, willingness to learn, and growth mindset. 

Preferred Qualifications: 

• Experience working with different LLM architectures and frameworks. 

• Knowledge of NLP concepts and techniques. 

• Familiarity with human evaluation methods for LLMs. 

• Experience in a research or academic setting. 

• Tech savvy: ability to read json not required, but we need candidates willing to grow their technical abilities 

• Bilingual preferred (Chinese, Spanish, Portuguese, Arabic, French, German). 

DataForce by TransPerfect is part of the TransPerfect family of companies, the world’s largest provider of language and technology solutions for global business, with offices in more than 100 cities worldwide. We offer high-quality data for Human-Machine Interaction to some of the most prestigious technology companies in the world. Our department focuses on gathering, enriching and processing data for Machine Learning in different AI domains. To learn more about DataForce please visit us at https://www.transperfect.com/dataforce.

For more information on the TransPerfect Family of Companies, please visit our website at www.transperfect.com