Role OverviewEvaluate full shopper and agent conversations, including tool calls and their outcomes. Classify the issue with one primary tag and a specific secondary tag. Write concise comments explaining the shopper's intent, the agent's actions, and the rationale for the tags.
What You Will Do
Work independently and asynchronously to maintain consistency across evaluations. Evaluate full shopper and agent conversations, including tool calls and their outcomes.
Why It Might Be a Fit
Must have rule discipline for consistent taxonomy application, comfort reading structured traces to understand agent actions, restraint under ambiguity to use the unknown label appropriately, and clear, concise written English for effective communication.
Requirements
- Rule discipline for consistent taxonomy application
- Comfort reading structured traces to understand agent actions
- Restraint under ambiguity to use the unknown label appropriately
- Clear, concise written English for effective communication
Benefits
]]>