Sample tasks
Every task we deliver ships with the files behind it, so your team can read the brief, inspect the labels, rebuild the environment and run the checks. These examples show the structure. The full library is shared with clients during scoping.
Egocentric · CaptureMake tea, recorded from the POV
A contributor wearing a head-mounted camera makes a cup of tea in their own kitchen. Reviewers mark hand keypoints and object contact, split the clip into labelled actions (reach, grasp, pour, place) and write a plain-language narration.
clip.mp4
capture.json
consent/record.pdf
labels/hands.jsonl
labels/objects.jsonl
labels/actions.jsonl
labels/narration.txt
review/notes.md
$ validate labels → schema and timing checks passed
Head-mounted cameraHand and object labelsAction segments
Coding · TerminalRepair a failing CI build
An agent must diagnose why a Python package fails to build in a pinned container, fix the dependency conflict without changing public APIs, and prove the fix with the project test suite.
instruction.md
task.toml
environment/Dockerfile
solution/solve.sh
tests/verifier.py
review/notes.md
$ run verifier → all checks passed
DockerPythonBuild systems
Evaluation · RubricGrade a multi-step data migration
Reviewers score model outputs that migrate records between two schemas, using a five-dimension rubric: correctness, edge cases, data loss, idempotency and explanation quality.
instruction.md
rubric.json
fixtures/source.csv
fixtures/expected.csv
review/scores.jsonl
$ run verifier → all checks passed
RubricsData engineeringScoring
Preference · RLHFRank two explanations of a proof
A mathematician compares two model explanations of the same proof, picks the better one and writes a short rationale naming the step where the weaker answer goes wrong.
prompt.md
response_a.md
response_b.md
judgement.json
review/notes.md
$ run verifier → all checks passed
MathematicsPreference dataRationales
Want the full task library?
We share [NUMBER] samples across [NUMBER] categories with teams during scoping.
Related reading: How to build agent environments that test real work.