Preference and critique data

The operating task

Collect discriminative judgements and critiques tailored to model failure modes. Specialist contribution is shaped around the quality of the signal and its usefulness to model development.

Softinator's contribution

Capture the distinctions experts make between outputs and the critiques that expose why a response fails. The commissioned output includes preference pairs and critique release.

Acceptance in use

Review consistency, discriminative value and coverage of the model behaviours the programme aims to change. The engagement defines the baseline, task scope and evidence used to assess the resulting contribution.