A Comparison of AI-Based Assessment Results with Those Conducted by Humans

Any vendor can announce that their AI agrees with expert judgement once the numbers are in and the awkward variants have quietly been dropped. Publishing the method first is a way of removing that option from ourselves. So, in plain terms: the validation trials described below have not been carried out yet. Nothing on this page is a result. What exists today is a finished study design, a defined scope, a defined participant group and the assessment tooling the human assessors will use.

The participant group consists of healthcare professionals and clinical leaders who have agreed to take part in the trials. This is a deliberate constraint rather than convenience. Communication competencies are context-bound. A model validated on graduate assessment-centre exercises tells you very little about a consultant explaining a treatment decision to a frightened family, or a ward manager reassigning duties mid-shift. If we intend the Digital Twin to be used in healthcare, the validation has to happen with people who actually have those conversations. A preliminary comparison, run on a single competency during model calibration, informed the design of this study. It was too small to constitute evidence of accuracy and we do not present it as such - it told us where the study had to be tighter.