Multi-laboratory validation and the statistical library DNAStatistX
DNAxs and DNAStatistX were used in a multi-laboratory validation study in which four laboratories participated. I.e. the Netherlands Forensic Institute in the Netherlands, Institute of Legal Medicine in Austria, National Forensic Laboratory in Slovenia, Institute of Legal Medicine (Cologne) in Germany, and Institut National de Police Scientifique (Ecully) in France. The study was partly funded by the European Union’s Internal Security Fund — Police (Proposal Number: 820838, Proposal Acronym: DNAxs2.0).
In this study, the software was modified to read multiple data formats. First, participants performed an exercise to explore all main functionalities of DNAxs and gave feedback on user-friendliness, installation and general performance. Next, every laboratory performed likelihood ratio (LR) calculations using their own dataset and a dataset provided by the organizing laboratory (NFI). The organizing laboratory performed LR calculations using all datasets. The datasets were generated with different STR typing kits or analysis systems and consisted of samples varying in DNA amounts, mixture ratios, number of contributors and drop-out level. Hypothesis sets had the correct, under- and over-assigned number of contributors and true and false donors as person of interest.
When comparing the results between laboratories, the LRs were foremost within the pre-set range of variation. The few LR results that deviated more had differences for the parameters estimated by the optimizer within DNAStatistX. Some of these were indicated by failed iteration results, others by a failed model validation, since unrealistic hypotheses were included. When these results that do not meet the quality criteria were excluded, as is in accordance with interpretation guidelines, none of the analyses in the different laboratories yielded a different statement in the casework report. Nonetheless, changes in software parameters were sought that minimized differences in outcomes, which made the DNAStatistX module more robust.
Overall, the software was found intuitive, user-friendly and valid for use in multiple laboratories. The dataset of the organizing laboratory is provided to aid the implementation of the DNAxs/DNAStatistX software within other laboratories. This test dataset can be downloaded here.