In 1998, the Forensic Science Service (FSS) developed a data interpretation program called STRess (STR Expert System Suite) to aid their STR profile processing. Interpretation guidelines drawn from approximately 100 000 samples processed by the FSS and used by experienced operators were incorporated into the programming of STRess (Gill et al. 1996, Dunbar et al. 1998). FSS geno-typing guidelines require that all samples are genotyped by two independent operators to ensure accuracy of DNA typing results followed by a third operator to review allele calls and confirm that they are concordant. The aim of STRess was to reduce the amount of manual effort needed to evaluate the STR data by replacing one of the genotype analysts. The FSS has estimated that incorporating the STRess program into routine analysis has resulted in a 10-20% time-savings at the interpretation stage with improved standardization and quality of interpretation (Martin Bill, personal communication).
The success of the original STRess program has spawned the development of a number of systems that automate interpretation and interact as a suite. The suite is currently being developed into a global package branded 'STRess2' that will organize the interaction between systems and be configurable for any multiplex (Figure 17.5). A breakdown of the software modules in STRess2 currently proposed by the FSS is given below (information kindly provided by Martin Bill, FSS):
■ The STRess2 core interpretation engine is responsible for batch validation, ladder assessment and sizing, allele designation and interpretation. STRess2 has been designed and tested with well over five years of expert system operational experience. The application is designed as an 'open book' where the DNA unit can customize all the thresholds within the software to determine how 'confident' the software will behave. The FSS began making this software commercially available in late 2004. For details on how to obtain a copy of STRess2 or any of the other applications discussed, please contact Dr Chris N. Maguire ([email protected]).
Figure 17.5 Modules in STRess2 showing data flow. Figure courtesy of Martin Bill, Forensic Science Service.
■ The cross-contamination module offers a rapid screen for potential cross-contamination between samples. The program can search across both major and minor profiles performing many thousands of comparisons in seconds. A secondary search is performed to analyze samples within and between cases to look for 'intelligence links'. These links identified before loading to the database can increase the overall load rate. This application has subsequently been integrated with STRess2.
■ A contamination database known as 'SPACE' allows the storage and comparison of staff and supplier profiles. Crime stains and convicted offender samples can be screened against this secondary database to help ensure the profile loaded is not a result of operator contamination.
■ A batch-processing module enables reprocessing decisions, auditing and electronic reports along with interaction with the LIMS. This is a critical area of expert system development. As laboratories automate the allele interpretation steps, the bottleneck invariably moves to the auditing and case file handling stages thereby generating a requirement for a new automated solution. Functionality from this application has subsequently been integrated into STRess2.
■ A mixture module known as 'Pendulum' performs mixture analysis for databasing or court analysis. The program can deal with all two-person mixtures and is capable of deriving both contributors. The program can decipher mixtures when one of the profiles is from a known individual (e.g., victim in a rape kit test) or when both contributors are unknown (e.g., databasing). This program has delivered significant gains in quality and consistency. The program requires minimal operator intervention as it has an automated link to the expert system suite. STRess2 searches the batch for potential mixtures, any potential mixtures are sent to Pendulum for analysis, and the resultant profile is sent back to STRess2 to be incorporated with the rest of the batch results. All calculations performed by Pendulum are audited and presented graphically to the operator. The FSS offers training courses on this interpretation approach. The program also offers a complex search feed for profiles that cannot be interpreted using conventional theory. This information is sent to the 'search algorithms suite' for comparison against the database.
■ The search algorithm module enables high-end search algorithms to provide intelligence for samples not interpretable using conventional approaches. These applications offer ways of generating useful intelligence and increasing the overall 'value' of DNA forensic services. This will be a significant area of growth over the coming years.
Introduction of the suite of expert systems described above has resulted in a significant increase in efficiency and quality at the FSS with a large reduction in unit cost. Based largely on issues discovered during the development of STRess2, Martin Bill of the Forensic Science Service offers the following seven recommendations for expert system development:
1. Integration. Ensure that the information technology (IT) infrastructure, support and storage issues are considered when designing and developing expert systems rather than concentrating solely on the interpretation aspects. Solutions that are selected without considering these IT issues may result in most of the financial benefits of the expert systems being lost in future IT expenditure.
2. External influences. Consider potential changes to the supply chain and ensure the system will still be able to perform as required. On occasion external influences may require a change to interpretation. Any expert system solution must be flexible enough to work around such changes without causing significant problems.
3. Process design. Implement a new process that encompasses the expert system; do not simply implement an expert system using existing protocols. Process re-engineering is invariably required to maximize benefits when implementing the expert system. If this is not considered, the benefits of the software may disappear and it can be difficult to undo the damage.
4. Benefits measurement. There is a trite statement 'What gets measured gets done'. Decide how the benefits of an expert system can be measured and ensure the measurement process takes place. Make sure the correct units of measurement are used. Remember you are trying to measure the actual benefits realized not just the potential of the software. Unit costs are invariably better indicators than timing exercises.
5. People and culture. Expect cultural issues until the scientists gain confidence in the expert system and include this aspect of the implementation in the project plan. Do not be surprised or disappointed if people need time to become accustomed to the idea of automated interpretation.
6. Success rate. This factor is often overlooked yet it is one of the most critical areas to consider. As a DNA unit moves from manual to automated interpretation, the success rate of the process will change. Expert systems will probably never have total concordance with manual interpretation because the computer is following a rigid set of rules. The change in success rate should be closely monitored during the initial phase of deployment.
7. Target setting. Set realistic targets for the project. Analysts do not spend all of their time analyzing data and therefore it is impossible to realize 100% analyst reduction irrespective of how good the expert system is. This aspect closely links with the process re-engineering. The same problem exists with projected error rates. Many laboratories refuse to acknowledge that an error rate exists. We should recognize that there are many opportunities for error to occur, some within and some outside the control of the DNA unit. It is better to openly acknowledge that error can occur, as it is easier to look for solutions. No expert system will ever be designed that has an error rate of zero and therefore setting a target of zero is self-defeating. The real benefit of expert systems is that they behave predictably. It is this predictability and standardization that improves quality. As a starting point the objective should be to improve on the manual error rate (therefore making forward progress). When using expert systems, error rate and success rates are closely linked, one effectively determines the other. This level of control is extremely useful when attempting to optimize the output from a DNA unit.
Was this article helpful?
Discover Simple Techniques to Help Control Your Stutter. Stuttering is annoying and embarrassing. If you or a member of your family stutters, you already know the impact it can have on your everyday life. Stuttering interferes with communication, and can make social situations very difficult. It can even be harmful to your school or business life.