Exploring how police and forensic experts take digital forensic evidence from crime scene to court and where the system itself might be vulnerable to bias or errors
With the first Clarus work package [WP1] coming to a close, we now have a deeper insight into how practitioners within our 5 data partner countries–Czech Republic, Finland, Greece, Portugal and the UK–interact with digital evidence and with each other, our findings are mapped out in three interlinked reports (Deliverables 1.2, 1.3 and 1.4). From official documents, such as regulations and written procedures, our partners developed comprehensive country specific overviews of the respective digital forensic systems. Collating these into our first report (D1.2) revealed some interesting insights – for example even the use of key terminology such as ‘digital forensics’ deviates across systems (someone should develop a lexicon!). Despite substantial flexibility in how a digital device and the data it contains might travel from crime scene to court, we also found several important common formal processes operating across these systems (see an example of this in Figure 1 below). These commonalities–and understanding the nuances of how they differ in each context– will be important in how we later look to apply the tools we are developing (a shared lexicon and a bias checker). The analysis for D1.2 was also our essential first step in mapping out the formal-official ways that the relevant agencies and teams recognise the potential for systemic (rather than individual) bias in their work and try to avoid or mitigate it. Some potential critical factors were identified such content and routes for information sharing across teams, the terminology used and at what points in an investigation the communications happen.
Our second report (D1.3) focusses on the perspectives of those working within each system. We asked a sample of practitioners from each country, representing the key roles/units for each jurisdiction, to fill out a daily diary for 5 days and to be interviewed after. The results have provided invaluable insight into the substantial informal processes and communications that happen throughout these digital forensic pathways. Among other things, the role of peer-peer consultation and mentoring for example became very clear, as did the value placed on informal, quick-time conversations across teams aimed at clarifying, and ensuring the accuracy of, what is then written into official modes of communication such as analysis requests and evidence reports. D1.3 also reports on how practitioners understand errors and the potential for bias in their work. We found a wide variety of concerns which included data overload, digital competency levels in non-experts involved in the system, and miscommunication. The types of bias explicitly noted were primarily linked to information sharing and confirmation bias.
Our final report specifically focussed on systemic bias. D1.4 reports on a secondary analysis which systematically examined D1.2 and D1.3 to identify properties of the system and practitioners’ perceptions of it which may indicate residual sources of bias (those which have not yet been identified and mitigated) . Based on existing understandings of systemic bias we looked at six potential sources, which include organisational pressures, peer review and cognitive bias embedded in workflows. A further three potential sources were drawn from our own reports–Knowledge and training gaps, Language and terminology inconsistencies, and the influence of tools and technology– and explored in detail. Finally, we discuss some suggested measures to mitigate each of the bias residuals we identified through our analysis.
Forthcoming: More detailed executive summaries of the three reports will be published after the summer, and academic journal articles are now in the planning stage.

Written by Amy Long – UK Postdoctoral Fellow, CLARUS
WP1 Team: D1.2, Jarmo Houtsonen (Work Package Lead) and Johanna Parviainen; D1.3, Amy Long and Megan O’Neill; D1.4, Nina Sunde, Laura vom Kolke & Brita Bjørkelo
