AI, specifically Large Language Models (LLMs), are both promising and risky. They can enable investigators to interact with data in novel ways, but can also introduce errors.
Digital investigations are high risk and impact people’s lives and the security of organizations. Because of this, Sleuth Kit Labs has formed a set of AI principles that it will follow to instill trust from our users. Our AI principles are built to align with internationally recognized standards, including the OECD AI Principles.
- Human in Control: The investigator will always have a chance to review results from automated scoring and generative AI. The software is designed to support, not replace, human expertise.
- Traceability: Results will include references to the original source data (such as files and registry keys) so that the investigator can manually verify them.
- Explainability: Results will include information about why a conclusion was made so the investigator can more easily evaluate them.
- Disclose Non-Determinism: When a technique is used that is non-deterministic, the investigator will be notified so that they know to:
- Not be surprised when they get a different result next time
- Not assume the results are exhaustive
- Disclose Generative AI: The user will be notified when generative AI is used so that they know to review it for accuracy.
- Verify Generative AI: Where possible, structured data such as file paths, hashes, timestamps, and URLs in generative AI output are automatically cross-checked against source evidence to reduce the risk of AI “hallucinations.”
- Refute: If applicable, the AI techniques should attempt to both refute and support its hypotheses in order to come to the best conclusion. This is inline with the scientific method of coming to the best conclusion based on observations.
We will likely update these as the community uses more AI techniques. If you feel like something is missed by these, we’d love to hear from you. Please reach out at ai-principles@sleuthkitlabs.com.