Week 7’s topic was security tools and applications, and our class looked at what happens when AI is applied to health care. In Lab 7, I was the Recorder for Breakout Room 1, where our group tested a cancer diagnosis model, and my assignment that week applied ethical frameworks to AI security tools. This week’s source is a security standard that helped me understand what our group found.
The Source
I selected OWASP’s LLM04:2025, Data and Model Poisoning, part of the OWASP Top 10 for Large Language Model Applications. OWASP defines data poisoning as manipulating pre-training, fine-tuning, or embedding data to introduce vulnerabilities, backdoors, or biases into a model. Its recommended defenses include tracking where data comes from and how it changes, using data version control, limiting a model’s exposure to unverified data sources, using anomaly detection to filter adversarial data, vetting data vendors, and testing models through red teaming (OWASP Gen AI Security Project, 2025).
Connecting It to the Lab
OWASP writes for large language models, and our lab used a Random Forest, but the problem is the same: a model learns whatever its labels tell it. Our model reached 95.3 percent recall on the Breast Cancer Wisconsin dataset. When we flipped 10 percent of labels at random, recall did not change. When we flipped 10 percent of the cancer cases to benign, recall dropped to 89.1 percent. The targeted attack needed no more changes than random noise, but it caused exactly the error that hurts patients.
Our group’s proposed control was to hash each version of the training data and block any retrained model whose recall falls more than 2 points below the baseline. That lines up closely with OWASP’s advice on data version control. We also named a labeling vendor or insider as a realistic attacker, which is why OWASP’s point about vetting data vendors stood out.
Why This Matters for Security
In CIA terms, this is an integrity attack, and in health care integrity failures land on patients. The lab also showed a limit that no poisoning defense fixes. The dataset had no demographic columns, so our group could not test whether the model missed more cancers for some patients than others. OWASP warns that poisoning can introduce bias, but in our case we could not even measure bias to begin with.
My Takeaway
In my Week 7 assignment, I concluded that how strictly AI is governed should depend on how much harm it can cause, and that if a system can hurt people when it fails, it should be tested and monitored before it is used. This lab gave me a concrete example of that principle. Our group recommended the model only as support for doctors, after local testing and with data integrity checks in place.
Conclusion
Security tools usually protect systems from outside attackers. For AI applications, the training data itself becomes part of the attack surface, and protecting it takes the same discipline as protecting any other critical asset.
References
OWASP Gen AI Security Project. (2025). LLM04:2025 Data and model poisoning. OWASP Top 10 for LLM Applications 2025. https://genai.owasp.org/llmrisk/llm042025-data-and-model-poisoning/
AI use disclosure
Claude (Anthropic) was used at Level 3, AI Collaboration, to assist with source discovery, organization, drafting, and revision. The lab results, group role, and assignment connections are based on my own Week 7 work. I reviewed the source, verified the factual information, and approved the final entry.
Leave a Reply