Week 4:Machine Learning Fundamentals

Beyond Accuracy: What Machine Learning Fundamentals Mean for Cybersecurity

In Monday’s CYBR 325 lab, I built and trained a small neural network in TensorFlow that classifies iris flowers into three species from four measurements. We ran our notebooks individually, then discussed results and difficulties in a Group 5 breakout, and I presented for the group afterward. My model reached 93.33 percent accuracy on 30 unseen test flowers, missing two: a versicolor predicted as virginica, and a virginica predicted as versicolor. Results can differ between runs because the network starts with random weights.

Our class also included a separate Group 5 Humanity/Ethics Panel discussion about who should decide the future of AI. A teammate led that presentation, with contributions from the rest of us. The discussion raised questions about technical control and concentrated decision-making power. Alongside the lab, it provides another way to examine AI systems: their performance and the human decisions surrounding their use.

For the evaluation side I used Google’s Machine Learning Crash Course page on classification metrics (Google, 2026). It defines accuracy as the proportion of all classifications that were correct, recall as the proportion of actual positives identified correctly, and precision as the proportion of positive classifications that are actually positive. Which metric matters, it argues, depends on class balance and on the cost of each kind of mistake. On a heavily imbalanced dataset, where one class appears about one percent of the time, a model predicting negative every time scores 99 percent accuracy while being useless.

That framing sharpened what my results showed. Validation accuracy reached 100 percent while test accuracy was 93.33 percent. Both came from subsets of the same static dataset, so the gap says nothing about change over time. Small samples and differences in example difficulty could explain it. Validation results can guide model selection, while an untouched test set gives a separate estimate of performance.

For security, the imbalance point carries. If one event in a thousand is malicious, a model labeling everything benign scores 99.9 percent and catches nothing. The confusion matrix and per-class report showed exactly which species my model confused. On imbalanced data the errors that matter can hide inside a high accuracy score, which is why recall on the attack class and precision on the alerts say more than the headline number.

Deployment adds a separate concern. Joint guidance from the NSA Artificial Intelligence Security Center, CISA, the FBI, and partner agencies in Australia, New Zealand, and the UK treats data drift as one of three major data security risks, alongside the data supply chain and maliciously modified data. Drift is a shift in the statistical properties of input data reaching an operational system, and the guidance calls the resulting degradation natural rather than an attack. One example it gives is a new type of malware the machine learning layer of an antivirus product has not seen (National Security Agency et al., 2025). The lab explained fitting StandardScaler only on training data to prevent data leakage. The guidance helped me consider the separate challenge of changing inputs after deployment. Our saved Iris dataset was static. Live network traffic can change.

The lab introduced core steps that security classification also uses: load, explore, split, scale, train, evaluate, predict. Production work adds adversaries, shifting data, monitoring, and constraints a flower dataset never raises. What I take from the week is that a single percentage starts a question rather than answering one.

References

Google. (2026, January 12). Classification: Accuracy, recall, precision, and related metrics.

Google for Developers. https://developers.google.com/machine-learning/crash-course/classification/accuracy-precision-recall

National Security Agency, Cybersecurity and Infrastructure Security Agency, Federal Bureau of Investigation, Australian Signals Directorate’s Australian Cyber Security Centre, National Cyber Security Centre New Zealand, & National Cyber Security Centre United Kingdom. (2025, May 22). AI data security: Best practices for securing data used to train and operate AI systems [Joint cybersecurity information sheet]. https://www.ic3.gov/CSA/2025/250522.pdf

AI use disclosure: Anthropic and OpenAI were used at Level 3, AI Collaboration, to assist with source discovery, organization, drafting, and revision, and to review and correct each other’s drafts. I reviewed the sources, verified the factual information, added my own analysis and course connections, and approved the final entry.