Resilient Controls and Instrumentation Systems

Data resilience represents the safeguards and validation of integrity with data that is utilized in the training of machine learning models. The Cyber & Data Resilience Pillar envisions several grand challenge areas. Each is important for various reasons offering a range of capabilities: the first is the trustworthiness of training data, currently called data resilience. The second area deals with adversarial machine learning (AML) and the malicious use of artificial intelligence (MUAI). A third potential area deals with understanding the gap between human and artificial intelligence.

Data Resilience

The DARPA GARD (Guaranteed AI Robustness of Data) program begins to address the poisoning aspect of this problem, but training data can have other problems that have yet to be discovered such as missing data, improper data ordering or labeling. The ability to detect when data has been poisoned, or is otherwise inaccurate, is fundamental to trustworthy AI/ML. The data resilience initiative would work by first validating data obtained in the cyber-physical system (CPS) environment. Rather than the traditional validating through source authentication, this effort involves three pillars of information theory (temporal, contextual and descriptive) and collecting information from both physical systems and cybersecurity in order to create an accurate operational profile with acceptable deviations for the six meta states of processing (initialization, normal processing, busy processing, stressed processing, system failure, no processing). These states, and the deviations of the time, context and descriptive data points, provide the necessary baseline values that can be compared at any point in the processing lifecycle. When comparisons reveal significant discrepancies, the ability to reconstitute to the last known good state is enacted.

A series of studies involves several institutions in various roles as we work through the iterations in a water, electric, or nuclear CPS environment. The University of Wyoming dedicates their portion of the effort to developing meaningful metrics. Boise State University has students with knowledge of CPS security. Prof. Sin Ming Loo has the background in physical systems and cybersecurity to oversee the development of solutions that capture the data of interest. Indiana University (both IU and IUPUI) and Purdue University have access to a test environment (the virtual city) and students with both cyber and control systems knowledge that provides the researchers with capable students physically located, if necessary, to collect the data under our guidance. Data evaluation will be a combined effort by the team.

The above work describes creating trustworthy data for use in AI/ML. The actual deployment and testing of the AI/ML software would be led by INL with Purdue (lead) with IUPUI, and Boise State also participating in various aspects. Evaluations for efficacy will be performed by the University of Wyoming as part of their AI metrics efforts.

Adversarial Machine Learning (AML) and Malicious Use of Artificial Intelligence (MUAI)

Adversarial machine learning (AML) and the Malicious Use of Artificial Intelligence (MUAI) represent a rapidly growing challenge area that goes straight to the heart of our mission (protection of our nuclear arsenal). We have already witnessed manipulation of ML algorithms in other areas such as the Tay example in social media, where the chatbot learned new undesirable lessons (racism, etc.). Furthermore we have witnessed the work that bots can do to manipulate AI algorithms through flooding AML can go beyond the manipulation of training data to include algorithm manipulation. We have observed cases where AI misidentified characteristics about the target through erroneous assumptions. We propose several efforts in this space ranging from discovery vulnerabilities and unintended outcomes to countering through manipulation of AI.

Current work at INL in this space can serve as a departure point to many of the proposed studies. A newly approved LDRD on red teaming AI serves as the starting point along with the Microsoft/Harvard report on “Failure Modes in Machine Learning,” however, these studies represent a launching point of a much broader problem that requires a comprehensive, holistic approach that is interdisciplinary in nature. For example, disruption of the learning process (consider how the human mind works when distracted or interrupted) may impact the classification process, allowing us to fingerprint the adversary algorithms and possibly manipulate weights.

Another possible idea in countering the MUAI would involve countering and manipulating bot behaviors. Bot behaviors have been used to manipulate human emotions by slowly increasing inflammatory responses. One study might involve developing a bot to engage the malicious bot through “fingerprinting” and suggesting alternative behaviors forcing the malicious bot to potentially change its objective. This work has been discussed as an international joint effort with the University of Warwick, where a strong CPS program exists with a strong AI/ML program that has deep ties to UK defense research groups (DSTL and GCHQ).

Countering MUAI and AML are high priority focus areas since our adversaries are putting significant investments into AML/MUAI programs to gain knowledge and exploit our nuclear defenses. Automated responses with our own resilient AI/ML solutions offers an opportunity to assume the dominant position in this upcoming battle.

Exploring the Human Intelligence-Artificial Intelligence Gap

Human intelligence (HI) is the inspiration and model for artificial intelligence (AI). Consider the security council definition of AI, “a technology that performs tasks that mimic HI.” This definition opens up numerous possibilities to research into AI based in part on understanding the cognitive models that explain HI. Cognitive neuroscience is a relatively young discipline where new discoveries that explain the human thought process are still being made, which suggests that the HI-AI gap will present inspiration for many areas of research ranging from understanding algorithm biases to applying human cognitive disorders to AI. The partnering of cognitive neuroscience and cybersecurity has precedence in decision science. There are many potential areas for studies that may allow for active defense of adversarial behaviors. Studies in this space range from understanding ethics and values of different groups of people that can influence AI applications through recreating effects of human brain injuries in AI technology (such as causing trees to temporarily prune branches, disrupting NN processing). The overall goal is the identification of the HI-AI gaps and how those gaps inform potential vulnerabilities.

The resultant output, likely a framework or matrix, would provide a research roadmap for long-term future research in AI/ML.

Ideally this is a multilab/university/government collaborative effort. In addition to expertise at INL in cybersecurity and human factors, PNNL and Sandia both offer complementary skill sets. The Army Research Laboratory also has a long history of work in decision science.

Collaborations & Projects:

Publications: