Cybersecurity is a constant challenge facing many aspects of our society. Even in the face of significant threats, society seemingly has insatiable thirst for greater cyber capabilities, and that thirst is accelerating at an increasingly rapid pace. We eagerly adopt new cyber capabilities in the name of efficiency, convenience, entertainment, and even safety; but often, we do so without considering the potentially significant risk. Recent changes in the way we use computers have not only affected where and how computers are used but also the way networks are architected: today’s network boundaries constantly morph. We are rapidly moving toward a paradigm in which embedded systems will be highly interconnected and pervasive in nearly every aspect of our lives.

These changes bring inherent vulnerabilities and new risks. Addressing these new challenges will require cybersecurity research based on sound scientific principles. The scale and complexity of the problems will require that researchers apply new experimental methods that enable discovery, validation, and ongoing analysis. This research may require fundamentally different approaches to experimentation and testing. Current capabilities in experiment analysis, including cybersecurity and network testbeds, are often designed for short- to mid-term research and development and have limitations in fidelity, realism, scale, instrumentation, and metrics, among others. The challenges of tomorrow’s cyber world require a fundamental shift in the cybersecurity experimentation paradigm.

Both the public and private sectors are working on cyber experimentation capabilities precisely because of this issue. The 2011 Federal Research and Development Strategic Plan [1] specifically calls for the development of “measurement methods and testbeds for security properties.” The previous five years have seen significant changes in cybersecurity experimentation capabilities. Some of this work enables researchers to move from small- to very-large-scale experiments; from single isolated testbeds to widely federated experimentation facilities; and from fixed, wired networks to support for wireless networks and software-defined radios (SDR). In addition, support for highly specialized SCADA experiments, novel support for new interdisciplinary experimentation, and support for modeling and reasoning about human behavior have emerged. Cybersecurity experimentation technology and methodologies arising from current efforts may serve as a cornerstone on which new capabilities may be built, ultimately providing entirely new experimentation paradigms to broader communities.

To ensure that we meet future cybersecurity experimentation needs, we must first understand the capabilities and limitations of existing experimentation infrastructure and look ahead to future cybersecurity challenges in an objective manner. We must then chart a course that will lead to a new set of supporting capabilities and infrastructure to help researchers meet those challenges. These capabilities must encompass the broad needs of the community, across government, academia, and industry. Domains have different and sometimes unique needs, requiring disparate, although potentially overlapping sets of cyber-based experimentation capabilities. We must enable communities to develop their own required cybersecurity experiment capabilities, and we must provide a means to unify and combine capabilities across domains.

© 2017 Cybersecurity Experimentation of the Future.    All rights reserved.