Ayoosh Bansal

System auditing is an essential tool for detecting malicious events and conducting forensic analysis. Although used extensively on general-purpose systems, auditing frameworks have not been designed with consideration for the unique constraints and properties of Real-Time Systems (RTS). System auditing could provide tremendous benefits for security-critical RTS. However, a naı̈ve deployment of auditing on RTS could violate the temporal requirements of the system while also rendering auditing incomplete and ineffectual. To ensure effective auditing that meets the computational needs of recording complete audit information while adhering to the temporal requirements of the RTS, it is essential to carefully integrate auditing into the real-time (RT) schedule. This work adapts the Linux Audit framework for use in RT Linux by leveraging the common properties of such systems, such as special purpose and predictability. Ellipsis, an efficient system for auditing RTS is devised that learns the expected benign behaviors of the system and generates succinct descriptions of the expected activity. Evaluations using varied RT applications show that Ellipsis reduces the volume of audit records generated during benign activity by up to 97.55%, while recording detailed logs for suspicious activities. Empirical analyses establish that the auditing infrastructure adheres to the properties of predictability and isolation that are important to RTS. Furthermore, the schedulability of RT task sets under audit is comprehensively analyzed to enable the safe integration of auditing in RT task schedules.

Advances in deep learning have revolutionized cyber-physical applications, including the development of autonomous vehicles. However, real-world collisions involving autonomous control of vehicles have raised significant safety concerns regarding the use of deep neural networks (DNNs) in safety-critical tasks, particularly perception. The inherent unverifiability of DNNs poses a key challenge in ensuring their safe and reliable operation. In this work, we propose perception simplex (PS), a fault-tolerant application architecture designed for obstacle detection and collision avoidance. We analyse an existing LiDAR-based classical obstacle detection algorithm to establish strict bounds on its capabilities and limitations. Such analysis and verification have not been possible for deep learning-based perception systems yet. By employing verifiable obstacle detection algorithms, PS identifies obstacle existence detection faults in the output of unverifiable DNN-based object detectors. When faults with potential collision risks are detected, appropriate corrective actions are initiated. Through extensive analysis and software-in-the-loop simulations, we demonstrate that PS provides deterministic fault tolerance against obstacle existence detection faults, establishing a robust safety guarantee.

The advent of machine learning has enabled autonomy applications that may have remained impossible otherwise. However, the inherent limitations of machine learning make its use for safety-critical tasks perilous. This is evident in the plethora of safety challenges facing autonomous vehicles. Autonomous vehicles have the potential to save and improve lives, however, their widespread usage and adoption have been severely inhibited by challenges in ensuring the safe operation of these vehicles. This work redesigns the autonomy software with separated fulfillment of safety and mission responsibilities. Safety-critical requirements are disentangled from mission-critical requirements, reduced while maintaining sufficiency, and fulfilled using verifiable software. A safety-critical layer, composed of verifiable software only, monitors the existing complex learning-dependent mission-critical system for faults at runtime, overriding the actions of the vehicle when faults in the mission-critical system may lead to safety violations. Learning from a study of real-world fatal collisions involving autonomous driving, the proposed system design is first applied to the crucial challenge of obstacle existence detection faults, or false negatives, in autonomous vehicles. Requirements for obstacle existence detection, i.e., when is an obstacle considered sufficiently detected to enable collision avoidance, are analyzed and reduced. A LiDAR-based classical geometrical obstacle detection algorithm is then analyzed to determine the bounds for its capability to detect an obstacle and its limitations. This algorithm is evaluated against real-world datasets showing that it meets the reduced but sufficient safety-critical requirements. Finally, using this obstacle detection algorithm, a safety monitor for the autonomous driving system is designed to provide deterministic guarantees against collisions due to obstacle existence detection faults. The developed safety properties were validated using simulation for autonomous ground and air vehicles. To ensure the reliable operation of the autonomous system, it needs to be monitored for security incidents and malicious attacks. In general-purpose systems, system auditing is a crucial tool for the detection and analysis of such events. Therefore, in this work, system-level auditing is adapted to real-time systems, developing Ellipsis, the first system level security auditing system designed specifically for real-time systems. Ellipsis leverages the predictable repetition of behaviors in real-time systems to aggressively reduce the audit event logs generated by benign activity while preserving all security-relevant information and recording suspicious events in complete detail.

Machine learning has revolutionized the development of autonomous systems by enabling capabilities that would otherwise be infeasible. However, the integration of learning-based components introduces fundamental challenges for safety certification. This paper presents a perspective on achieving the safety certification of learning-enabled autonomous systems without compromising safety standards, with a focus on autonomous vehicles. This perspective explores ongoing research addressing the challenge of safety certification for learning-enabled systems and outlines promising future directions. The key recommendation is the use of certifiable classical software components to ensure that the autonomous system consistently fulfills its safety objectives, regardless of potential faults in learning-based components. However, the lack of certifiable classical alternatives for many safety-critical learning-enabled tasks poses a substantial challenge to the realization of such architectures.