Advances in deep learning have revolutionized cyber-physical applications, including the development of autonomous vehicles. However, real-world collisions involving autonomous control of vehicles have raised significant safety concerns regarding the use of deep neural networks (DNNs) in safety-critical tasks, particularly perception. The inherent unverifiability of DNNs poses a key challenge in ensuring their safe and reliable operation. In this work, we propose perception simplex (PS), a fault-tolerant application architecture designed for obstacle detection and collision avoidance. We analyse an existing LiDAR-based classical obstacle detection algorithm to establish strict bounds on its capabilities and limitations. Such analysis and verification have not been possible for deep learning-based perception systems yet. By employing verifiable obstacle detection algorithms, PS identifies obstacle existence detection faults in the output of unverifiable DNN-based object detectors. When faults with potential collision risks are detected, appropriate corrective actions are initiated. Through extensive analysis and software-in-the-loop simulations, we demonstrate that PS provides deterministic fault tolerance against obstacle existence detection faults, establishing a robust safety guarantee.
@article{bansal2024perception,author={Bansal, Ayoosh and Kim, Hunmin and Yu, Simon and Li, Bo and Hovakimyan, Naira and Caccamo, Marco and Sha, Lui},journal={Software Testing, Verification and Reliability},pages={e1879},publisher={Wiley Online Library},title={Perception simplex: Verifiable collision avoidance in autonomous vehicles amidst obstacle detection faults},year={2024},doi={10.1002/stvr.1879},url={https://doi.org/10.1002/stvr.1879},eprint={https://doi.org/10.1002/stvr.1879},bibtex_show=true,}
Conference
Synergistic Perception and Control Simplex for Verifiable Safe Vertical Landing
Ayoosh Bansal, Yang Zhao, James Zhu, Sheng Cheng, Yuliang Gu, Hyung Jin Yoon, Hunmin Kim, Naira Hovakimyan, and Lui R Sha
Perception, Planning, and Control form the essential components of autonomy in advanced air mobility. This work advances the holistic integration of these components to enhance the performance and robustness of the complete cyber-physical system. We adapt Perception Simplex, a system for verifiable collision avoidance amidst obstacle detection faults, to the vertical landing maneuver for autonomous air mobility vehicles. We improve upon this system by replacing static assumptions of control capabilities with dynamic confirmation, i.e., real-time confirmation of control limitations of the system, ensuring reliable fulfillment of safety maneuvers and overrides, without dependence on overly pessimistic assumptions. Parameters defining control system capabilities and limitations, e.g., maximum deceleration, are continuously tracked within the system and used to make safety-critical decisions. We apply these techniques to propose a verifiable collision avoidance solution for autonomous aerial mobility vehicles operating in cluttered and potentially unsafe environments.
@inproceedings{bansal2024synergistic,author={Bansal, Ayoosh and Zhao, Yang and Zhu, James and Cheng, Sheng and Gu, Yuliang and Yoon, Hyung Jin and Kim, Hunmin and Hovakimyan, Naira and Sha, Lui R},booktitle={AIAA SCITECH 2024 Forum},pages={1167},title={Synergistic Perception and Control Simplex for Verifiable Safe Vertical Landing},year={2024},doi={10.2514/6.2024-1167},url={https://arc.aiaa.org/doi/abs/10.2514/6.2024-1167},eprint={https://arc.aiaa.org/doi/pdf/10.2514/6.2024-1167},bibtex_show=true,}
Journal
Taming Algorithmic Priority Inversion in Mission-Critical Perception Pipelines
Shengzhong Liu, Shuochao Yao, Xinzhe Fu, Rohan Tabish, Simon Yu, Ayoosh Bansal, Heechul Yun, Lui Sha, and Tarek Abdelzaher
@article{liu2024taming,author={Liu, Shengzhong and Yao, Shuochao and Fu, Xinzhe and Tabish, Rohan and Yu, Simon and Bansal, Ayoosh and Yun, Heechul and Sha, Lui and Abdelzaher, Tarek},journal={Communications of the ACM},number={2},pages={110--117},publisher={ACM New York, NY, USA},title={Taming Algorithmic Priority Inversion in Mission-Critical Perception Pipelines},volume={67},year={2024},bibtex_show=true,}
Conference
Bayesian Data Augmentation and Training for Perception DNN in Autonomous Aerial Vehicles
Ashik E Rasul, Humaira Tasnim, Hyung-Jin Yoon, Ayoosh Bansal, Duo Wang, Naira Hovakimyan, Lui Sha, and Petros Voulgaris
@misc{rasul2024bayesian,archiveprefix={arXiv},author={Rasul, Ashik E and Tasnim, Humaira and Yoon, Hyung-Jin and Bansal, Ayoosh and Wang, Duo and Hovakimyan, Naira and Sha, Lui and Voulgaris, Petros},eprint={2412.07655},primaryclass={cs.RO},title={Bayesian Data Augmentation and Training for Perception DNN in Autonomous Aerial Vehicles},url={https://arxiv.org/abs/2412.07655},year={2024},bibtex_show=true,}
Conference
Verification and Validation of a Vision-Based Landing System for Autonomous VTOL Air Taxis
Ayoosh Bansal, Duo Wang, Mikael Yeghiazaryan, Yangge Li, Chuyuan Tao, Hyung-Jin Yoon, Prateek Arora, Christos Papachristos, Petros Voulgaris, Sayan Mitra, Lui Sha, and Naira Hovakimyan
2024
Ayoosh Bansal, Duo Wang, and Mikael Yeghiazaryan contributed equally to this work.
@misc{bansal2024verification,archiveprefix={arXiv},author={Bansal, Ayoosh and Wang, Duo and Yeghiazaryan, Mikael and Li, Yangge and Tao, Chuyuan and Yoon, Hyung-Jin and Arora, Prateek and Papachristos, Christos and Voulgaris, Petros and Mitra, Sayan and Sha, Lui and Hovakimyan, Naira},eprint={2412.08102},primaryclass={cs.RO},title={Verification and Validation of a Vision-Based Landing System for Autonomous VTOL Air Taxis},url={https://arxiv.org/abs/2412.08102},year={2024},note={Ayoosh Bansal, Duo Wang, and Mikael Yeghiazaryan contributed equally to this work.},bibtex_show=true,}
2023
Journal
System Auditing for Real-Time Systems
Ayoosh Bansal, Anant Kandikuppa, Monowar Hasan, Chien-Ying Chen, Adam Bates, and Sibin Mohan
ACM Transactions on Privacy and Security (TOPS), Sep 2023
System auditing is an essential tool for detecting malicious events and conducting forensic analysis. Although used extensively on general-purpose systems, auditing frameworks have not been designed with consideration for the unique constraints and properties of Real-Time Systems (RTS). System auditing could provide tremendous benefits for security-critical RTS. However, a naı̈ve deployment of auditing on RTS could violate the temporal requirements of the system while also rendering auditing incomplete and ineffectual. To ensure effective auditing that meets the computational needs of recording complete audit information while adhering to the temporal requirements of the RTS, it is essential to carefully integrate auditing into the real-time (RT) schedule. This work adapts the Linux Audit framework for use in RT Linux by leveraging the common properties of such systems, such as special purpose and predictability. Ellipsis, an efficient system for auditing RTS is devised that learns the expected benign behaviors of the system and generates succinct descriptions of the expected activity. Evaluations using varied RT applications show that Ellipsis reduces the volume of audit records generated during benign activity by up to 97.55%, while recording detailed logs for suspicious activities. Empirical analyses establish that the auditing infrastructure adheres to the properties of predictability and isolation that are important to RTS. Furthermore, the schedulability of RT task sets under audit is comprehensively analyzed to enable the safe integration of auditing in RT task schedules.
@article{bansal2023system,address={New York, NY, USA},author={Bansal, Ayoosh and Kandikuppa, Anant and Hasan, Monowar and Chen, Chien-Ying and Bates, Adam and Mohan, Sibin},doi={10.1145/3625229},issn={2471-2566},journal={ACM Transactions on Privacy and Security (TOPS)},keywords={security auditing, model-based reduction, cyber-physical systems},month=sep,publisher={Association for Computing Machinery},title={System Auditing for Real-Time Systems},url={https://doi.org/10.1145/3625229},year={2023}}
Journal
SchedGuard++: Protecting against Schedule Leaks Using Linux Containers on Multi-Core Processors
Jiyang Chen, Tomasz Kloda, Rohan Tabish, Ayoosh Bansal, Chien-Ying Chen, Bo Liu, Sibin Mohan, Marco Caccamo, and Lui Sha
ACM Transactions on Cyber-Physical Systems, Sep 2023
@article{chen2023schedguard++,title={SchedGuard++: Protecting against Schedule Leaks Using Linux Containers on Multi-Core Processors},author={Chen, Jiyang and Kloda, Tomasz and Tabish, Rohan and Bansal, Ayoosh and Chen, Chien-Ying and Liu, Bo and Mohan, Sibin and Caccamo, Marco and Sha, Lui},journal={ACM Transactions on Cyber-Physical Systems},volume={7},number={1},pages={1--25},year={2023},publisher={ACM New York, NY},bibtex_show=true}
2022
Preprint
Ellipsis: Towards Efficient System Auditing for Real-Time Systems
Ayoosh Bansal, Anant Kandikuppa, Chien-Ying Chen, Monowar Hasan, Adam Bates, and Sibin Mohan
@article{bansal2022ellipsis,title={Ellipsis: Towards Efficient System Auditing for Real-Time Systems},author={Bansal, Ayoosh and Kandikuppa, Anant and Chen, Chien-Ying and Hasan, Monowar and Bates, Adam and Mohan, Sibin},journal={arXiv preprint arXiv:2208.02699},year={2022},bibtex_show=true}
Conference
Verifiable obstacle detection
Ayoosh Bansal, Hunmin Kim, Simon Yu, Bo Li, Naira Hovakimyan, Marco Caccamo, and Lui Sha
In 2022 IEEE 33rd International Symposium on Software Reliability Engineering (ISSRE) , Sep 2022
@inproceedings{bansal2022verifiable,title={Verifiable obstacle detection},author={Bansal, Ayoosh and Kim, Hunmin and Yu, Simon and Li, Bo and Hovakimyan, Naira and Caccamo, Marco and Sha, Lui},booktitle={2022 IEEE 33rd International Symposium on Software Reliability Engineering (ISSRE)},pages={61--72},year={2022},organization={IEEE},bibtex_show=true,}
Preprint
Synergistic Redundancy: Towards Verifiable Safety for Autonomous Vehicles
Ayoosh Bansal, Simon Yu, Hunmin Kim, Bo Li, Naira Hovakimyan, Marco Caccamo, and Lui Sha
@article{bansal2022synergistic,title={Synergistic Redundancy: Towards Verifiable Safety for Autonomous Vehicles},author={Bansal, Ayoosh and Yu, Simon and Kim, Hunmin and Li, Bo and Hovakimyan, Naira and Caccamo, Marco and Sha, Lui},journal={arXiv preprint arXiv:2209.01710},year={2022},bibtex_show=true}
Conference
Towards Efficient Auditing for Real-Time Systems
Ayoosh Bansal, Anant Kandikuppa, Chien-Ying Chen, Monowar Hasan, Adam Bates, and Sibin Mohan
In European Symposium on Research in Computer Security , Sep 2022
@inproceedings{bansal2022towards,title={Towards Efficient Auditing for Real-Time Systems},author={Bansal, Ayoosh and Kandikuppa, Anant and Chen, Chien-Ying and Hasan, Monowar and Bates, Adam and Mohan, Sibin},booktitle={European Symposium on Research in Computer Security},pages={614--634},year={2022},organization={Springer Nature Switzerland Cham},bibtex_show=true,}
2021
Conference
SchedGuard: Protecting against Schedule Leaks Using Linux Containers
Jiyang Chen, Tomasz Kloda, Ayoosh Bansal, Rohan Tabish, Chien-Ying Chen, Bo Liu, Sibin Mohan, Marco Caccamo, and Lui Sha
In 27th IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS’21) , Sep 2021
@inproceedings{chen2021schedguard,title={SchedGuard: Protecting against Schedule Leaks Using Linux Containers},author={Chen, Jiyang and Kloda, Tomasz and Bansal, Ayoosh and Tabish, Rohan and Chen, Chien-Ying and Liu, Bo and Mohan, Sibin and Caccamo, Marco and Sha, Lui},booktitle={27th IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS'21)},year={2021},bibtex_show=true}
Conference
Risk Ranked Recall: Collision Safety Metric for Object Detection Systems in Autonomous Vehicles
Ayoosh Bansal, Jayati Singh, Micaela Verucchi, Marco Caccamo, and Lui Sha
In 2021 10th Mediterranean Conference on Embedded Computing (MECO) , Sep 2021
@inproceedings{bansal2021risk,title={Risk Ranked Recall: Collision Safety Metric for Object Detection Systems in Autonomous Vehicles},author={Bansal, Ayoosh and Singh, Jayati and Verucchi, Micaela and Caccamo, Marco and Sha, Lui},booktitle={2021 10th Mediterranean Conference on Embedded Computing (MECO)},year={2021},bibtex_show=true,}
Journal
Real-time task scheduling for machine perception in intelligent cyber-physical systems
Shengzhong Liu, Shuochao Yao, Xinzhe Fu, Huajie Shao, Rohan Tabish, Simon Yu, Ayoosh Bansal, Heechul Yun, Lui Sha, and Tarek Abdelzaher
@article{liu2021real,title={Real-time task scheduling for machine perception in intelligent cyber-physical systems},author={Liu, Shengzhong and Yao, Shuochao and Fu, Xinzhe and Shao, Huajie and Tabish, Rohan and Yu, Simon and Bansal, Ayoosh and Yun, Heechul and Sha, Lui and Abdelzaher, Tarek},journal={IEEE Transactions on Computers},volume={71},number={8},pages={1770--1783},year={2021},publisher={IEEE},bibtex_show=true}
Preprint
Lidar cluster first and camera inference later: A new perspective towards autonomous driving
Jiyang Chen, Simon Yu, Rohan Tabish, Ayoosh Bansal, Shengzhong Liu, Tarek Abdelzaher, and Lui Sha
@article{chen2021lidar,title={Lidar cluster first and camera inference later: A new perspective towards autonomous driving},author={Chen, Jiyang and Yu, Simon and Tabish, Rohan and Bansal, Ayoosh and Liu, Shengzhong and Abdelzaher, Tarek and Sha, Lui},journal={arXiv preprint arXiv:2111.09799},year={2021},bibtex_show=true}
2020
Conference
On removing algorithmic priority inversion from mission-critical machine inference pipelines
Shengzhong Liu, Shuochao Yao, Xinzhe Fu, Rohan Tabish, Simon Yu, Ayoosh Bansal, Heechul Yun, Lui Sha, and Tarek Abdelzaher
In 2020 IEEE Real-Time Systems Symposium (RTSS) , Sep 2020
@inproceedings{liu2020removing,title={On removing algorithmic priority inversion from mission-critical machine inference pipelines},author={Liu, Shengzhong and Yao, Shuochao and Fu, Xinzhe and Tabish, Rohan and Yu, Simon and Bansal, Ayoosh and Yun, Heechul and Sha, Lui and Abdelzaher, Tarek},booktitle={2020 IEEE Real-Time Systems Symposium (RTSS)},pages={319--332},year={2020},organization={IEEE},bibtex_show=true}
Conference
Reconciling predictability and coherent caching
Ayoosh Bansal, Jayati Singh, Yifan Hao, Jen-Yang Wen, Renato Mancuso, and Marco Caccamo
In 2020 9th Mediterranean Conference on Embedded Computing (MECO) , Sep 2020
@inproceedings{bansal2020reconciling,title={Reconciling predictability and coherent caching},author={Bansal, Ayoosh and Singh, Jayati and Hao, Yifan and Wen, Jen-Yang and Mancuso, Renato and Caccamo, Marco},booktitle={2020 9th Mediterranean Conference on Embedded Computing (MECO)},pages={1--6},year={2020},organization={IEEE},bibtex_show=true,}
2019
Preprint
Sidebar: Scratchpad Based Communication Between CPUs and Accelerators
Ayoosh Bansal, Chance Coats, Evan Lissoos, and Benjamin Schreiber
@article{bansal2019sidebar,title={Sidebar: Scratchpad Based Communication Between CPUs and Accelerators},author={Bansal, Ayoosh and Coats, Chance and Lissoos, Evan and Schreiber, Benjamin},journal={arXiv preprint arXiv:1910.10794},year={2019},bibtex_show=true}
2018
Conference
Evaluating the memory subsystem of a configurable heterogeneous mpsoc
Ayoosh Bansal, Rohan Tabish, Giovani Gracioli, Renato Mancuso, Rodolfo Pellizzoni, and Marco Caccamo
In Workshop on Operating Systems Platforms for Embedded Real-Time Applications (OSPERT) , Sep 2018
@inproceedings{bansal2018evaluating,title={Evaluating the memory subsystem of a configurable heterogeneous mpsoc},author={Bansal, Ayoosh and Tabish, Rohan and Gracioli, Giovani and Mancuso, Renato and Pellizzoni, Rodolfo and Caccamo, Marco},booktitle={Workshop on Operating Systems Platforms for Embedded Real-Time Applications (OSPERT)},volume={7},pages={55},year={2018},bibtex_show=true}
2015
Conference
Thermal extension of the total bandwidth server
Rehan Ahmed, Ayoosh Bansal, Bhuvana Kakunoori, Parameswaran Ramanathan, and Kewal K Saluja
In 2015 28th International Conference on VLSI Design , Sep 2015
@inproceedings{ahmed2015thermal,title={Thermal extension of the total bandwidth server},author={Ahmed, Rehan and Bansal, Ayoosh and Kakunoori, Bhuvana and Ramanathan, Parameswaran and Saluja, Kewal K},booktitle={2015 28th International Conference on VLSI Design},pages={47--52},year={2015},organization={IEEE},bibtex_show=true}