DC11 Project:Enhancing Trustworthy AI Integration in Safety-Critical Systems

PhD Supervisor: Prof. Shen Yin (NTNU); Auxiliary supervisors: David Camacho (UPM); Volker Stolz (HVL), Francesco Piccialli (UNINA); R&D cooperation: CONTI

Objectives: The primary objectives of this research are to develop AI algorithms that instill a high degree of trustworthiness within safety-critical systems in order to ensure reliability and predictability. Emphasis will be placed on improving the transparency and interpretability of the AI algorithms, thereby enabling operators and decision-makers to comprehend and trust their decision-making processes. Additionally, the research will attempt to create AI algorithms with robust fault tolerance mechanisms to maintain system stability and safety during abnormal scenarios. Furthermore, it will aim to ensure that the design and implementation of the AI algorithms align with the requirements of functional safety standards including estimating performance and meeting safety specifications.

Expected Results:  The DC11 will deliver highly trustworthy AI algorithms for safety-critical systems, which will help to accelerate the green transition. The outcomes will include practical solutions for transparent AI, which will foster trust among operators and decision-makers. Additionally, robust fault-tolerant AI algorithms will ensure system stability. The research will also provide performance assessment tools and compliance guidelines, which will improve system reliability for support during the green transition.

The research result of this PhD project will be applied to the cyber-physical electricity systems in the National Smart Grid Laboratory at NTNU/SINTEF lab in collaboration with the NTNU Cyber Range.  Applied research: Prototypes will be developed for and evaluated with cooperation with CONTI, to verify of correct operation in production conditions, improve the ongoing processes and testing procedures.

Planned secondments: UPM(4 months); HVL(4 months); UNINA (4 months)                   

Enrolment in Doctoral degree: NTNU