- Home
- About TUAI
- Research
- Doctoral Candidates
- Networking
- Consortium
- Contact
WP5 – Research Progress 2026
WP5 aims to develop sustainable and trustworthy AI solutions for safety-critical cyber-physical systems and practical industrial applications. The work focuses on two closely related challenges: ensuring that AI systems operating in safety-critical environments are transparent, interpretable and resilient to faults, and developing sustainable methods for real-time anomaly detection across different industrial sectors. Particular attention is given to maintaining reliable performance under demanding operational conditions while taking into account resource efficiency, scalability, data privacy, robustness and the requirements of functional safety.
The Work Package addresses these objectives through three complementary research directions. The first focuses on strengthening trust in safety-critical AI systems through transparent and robust algorithms, with particular relevance to sectors such as automotive and aerospace. The second investigates sustainable anomaly detection methods, evaluating them against criteria including energy efficiency, scalability, privacy, resilience, interpretability and adaptability. The third seeks to combine real-time responsiveness with trustworthy AI through pilot applications in which both operational performance and trustworthiness are assessed. During the Summer School, doctoral candidates involved in WP5 presented their individual research projects in the context of these objectives, illustrating different approaches to making AI systems more reliable, resource-efficient and suitable for practical deployment in safety-critical and other demanding environments.
Dina Tri Utari (NTNU) presented her research on developing a comprehensive framework for assessing the trustworthiness and sustainability of AI in safety-critical cyber-physical systems (CPS). Her work contributes to WP5 by addressing the need for systematic and quantitative methods to evaluate AI systems beyond conventional performance metrics, particularly when failures may have serious operational or societal consequences. The research focuses on transparency, robustness, fault tolerance and alignment with industrial safety requirements, with potential applications in automotive systems, industrial automation, medical devices and aviation.
The main contribution presented was a multi-dimensional Trustworthiness Index (TWI) for AI-enabled safety-critical CPS. The framework comprises eight dimensions: human agency, robustness and safety, privacy, interpretability, fairness, sustainability, accountability, and security and cyber-resilience. It integrates evidence from three stages of the AI lifecycle: development processes, verification and operational performance. Dimension scores are calculated from underlying metrics, while non-compensatory thresholds for critical dimensions prevent strong results in some areas from masking a critical weakness in another. The framework also connects domain-specific safety standards, including ISO 26262, IEC 61508 and IEC 62304, with horizontal AI regulation such as the EU AI Act.
Dina presented an illustrative assessment of an automotive Level 3 automated driving system, obtaining a TWI score of 0.7462. The analysis identified interpretability as a major gap, with a score of 0.58, while accountability was identified as a relative strength, scoring 0.88. Further analyses explored the sensitivity of the index to changes in dimension weights and demonstrated how assessment priorities differ between automotive, industrial and medical applications. The presentation also outlined the limitations of the current work, including reliance on expert estimates and synthetic scenarios, uncalibrated domain-specific thresholds, and the need for a stronger theoretical foundation. Future work will address real-world validation, stakeholder-based weighting, threshold calibration and formal analysis of the framework. The research progress included a paper accepted and presented at RAMS-Europe 2026, a related paper on a composite human agency–fairness index submitted to IEEE Transactions on Artificial Intelligence, and ongoing work on robustness and interpretability indices.
ENHANCHING_TRUSTWORTHY_AI_INTEGRATION_IN_SAFETY-CRITICAL_SYSTEM.pdf
Athanasia Kollarou (NTNU) presented her research on sustainable real-time anomaly detection, focusing on the requirements that AI-based detectors must satisfy to be deployable in practical and safety-critical cyber-physical systems. Her work contributes to WP5 by addressing the need to combine detection performance with resource efficiency, real-time responsiveness, operational resilience, data confidentiality, adversarial robustness and explainability. An important aspect of the research is that high detection accuracy alone does not establish that an anomaly detection system is ready for real-world deployment.
The first major contribution was a systematic survey and deployability assessment of anomaly detection methods for industrial cyber-physical systems and industrial control systems (CPS/ICS). Using a PRISMA-based literature review, Athanasia assessed 26 implementation studies and derived nine deployability criteria: detection accuracy, operational resilience, adaptability, real-time responsiveness, resource efficiency, adversarial robustness, data confidentiality, explainability and verification. The analysis showed that detection accuracy was considered in all 26 studies, whereas coverage of the other criteria varied considerably. In particular, adversarial robustness and data confidentiality were among the least-supported dimensions. This work provides an evidence-based framework for assessing deployment readiness and identifying gaps between reported model performance and practical operational requirements. The associated paper, “Towards Deployable Anomaly Detection in Industrial CPS/ICS: A Survey and Deployability Assessment,” was reported as being under major revision at the International Journal of Information Security.
Building on these findings, Athanasia presented a second study, “From Adversarial Threat Models to Security Requirements for Anomaly Detectors,” submitted to CyberICPS 2026 and reported as under review. This work analyses how anomaly detectors themselves may be targeted by poisoning, evasion and privacy attacks. Using the Security Threat-Oriented Requirements Engineering (STORE) methodology and the NIST AI 100-2e taxonomy, the study maps 15 attack subclasses to stages of the anomaly-detection lifecycle, reviews corresponding defence strategies, evaluates risks qualitatively and derives security requirements linked to the attacks they are intended to mitigate. At the time of the Summer School, the systematic state-of-the-art analysis and security-requirements work had been completed, while the design of a sustainable real-time anomaly-detection solution was starting. Implementation, demonstration on safety-critical CPS and evaluation using sustainability metrics remained future work. The next steps include finalising the revised survey, designing and implementing the solution based on the identified criteria and security requirements, and evaluating its practical performance and sustainability.
Sustainable_Real-time_Anomaly_Detection_for_Practical_Applications.pdf