Change Impact in Safety-Critical Systems Using Requirements Traceability Tools: Controlled Experiment

Grace Brooks; Jennifer Ma; Victor Kan

Authors

Grace Brooks Department of Civil and Environmental Engineering, Newark College of Engineering, New Jersey Institute of Technology, Newark, New Jersey, 07102, USA Author
Jennifer Ma Department of Industrial and Manufacturing Systems Engineering, Faculty of Engineering, University of Hong Kong, Hong Kong, Hong Kong SAR, China Author
Victor Kan Department of Industrial and Manufacturing Systems Engineering, Faculty of Engineering, University of Hong Kong, Hong Kong, Hong Kong SAR, China Author

Keywords:

Requirements Traceability, Change Impact Analysis, Safety-Critical Systems, Controlled Experiment, Software Engineering

Abstract

The increasing complexity of software systems in safety-critical domains such as aviation, healthcare, and railway transportation demands rigorous requirements traceability to ensure system reliability and compliance with stringent regulatory standards. When changes to requirements occur, accurate change impact analysis is essential to identify all affected artifacts, thereby preventing the introduction of unforeseen vulnerabilities. Despite the proliferation of requirements traceability tools designed to assist in this process, empirical evidence regarding their actual effectiveness and efficiency in industrial safety-critical contexts remains scarce. This paper presents a comprehensive controlled experiment designed to evaluate the capability of advanced requirements traceability tools in predicting change impact. By engaging professional software engineers in a simulated environment based on a train control management system, we meticulously measure the precision, recall, and time efficiency of participants conducting change impact analysis with and without specialized traceability support. The findings indicate that the utilization of automated traceability tools significantly enhances the accuracy of identifying impacted components while simultaneously reducing the cognitive load and time required by engineers. Furthermore, the study highlights critical challenges related to tool usability and the initial overhead of establishing trace links. These insights provide valuable guidance for practitioners aiming to optimize change management processes and for tool vendors seeking to refine their solutions for safety-critical applications.

References

1. Montero Guerra, J. M., Danvila-del-Valle, I., & Méndez-Suárez, M. (2023). The impact of digital transformation on talent management. Technological Forecasting and Social Change, 188, 122291.

2. Al-Banna, A.; Yaqot, M.; Menezes, B.C. Investment strategies in Industry 4.0 for enhanced supply chain resilience: An empirical analysis. Cogent Bus. Manag. 2024, 11, 2298187.

3. Okoli, C., & Schabram, K. (2010). A guide to conducting a systematic literature review of information systems. Sprouts.

4. Heavin, C., & Power, D. J. (2018). Challenges for digital transformation—Towards a conceptual decision support guide for managers. Journal of Decision System, 27(Suppl. S1), 38–45.

5. Lee, W.; Sunwoo, D.; Gerstlauer, A.; John, L.K. Cloud-Guided QoS and Energy Management for Mobile Interactive Web Applications. In Proceedings of the 2017 IEEE/ACM 4th International Conference on Mobile Software Engineering and Systems (MOBILESoft), Buenos Aires, Argentina, 22–23 May 2017; pp. 25–29.

6. Di Nardo, V.; Fino, R.; Fiore, M.; Mignogna, G.; Mongiello, M.; Simeone, G. Usage of Gamification Techniques in Software Engineering Education and Training: A Systematic Review. Computers 2024, 13, 196.

7. Kuhlmann, S., & Heuberger, M. (2023). Digital transformation going local: Implementation, impacts and constraints from a German perspective. Public Money & Management, 43(2), 147–155.

8. Fuchs, J.; Schneider, R.; Oks, S.; Franke, J. Service-based integration of modular control components in digital manufacturing platforms. In Proceedings of the IEEE 19th International Conference on Industrial Informatics (INDIN), Palma de Mallorca, Spain, 21–23 July 2021.

9. Ferreira, J.; Santos, B.; Oliveira, W.; Antunes, N.; Cabral, B.; Fernandes, J.P. On Security and Energy Efficiency in Android Smartphones. In Proceedings of the 2023 IEEE/ACM 10th International Conference on Mobile Software Engineering and Systems (MOBILESoft), Melbourne, Australia, 14–15 May 2023; pp. 87–95.

10. Kessel, L., & Graf-Vlachy, L. (2022). Chief digital officers: The state of the art and the road ahead. Management Review Quarterly, 72(4), 1249–1286.

11. Moi, L., & Cabiddu, F. (2021). Leading digital transformation through an Agile marketing capability: The case of Spotahome. Journal of Management & Governance, 25(4), 1145–1177.

12. Enyejo, J.O.; Fajana, O.P.; Jok, I.S.; Ihejirika, C.J.; Awotiwon, B.O.; Olola, T.M. Digital twin technology, predictive analytics, and sustainable project management in global supply chains for risk mitigation, optimization, and carbon footprint reduction through green initiatives. Int. J. Innov. Sci. Res. Technol. 2024, 9, 1344.

13. Jin, M.; Chen, Y. Has green innovation been improved by intelligent manufacturing?—Evidence from listed Chinese manufacturing enterprises. Technol. Forecast. Soc. Change 2024, 205, 123487.

14. Collart, A.J.; Canales, E. How might broad adoption of blockchain-based traceability impact the US fresh produce supply chain? Appl. Econ. Perspect. Policy 2022, 44, 219–236.

15. Tsekouropoulos, G., Vasileiou, A., Hoxha, G., Theocharis, D., Theodoridou, E., & Grigoriadis, T. (2025). Leadership 4.0: Navigating the challenges of the digital transformation in healthcare and beyond. Administrative Sciences, 15(6), 194.

16. Shadish, W.R.; Cook, T.D.; Campbell, D.T. Experimental and Quasi-Experimental Designs for Generalized Causal Inference; Houghton, Mifflin and Company: Boston, MA, USA, 2002.