Zachary Sally
University of Central Florida
Software Engineer Intern
NASA - Kennedy Space Center
Summer 2023
Overview
During the Summer of 2023, I had the opportunity to come work for NASA at Kennedy Space Center as a software engineer intern under the Spaceport Command and Control System. I worked with the team responsible for monitoring and controlling the systems involved during launch operations.
What I Contributed
Overview
Throughout the term, I contributed to the development of Class A, human-rated, safety critical group support software used to test the Launch Control System's (LCS) capabilities. Specifically, I constructed Computer Software Configuration Item (CSCI) Functional Tests (CFTs) to test a system monitor. This system monitor provides operators information about different system data, including CPU usage, disk space usage, swap space usage, and the status of tracked processes. The monitor will alert operators when any of these usage levels exceed a certain threshold and give updates on the status of tracked processes.
Development Process
To construct these CFTs, I utilized a keyword-driven automated testing framework. In addition, I spearheaded the development and application of system call mocks that can be used by the monitor and any other software application that requires mock system calls for testing. Once the CFTs were drafted, I participated in code reviews with technical experts across different disciplines. Finally, I documented my procedures in a way that allows engineers and new interns to have a streamlined process to review and expand the CFTs.
Developing these CFTs involved extensive research and collaboration with peers. I analyzed existing documentation and source code for both the system monitor and the automated testing framework to learn how to write the CFTs. I collaborated closely with my team's technical points of contact to learn how the monitor works and how it fits into LCS. Also, I participated in status meetings with my team to give updates and raise concerns throughout the development lifecycle.
Project Impact
The CFTs I built will optimize the testing of the system monitor in the firing rooms. These automated tests will help Master Console Operators by removing the need to manually set up and run the tests. This will accelerate the speed at which these tests will be done, allowing the operators to focus on other priorities prior to launch operations.
Encountered Challenges & Solutions
Mocking System Calls for Testing
System Overview: The source code for the system monitor included system calls from standard libraries. These system calls would be used to calculate disk and swap space usage on the system. The monitor would then report whether the usage has reached preconfigured threshold values. For our CFTs, we needed to test that the monitor is able to report that usage has exceeded each preconfigured threshold.
​
Problem: For our CFTs, we needed to figure out a way to mock these calls in order to get a pre-determined output. In the case of the system monitor I worked on, it was not preferable, and in some cases impossible, to just fill directories with dummy data to reach disk and swap space usage threshold levels. It was also preferable to not alter the source code or recompile it, as that defeats the point of the tests. Therefore, we had to find a way to intercept the system call and change the values returned during run-time. This problem spanned multiple applications both within my intern group and other CSCIs.
​
Solution: I solved this problem by using Test-Driven development. I wrote small tests that called the functions I wanted to mock and verified that the data returned was my dummy data. There were two main issues to tackle:
-
I needed to create a mock shared library that alters the data returned by the original system call. In my case, I wanted to call the actual system call within my mock to verify directories were valid and so I didn't need to handle dynamic memory management associated with the system call. After relentless googling, I found a function that allowed me to access the original system call, and implemented it into my mocks. Once I called the original system call, I modified the data and returned it.
-
Now that I had the mock shared library, I needed a way to call it BEFORE the system call. I collaborated with fellow interns who were had similar issues with system calls and we eventually found an environment variable that allowed us to load our mock version before the system call at compile time.
With those two issues solved, I was able to get my dummy data that allowed me to test my application. It was simple to integrate the mocks into the automated tests. I made sure to document my process and supply templates of these mocks so that interns and engineers can replicate the process for any application that requires it for testing.
​
What I Learned: My impact as an intern is much more expansive than I previously thought. I was able to successfully identify, analyze, research, and test solutions for a problem that spans across multiple CSCIs. I helped paved the path for system call mocks, which will have a massive impact on future test development at Kennedy Space Center.