In this paper, we propose and develop a test automation program tool that automates GUI based testing using black box testing technique in Windows environment. The main features of the proposed test automation program tool are as follows. First, an error condition is designated as an image, a screen is captured for each test step, and an error message is detected through comparison of image similarity. Second, the proposed system supports various setting options such as event waiting time during execution and coordinate increment value between each test step. Such black box test automation research was common in environments such as Android and Web, but not in Windows environment. The results of performance evaluation show that the proposed system performs GUI test automation as an image comparison module and confirms that the test is performed normally by confirming process status and error image detection.