Any IT application requires testing before a successful Go-Live. The automation of these tasks allows the inexpensive repetition and continuous monitoring of functionalities and has become commonplace. It is an integral part of modern Continuous Integration and Continuous Delivery (CI/CD). As costs for implementation and adaptation are the driving factors in deciding whether to automate testing, this will be the focus of this paper. By hiring 12 test automation professionals where each of them has the same task but uses their respective favourite tool, we can benchmark these tools. Assuming that all tools find the correct issues, we will evaluate the tools based on the time it takes to create and adapt tests.