Wherever possible I think that automating your testing is essential. For this project there is no way that I can manually rerun the same tests over and over again - the time commitment is just too much. I need to be able to click a button and have a framework run all my tests automatically and report back to me the number of tests passing, failing etc. I know that this won't catch all the bugs in the code and that I'll still need to do some manual testing for each release. However, the more I can push into this automated testing, the more time I'll save on the manual testing later.
Based on my previous post I've mentally divided up the code into different layers which build on top of each other. The app GUI is one layer. Then we have the model classes which in turn depend on utility classes etc. These different levels of code require different levels of testing.
For the utility and model layers I plan to use an automated unit testing framework such as JUnit. Xcode provides OCUnit which is a JUnit like framework for implementing unit tests. This framework provides the usual fail and assert macros and functions. There are some annoying things about running unit tests in Xcode. One is that it doesn't have a separate test runner panel like JUnit has in Eclipse. Instead it treats the unit test errors as build failures. All the print output for the tests is dumped into the same console output which can make it tricky to identify which output is for which test. Also there is no way to selectively run these tests. You have to rerun them all together every time. There are other frameworks out there e.g. GHUnit but for the moment I'll stick with the default.
For the higher level GUI code, unit testing is not useful so I'll need a different testing approach and toolset. There seem to be number of options out there. I'm not sure what is available and what is the best to use. Apple have an Instruments application which allows you do perform UI testing using Javascript. I've also seen these Frank and Zucchini frameworks. I haven't evaluated any of these yet as I don't have a fully functional app to test it on. Once I get a nearly complete app I can see how much my unit testing provides, compare that to what I have left to test, and then make a judgement on where to go from there.
→ posted on February 13, 2012development