I spent a lot of time in my childhood and early twenties traveling on the Irish Rail Network. Mostly this seemed to involve waiting for ages at Limerick Junction or barely finding a space to stand on a packed train from Dublin. Those experiences didn't exactly leave me with a great impression of Iarnród Éireann.
Given this, I was amazed to find out recently, that Iarnród Éireann have an XML API for accessing realtime data about the trains running on their network. Credit where credit is due, this is an excellent idea and I wish more companies would implement something similar.
The API provides functionality for getting a list of the stations and what trains are due at those stations in the next ninety minutes. It also gives a list of all trains active on the network. It can filter by DART, Suburban or Mainline trains (that leaves some trains in an 'other' category - I'm not too sure what these actually are).
I used this API as part of a Backbone learning project. It was quite fun to do. The API returns latitude and longitude coordinates for each station and train, allowing them to be plotted on a Google Map widget. I never realised there were so many stations in Ireland until I saw them plotted on the map.
One issue I ran into was testing the application with live data. Given that I was programming this at night after work, you'd soon reach a time when there are very few trains left on the network!
One technical detail about the API is that it is in XML rather than JSON. This means that I can't use Ajax or JSONP to get the data, due to the same origin policy. Instead I had to bounce the results through YQL. YQL exposes a SQL like interface to web data. I'm only using a basic 'select all' query here but looking at their site you can do lots of cool and complex stuff. I found a good tutorial from Cypress North on how to use YQL in your code to consume an XML API.
→ posted on April 23, 2013development
I found this talk by Uncle Bob Martin to be really enjoyable and thought provoking. In it he talks about decoupling your application from the database and the web. It's ostensibly a Rails talk but I think it applies to every language.
It really crystalized the issues I've been having with developing Objective-C applications i.e. the intermixing of the database (specifically Core Data) and UI code. After viewing this, I was able to come to iOS development with a whole new perspective. It's so much clearer to me now how to architect applications, where code belongs and what interfaces are needed. After completely separating Core Data and UI code, the whole application has gotten a lot simpler and is much easier to unit test.
It also introduced me to Alistair Cockburn's Hexagonal Architecture. This is a really nice architecture for isolating the application from external entities such as the GUI or database. I've seen this crop up more and more on various Rails blogs under the term 'Hexagonal Rails'.
As a reference, here and here are a couple of other links from Bob Martin's blog talking about "clean architecture".
→ posted on April 17, 2013architecture
I've been working on a side project for the last month in order to learn about front-end web development. While not complete by any means, I think it's good enough to release as a beta. I plan to add features, improve the UI and refactor the code in future.
The motivation behind it was that I wanted to do a project using one of the JavaScript MVC Frameworks - for this one I chose Backbone. Also I'll be working more with HTML and CSS in work over the coming year and I need to ramp up on these.
Iarnród Éireann (the Irish Rail company) has a REST API which provides details of the stations in its network and the current status of its trains. My webpage plots these stations and trains on a Google Map and shows upcoming train information for each station.
I plan to write a series of posts on how I implemented this and the lessons I learned along the way.
The project is available here and the code is available on Github under the MIT licence. The roadmap.md document is where I'm tracking future work that needs to be done on the project. Any feedback would be much appreciated.
As far as the code goes:
→ posted on March 18, 2013projects
One of the things to learn about iOS is the MVC model. The Cocoa implementation of MVC has some differences compared to the traditional approach. For example, in Cocoa, views are not aware of the models and don't listen for model updates. Instead the all events pass through the controller, i.e. it listens for model changes and then tells the view to update itself.
In Rails, the constant refrain is that controllers should be thin. However on iOS they seem to be absolutely huge. One joke that I saw on Twitter was that on iOS, MVC stood for Massive View Controller. For example, the Recipe sample application from Apple has controllers with hundreds of lines of code with one topping out at 600 LOC.
One of my issues with these view controllers is that they don't follow the Single Responsibility Principle, but instead combine multiple functions. They act as delegates for multiple protocols e.g. table data source, fetched results controller delegate etc. I find it hard to distinguish the separate elements of MVC when one class is doing everything. Also in Objective C, once everything is in the same file, it's not obvious to which protocol an item belongs. I think this risks breaking the MVC boundaries. For example, during a refactor, if you're not careful, you can easily get model variables depending on controller variables and vice-versa.
I'd much prefer it if these controllers were split out into lots of different classes, each with a single job as per the SRP. This would lead to a more composition-based rather than inheritance-based codebase. I also think that this greatly helps with code navigation. Jumping to a small, focused file has the effect of filtering out irrelevant code. I've started using Sublime Text recently and it has great functionality for navigating between files, so I prefer having lots of smaller files rather than a few large monolithic classes.
(On a side note this is one thing that really annoys me about XCode. Given a properly nested folder structure with well named files, I think it becomes a lot easier to find your way around the app. For example even after only a few weeks learning Rails, I know exactly where to look to find the controllers, models, db code etc. But XCode is a disaster here. It doesn't push the groupings made in the app on to the file system underneath. It requires duplicate effort of organising the code both inside and outside the application to keep the codebase properly organised.)
→ posted on March 17, 2013development
I recently subscribed to Gary Bernhardt's Destroy All Software screencasts. I found these absolutely fantastic. There are only about ten to fifteen minutes each in length so it's no hassle to find the time to watch them. However they cover a lot in the short time.
They are mostly Ruby and in particular Rails focused but the topics are applicable to any software programming. For example, I've really learned a great deal about TDD from watching this.
The Github archive is useful, especially the Suck/Rocks Rails app. His Git config file is also really handy and I really like his log formatting. I would have liked to see him post a larger sample Rails app to Github (or for the ultimate in wishful thinking - the code for the Destroy All Software site). It would expand upon how his ideas work when confronted with the complexity and messiness of a real world system.
Overall I highly recommend the series for any programmer regardless of what language you are using. Unfortunately I believe he's stopping soon but you get access to the entire back catalog with a subscription and they're well worth it.
→ posted on February 27, 2013development
It's been a year, give or take, since I started looking into iOS development, and I wanted to write a post to reflect on where I am and what are the next steps.
Progress on iOS to Date
I spent the first three months of the year learning Objective C and hacking on some small programs.
I spent some time experimenting with Core Data and MVC and trying to get an automated testing system up and running.
The summer months were quiet due to work commitments. The project I was working on switched to JavaScript which I hadn't really used before. Due to this, I spent a lot of time learning JavaScript and web frontend and backend development.
After the summer I came back to the app with a better understanding of MVC as implemented in iOS and Core Data and I started working on it again. I read up on TDD and started implemented tests using Cucumber and Calabash. That took me up to about October.
Work has stalled on it since then due to a combination of real life commitments from both family life and work. And also Skyrim! Curse you Skyrim and the hours I sank into it :)
Current Status
Here's where I am currently
App work: I didn't get as much done as I had planned at the start of last year. I got diverted in other areas, namely:
Testing - TDD & BDD: I hadn't planned to look at any of this but ended up spending a lot of time here and learning a great deal. Familiarizing myself with this way of programming was probably the biggest benefit I gained from the last years experimentation. I probably could have completed the app if I didn't spend so much time on TDD and tinkering with automated testing setups on Jenkins. However I wouldn't have learned anywhere near as much about proper programming techniques if I did.
JavaScript: I spent a good deal of time learning proper JavaScript development. The basic syntax is very similar to Java which I already knew. However I wanted to learn how to program using JavaScript correctly i.e. modular code using RequireJS, TDD using Jasmine etc.
Ruby and Rails: again I had not planned on researching any of this but I got interested in it while looking at JavaScript. Have done a few tutorials and would like to write a Rails app at some stage.
I spent a lot of time over the last year becoming proficient in web app and backend development in JavaScript for a work project. I've been really enjoying this. My only previous experience of JavaScript and web development was back in college around 2000 and I wasn't too taken with it compared to Java and C development. But looking at it now and in combination with backends like Rails, I've been really impressed. I'm kind of kicking myself for tuning out on what was happening in web development over the past decade. I plan to branch out and do some posts on what I used to ramp up on this.
On the iOS front I feel that I have a good knowledge of Objective C. However the issue is finding time to finish the app. Any time I was able to set aside a sizable chunk of time I got a lot done but getting that time is the key.
Reflections
I think working on this side project has made me expand my horizons more as a developer. I couldn't have predicted the areas that I would have become interested in at the start of last year. I don't think it's a coincidence that as I've started looking at more stuff outside of work, that my work projects have become more interesting. I feel like I'm able to bring so much more to a project now especially in the areas of TDD and design.
Admittedly I haven't pursued the iOS work 100%, but I feel that the other languages I've been exposed to compensate for that. I think it was better to start out with iOS and then divert into other topics rather than not do any side work at all.
However at the end of the day the reality is that I haven't shipped anything on iOS yet. I've been justifying this to myself by claiming all these other intangible benefits. Given what I've learned, I don't think that I've wasted my time but still I'd like to get some iOS project finished. Unfortunately given my current life and work commitments I probably won't have time to work on it over the next three months. In the meantime, I'll post about my JavaScript and TDD experiences.
As regards the blog, looking back I am happy enough with what was posted in the past year. The main thing I'm not happy with is the regularity of the postings. The same things that have stopped me doing much software work, have also stopped me publishing much. In future I want to maintain a more regular posting rate. I aim to post at least once a fortnight and hopefully once a week.
→ posted on February 23, 2013development
I previously recommended the Stanford iOS course and they now have a companion book available for the iPad through iBooks.
The content of this book is excellent and it really complements the course videos. I really like the code samples and how they are displayed dynamically on the iPad. They have really used the iBooks format well for these. Additionally all the code is available on Github.
I highly recommend the book and I wish it was available when I originally watched the videos. Given the quality of the lectures and the book, Stanford could easily charge for this stuff - it was certainly equal to any introductory programming course I've attended at college. It seems inconceivable to me that this approach won't form the basis of education in the years to come.
It's maybe a bit churlish to criticise a free book but still I do have some issues. These concern the format the book was released in and not its content.
Firstly it was only released for iPad and not for iPhone. I would have liked to have had a reference version for iPhone.
Also I don't like the iBooks lock in. I really want to have this on my Macbook and/or Kindle to have open when programming. Here's where an epub or pdf version would be a lot better. The Pragmatic Programmers company have shown the way forward with their ebook releases, giving multiple DRM free versions of their books.
Apart from those minor nits though, this is an essential resource for learning iOS.
→ posted on January 28, 2013apple
I've been using Git for a while now, both at work and also for my iOS projects. I'm really liking it and have nearly sorted out my workflow with it. One problem that remains for me though is how to manage external dependencies. If I need to include a library from Github for example, how do I integrate it with my source.
The main issues that I want to solve are
What happens if the external library disappears? This is especially relevant for online sources like Github. Given the brittle nature of the web, I think we can take it as a given that a lot of these Github repositories will be deleted over the coming years. In this scenario I need to have a local copy of the code as a backup.
I should be able to update the local code with the latest or a specific tag from the external repository.
I want to be able to create common libraries for my own code which I can share between applications.
It may be that I will need to modify the code, so how do I maintain those changes. Perhaps I will need to push those changes to the external repository. This seems likely for my own common libraries.
I want to be able to check out all the code needed to build an application from the source control using a single checkout/clone command. This means that each application will have a full copy of all the code that is needed to build it.
One solution that I've seen used in other projects is to use git submodule links. A git submodule points to a version of an external repository. The ones I've seen for iOS libraries will usually point to Github. When you run the git submodule init command, git will follow these links and download the source into your project.
However in researching this, other people have reported problems using git submodules. They don't seem as simple and as usable as I would need. I'm seeing issues on sites regarding detached heads, branching, different versions etc. I just know that I will hit all these issues and probably a few more besides!
In looking for an alternative I came across git subtrees on Wolf Rentzsch's site. This seems to fit all my needs and appears less error prone than git submodules. The code can be downloaded here and here's a good tutorial on using subtrees.
My basic usage of it, i.e. to integrate a library from Github would be something like
git subtree add --prefix=GCUtils/External/CocoaLumberjack --squash https://github.com/robbiehanson/CocoaLumberjack.git master
I use the squash option to compress all the commit history of the library to just one commit message in my repository.
One issue I ran into was adding a project which itself contained submodules. In this situation I search and find all the .gitmodules files and add those manually as subtrees in my repository.
→ posted on November 5, 2012development
I've been implementing some new features in iOS and in the process refactoring some existing code. As I've been making these changes, I began to feel that the code was reaching a tipping point where I was as likely to break existing features as successfully add the new functionality. In particular I was experimenting with table view controllers and there are a lot of functions to override depending on what behaviour you want. I was finding that changes for one type of behaviour were impacting functionality that I thought I had solved previously. I hadn't been doing much automated testing up to now apart from some unit testing, so I was worried about adding regression bugs.
I decided to take a break from coding in order to implement some proper tests and specifically I wanted to do some UI testing. For this I chose the Calabash framework. This is developed by the people at Less Painful. One of them, Karl Krukow, has a detailed post comparing Calabash to the other automated frameworks out there.
Cucumber
Calabash is based on a Ruby test framework called Cucumber. Cucumber allows for tests to be written in high level language called Gherkin. Gherkin is written in natural language and so can be understood by anyone - not just developers. It acts as documentation as well as a test framework. I could see this as being very useful when writing an application for a client. By giving them the Cucumber tests, they can see in plain English what is being implemented and should be able to give constructive feedback. Also the tests are automated and are run from the command line.
While learning Cucumber the two main resources I used were
Installation
Cucumber requires Ruby to be installed on your system (I had previously installed it while setting up Octopress). The Calabash install process is documented on their Github page. I used the Fast Track installer using these steps.
gem install calabash-cucumbercalabash-ios setupcalabash-ios sim acccalabash-ios genThis created initial test files which you can run from the command line using cucumber.
One thing that worries me about the install process is that it creates a new scheme specifically for Calabash use. I don't really like this solution as it breaks DRY and requires keeping the original scheme and the new Calabash scheme in sync. From my experience in programming, anything that is required to be kept manually in sync, won't be. I would have preferred for the Calabash libraries to only be included in the Debug target of the main scheme or for a new target to be created on that scheme. Update: Since I wrote this post Trevor Harmon has been in touch with an alternative approach to just link in the libraries for the Debug build - more info here.
A good blog on the Calabash install and initial test setup is here.
First Steps with Calabash
I think the Calabash wiki is the best starting point for Calabash.
The first thing I tried was the console using calabash-ios console. I recommend this to play around and see what objects are visible to Calabash.
The command query("view") shows everything on screen. We can isolate specific views by filtering on the accessibility label. In some cases this wasn't useful for me. For example, I have a table where each cell has a text field. Each of these has the same accessibility label so instead I filtered using the :text property.
We can find out what accessibility labels are visible using label "view". Once we have a label we can do things like touching or swiping on the view or checking that the view exists using view_with_mark_exists().
The accessibility inspector can be used to identify the classes of UI elements and the on-screen hierarchy. Launch this from XCode under the menu item XCode->Open Developer Tool->Accessibility Inspector.
I found that the built-in Calabash steps were very useful to base mine on.
A problem I had was querying for the currently active textfield from a table of textfields. The issue was that all those textfields had the same placeholder text. To get the correct one I filtered by isUserInteractionEnabled.
Errors in the name of the selector passed to the backdoor command show up as connection errors.
Initially I struggled with writing the tests at the correct level. The temptation is to be really specific in terms of UI elements e.g. When I touch X button and swipe on Y label. However you really need to describe them at a higher level e.g. when I add a new contact, when I delete an appointment etc.
Initially I was unable to run a backdoor command in the before hook, which runs before each test. I had wanted to reset my Core Data database & UI at this time. The reason here is that the Calabash framework itself uses before hooks to connect to the app and I think my hook was being called before theirs. To solve this Calabash added support for defining an on_launch function which is called after the simulator has started. To implement use the following template in the hooks.rb file.
class CallbackWorld
include Calabash::Cucumber::Operations
def on_launch
# here I can call backdoor and reset the app state
end
end
World do
CallbackWorld.new
end
My typical test layout is to use
Issues with Calabash.
It wasn't all smooth sailing as I did run into a few issues along the way.
I wasn't able to get it to integrate with my installation of Jenkins, which is unfortunate as Cucumber can output in JUnit format so it's a perfect fit for Jenkins. The problem here is on my side as lots of users on the Calabash Google Group are running Calabash from their CI system. I think it due to the jenkins user not being able to launch the iOS simulator. I need to investigate further on this.
Update: Ru Cindrea emailed me to suggest another way of running Jenkins which may fix this problem.
I used to launch Jenkins like this: "sudo launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist" - which seems to be the recommended way on a lot of sites I have found.
However, I now launch it under my own user, using "nohup java -jar /Applications/Jenkins/jenkins.war --httpPort=8080" - and everything works perfectly! I can simply use "cucumber -f junit -o results -f pretty" to start the simulator, run the tests and get my JUnit report.
I had found that Calabash would frequently drop the connection to the app and fail the tests as a result. Once a test run started seeing these errors for a test (Unable to make connection to Calabash Server at http://localhost:37265/) then each following test in the run would show the same error. Rerunning the tests would normally sort it out in the next run or two. Both the app and Calabash would be running fine but just couldn't seem to connect to each other. This unreliability was the most disappointing part of the tests for me.
Update: Karl pointed me to the Google Group and I saw some updates that were needed to work with the new iOS 6 Simulator. I had seen that black screen issue as part of trying to get it working with Jenkins. So I've updated to the latest version of Calabash and on the first test run, everything worked fine. Hopefully this will resolve the issue in future.
Initial Impressions & Next Steps
I have to say that I'm very impressed with Calabash and indeed Cucumber in general. I think that writing these type of tests beforehand would be helpful for requirements gathering and feature design. The tests themselves are not only useful as tests but also as documentation. Unlike Word docs, we can be guaranteed that these accurately describe the current state of the system, given that they are actually run against it.
My plans for the future
Update: After my initial post Karl Krukow emailed me with some updates on the issues I was having and I've integrated those into the blog post. He pointed out that there is a Google Group for Calabash on iOS where you can ask questions and share information.
→ posted on October 22, 2012development
Last time I went through how I set up jenkins on iOS and built a simple static library. All in all, that process went ok. The next step was to build an actual app. Unfortunately I encountered a lot more issues with this.
The first error I got was
Code Sign error: The identity 'iPhone Developer' doesn't match any valid certificate/private key pair in the default keychain`
This post on Stack Overflow has a really detailed set of instructions on how best to fix this. Unfortunately as my iPhone developer account is not a business one, I can't add a jenkins user to it. So to solve my problem I added my certificate to the system keychain so that now my local jenkins user can access it.
Once that was out of the way, the next issue to rear its head was a provisioning profile error.
Code Sign error: A valid provisioning profile matching the application's Identifier '...' could not be found
I found the solution to this again on Stack Overflow.
I needed to copy my provisioning profile from ~/Library/MobileDevice/Provisioning to the jenkins user's folder at /Users/Shared/Jenkins/Home/Library/MobileDevice/Provisioning Profiles.
The next reason for the build failing was that I was invoking xcodebuild using a target instead of a scheme. The target worked fine for the static library but was failing for the app as it was unable to find all the include files. Once I switched to building a scheme, the build worked fine. Building via a scheme or target is easy to configure using the Jenkins XCode plugin. You set the XCode Schema File entry to the name of the XCode build scheme.
Nearly there! Now that the build was working I turned my attention to getting the unit tests running. The TEST_AFTER_BUILD flag that had worked before wasn't working this time. Fortunately I found the solution on Peter Jihoon Kim's blog. I needed to create a new scheme to run my tests. Once that was done I got this error
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/Tools/Tools/RunPlatformUnitTests:95: warning: Skipping tests; the iPhoneSimulator platform does not currently support application-hosted tests (TEST_HOST set).
Peter's blog describes the fix for this. You need to patch the RunPlatformUnitTests file in order to remove the warning printout and invoke the tests.
The last error I had to deal with was with the code coverage. I started seeing errors with fopen$UNIX2003 and fwrite$UNIX2003. I had seen this issue on other blogs before but hadn't encountered it myself until I tried building an app using the command line. The solution as described here is to add implementations of these into your application which simply call the standard C functions.
#include <stdio.h>
FILE *fopen$UNIX2003( const char *filename, const char *mode )
{
return fopen(filename, mode);
}
size_t fwrite$UNIX2003( const void *a, size_t b, size_t c, FILE *d )
{
return fwrite(a, b, c, d);
}
One unresolved issue was that I couldn't get clang scan plugin working for this setup. I think the issue here is that it was using targets instead of schemes. I left this one go as I didn't feel it important enough to get working. XCode runs this while I'm developing anyway. I may come back to it in future once everything else is stable.
XCode 4.5 Updates
So, just when I got everything working for XCode 4.4, along comes XCode 4.5. Ever since the 4.2 to 4.3 transition broke code coverage for me, I've been a bit wary of XCode updates. Sure enough after installing 4.5 and trying to build my app, I got an error message saying that the calabash framework was missing a armv7s slice ld: file is universal (2 slices) but does not contain a(n) armv7s slice: ...calabash.framework/calabash for architecture armv7s Calabash is part of an iOS testing framework based on Cucumber. I've just started to implement this form of testing and it looks really nice. Getting back to the build issue, Ash Furrow has a detailed post on this issue. We need to remove armv7s (which refers to the new A6 chip in the iPhone 5) from our valid architectures. Once Calabash gets updated to support this architecture, I can reenable it.
Once everything built in XCode, it was time to try the jenkins build. Naturally this broke also. The problem was that the XCode plugin couldn't find the 5.1 simulator.
xcodebuild: error: SDK "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator5.1.sdk" cannot be located.
XCode 4.5 had wiped out the simulators for 5.1 so I needed to redownload them. I was able to download these from Preferences->Downloads->Components. XCode also prompted me to download the 6.0 simulator so I got that out of the way while installing 5.1.
Once the simulators were installed I started hitting the same RunPlatformUnitTests error as above. When I opened the script file to reinsert Peter's edits I saw that it had changed slightly. Now the error printout was happening in a function on line 80 which overrode RunTestsForApplication(). I deleted that function and I added back in the original patch at line 118 so that now it looks like
if [ "${TEST_HOST}" != "" ]; then
# All applications are tested the same way, by injecting a bundle.
# The bundle needs to configure and run the tests itself somehow.
export CFFIXED_USER_HOME="${BUILT_PRODUCTS_DIR}/UserHome/"
mkdir -p "${CFFIXED_USER_HOME}"
mkdir -p "${CFFIXED_USER_HOME}/Library/Caches"
mkdir "${CFFIXED_USER_HOME}/Library/Preferences"
mkdir "${CFFIXED_USER_HOME}/Documents"
export OTHER_TEST_FLAGS="${OTHER_TEST_FLAGS} -RegisterForSystemEvents"
RunTestsForApplication "${TEST_HOST}" "${TEST_BUNDLE_PATH}"
else
# If no TEST_HOST is specified, assume we're running the test bundle.
RunTestsForBundle "${TEST_BUNDLE_PATH}"
fi
These changes fixed the 4.5 build issues and now I have a working build and test setup again. Finally!!
→ posted on September 20, 2012development