App Architecture: Fetching JSON

11 Jun

I’m working on an app that needs to fetch some JSON from a server. The JSON doesn’t change frequently, but it may change a couple of times throughout the day at unset intervals. We also need to ensure we fetch new data if the user’s location has changed significantly, say by at least a kilometer.

The simplest (lazy) approach is to just fetch the latest JSON whenever the app needs to display the data. This is also the worst approach. It means we’re hitting the network constantly. What happens if some object requests the data multiple times within a few seconds? It doesn’t make sense to perform a whole new network request each time. It also means we have a significant delay in returning results, since we have to wait for the JSON to download each time. Even a very small JSON file can take upwards of a minute to download on a slow cell network (Yes, sometimes you still end up on 3G or even ::gasp:: Edge). So we can see that while this way would ensure we get the newest data each time, including when the user’s location changes, it does so at a significant performance penalty.

We need a better method. One obvious way we can improve on this is by only requesting data if the user’s location has changed significantly. Is this something the Fetcher class should care about or should it be left up to the requesting object? Let’s pick a path and see where it leads us…

Smart JSONFetcher
In this scenario we’ve chosen that JSONFetcher should handle deciding if the location change is significant enough to warrant a new fetch. This means we’ve made the JSONFetcher class aware about more than its basic task. In a sense we are piling extra work onto JSONFetcher. How does the fetcher know what we consider a significant change in location? Do we pass it each time the fetch is called? Is it a variable we set on an instance fetcher? Is it hardcoded into the fetcher class? All of these have trade offs in usability and flexibility. It also means that our JSONFetcher class is now specialized into fetching JSON that has a location associated with it. Is that desired?

Dumb JSONFetcher
How about if we decide the only thing the fetcher needs to know is what it wants to fetch? In this case we leave handling location changes up to the requesting object. We still have to do the work of deciding when to fetch based on location changes, but now the requester is handling that. The work still exists, but we’ve shifted it onto the user of the class rather than handling it for them.

At first glance you might think “Clearly, I want the Smart JSONFetcher”. But in my experience when you say you’ve made a class smart what you’re really saying is you’ve made the class complex. We want to avoid complexity. A side effect of removing the location information in the Dumb JSONFetcher is that it’s now capable of handling any requested JSON, not just ones that have a location associated with them. This is great, now we have a class that we can easily reuse anytime we need to fetch some JSON. We can build on top of this “Dumb” JSONFetcher to make it more specific to our app’s needs. We’ll take a look at this in a bit.

For now, let’s revisit our first requirement. Not fetching every time we’re asked. This sounds counter intuitive. After all, if some object is requesting the information clearly we want to give it the latest and greatest. But the requester likely doesn’t know (and shouldn’t need to know) that the JSON only changes a few times throughout the day. Luckily if we use NSURLSession (and you really should) we get caching for free. But there’s a catch. There’s always a catch. If the server doesn’t set cache headers you’ll run into issues. As Murphy would have it, the server I’m dealing with doesn’t set cache headers. So the work around is to override willCacheResponse. This allows you to set proper cache times, perhaps something like 5 minutes.

Ok, so now we have a JSONFetcher class that will fetch our JSON, but only if it’s truly stale. Let’s go back to our location problem. We can extend our JSONFetcher class to give it the location validation that our app needs. Let’s add a category JSONFetcher+Location. We’ll add a method fetchJSON:withLocation:. This method will take a location and only execute the fetchJSON: method if its outside of our threshold.

This is getting pretty good. Let’s step back and take a look at where we’re at. Now, the JSONFetcher+Location is ensuing our location is valid and returning the cached data if it’s within our time threshold. What happens if the location isn’t valid? The fetcher goes ahead and executes the download and we wait until we get new data. That seems appropriate, we wouldn’t want to display incorrect data to the user. Ok, what happens if the location is valid but we’ve exceeded the time threshold? The fetcher checks if it’s valid based on the cache policy we set and fetches only if needed.

So that’s the path I think I’m headed down for now. The main point I’m trying to make here isn’t the best way to download some JSON. Rather, it’s about investing the time to think these things through. I may still end up needing to handle this differently, but I’ve put enough forethought into it that I feel fairly confident. Keeping things modular and easy to refactor is key to good development.