This article attempts to provide a very short overview of the internals of the Content Downloader in the TalentLMS for Android app. It offers a quick mention of the main third party libraries used for the implementation and goes over the four main Java classes which do most of the work.
Libraries
In order to implement all of the features mentioned in Part 1 of this series in self-explanatory, reusable and extensible code, we relied heavily on some amazing third-party libraries which simplified the code and saved us a lot of time. Although there are more delightful open source projects used by TalentLMS for Android, this article only mentions the ones that were essential for the development of the Content Downloader.
RxJava / RxAndroid
Although it seems hard to wrap your mind around at first, Reactive Extensions are an amazing time-saving tool in the medium-to-long run, on any platform. If you have not used RxJava before, you should definitely invest some time to learn how it works.
Version 1.x is used extensively throughout TalentLMS for Android and the reactive approach helped us simplify the complex Content Downloader code immensely! I really cannot say enough good things about it!
Version 2 of RxJava has recently graduated from beta to stable and it looks amazing! We currently do not have any plans to use it in TalentLMS but I am definitely looking forward to trying it on a couple of my own pet projects.
Realm Mobile Database
Choosing Realm as the persistence layer of TalentLMS was one of the toughest decisions I had to make for the project. Although it is powerful, its approach is quite different from what many developers are used to. In order to really appreciate its many advantages over other alternatives, one really has to read and experiment with it a lot.
If you choose to go for it, beware of the many misleading blog posts about its use! Many of the people writing about it have not really understood how it works!
Most developers (including myself when I first started out with it) seem to run into threading issues, most of which are usually solved using much simpler code than you would expect. In the case of TalentLMS, Realm helped a lot with all offline aspects of the app and really made a huge difference in the Content Downloader implementation! Combined with RxJava (for which it offers first-class support) it helped us write some smart, elegant and powerful code!
Before starting any serious work with Realm, I would urge you to thoroughly read its documentation, as well as the following articles by @Zhuinden:
If you run into any serious issues no one else has written about, Realm’s vibrant community is always there for the rescue, but always keep in mind the following:
- All Realm-managed objects are live views into the underlying data. You don’t have to keep fetching them again and again.
- Focus on the zero-copy aspect of Realm! If you find yourself copying objects from/to realm, you are probably doing it wrong.
- Whenever you need to use an object in a different thread than the one it was created in, all you have to do (9 times out of 10) is fetch the object again from a Realm instance in that thread, using its primary-key value.
OkHttp 3
I have been using OkHttp since its first iteration and have always been very pleased with it. This time, in order to implement certain parts of the Content Downloader functionality, I had to dig deeper into its inner workings. Once more I was very satisfied by both its code quality and the great community around it. If you are also using it for more complex stuff, make sure you take a look at its recipes page first.
Dagger 2
In my opinion, Dependency Injection is a must for most medium to large-scale projects. The second version of Dagger offers many performance and usability improvements over the first one and helps improve code quality and testability a lot. If you choose to integrate it into your project using an MVP approach, make sure you read this article about process death. It talks about what has turned out to be one of the most serious pitfalls of DI on Android.
RetroLambda
While I love self-explanatory code with long names and a reasonable number of classes, verbosity has its limits. Code that does complex things should look as clean and simple as possible and RetroLambda almost always helps your eyes hurt less. It will also increase the lifespan of your trackpad or mouse scroll-wheel 🙂
Classes Overview
RxJava and Realm affected the architecture of the component considerably since they offered a (some might say non-standard) way to solve some of the most complex problems of the implementation in elegant ways. In fact, the four main classes mentioned below were structured the way they were in order to play nice with those two libraries.
The downloader is riddled with nested asynchronous operations that are often error-prone (e.g. network requests) and their errors have to be filtered and propagated to the proper nesting level. Since nested callbacks are never a good idea, the Rx approach greatly simplified the code almost everywhere. Most of the public methods of the main ContentDownloader class return Observables, since we never want to wait for any network or database operations.
Furthermore, since all content downloading and processing progress had to be preserved, the persistent storage layer had to be fast and comprehensive. Multiple database operations happen at the same time by various parts of the content downloader and for various reasons. Although I was initially skeptical whether older devices would be able to handle all that strain without everything slowing down, Realm proved to be a life-saver.
Here are the classes responsible for most of the work going on when content is being downloaded:
Unit Download Status
This class is a simple RealmObject that stores basic information about the status of a Unit’s downloadable content. It is only updated by the ContentDownloader class via Async Transactions whenever new data becomes available or when the status of a Unit’s content changes. Some of the fields it contains are the content’s URL, the local file’s path, the remote file’s size and other cached values that help speed up the entire download progress.
File Downloader
This class is responsible for a single file download. It is initialized with a remote file URL and a local file path and exposes methods for starting and stopping the download process and retrieving progress reports.
When the download is started, the FileDownloader checks the local file path and if the file already exists, it attempts to resume the download from the server. If the total file size has not been made available, it is retrieved from the server using an HTTP HEAD request to the file URL.
The FileDownloader uses an internal RxJava hot observable that emits download progress reports and errors encountered. The observable is created as soon as the download starts and is available throughout the lifetime of the FileDownloader instance.
Network operations are handled using an OkHttpClient and download progress reporting is based on this recipe. One of the biggest challenges faced when developing this class was bridging the progress reporting recipe with the creation and handling of the hot observable.
I finally opted to use the Observable.fromEmitter method which remains in experimental status in RxJava 1.x to this day. The fact that it worked very well and that the same approach is still present in RxJava 2.x (renamed to Flowable.create), makes me think that it will graduate to supported status at some point in the future.
I could not find any good documentation or examples about this method when I was developing the FileDownloader class, so I will attempt to present an example of how I used it in part 4 of this series.
ActiveDownload
An ActiveDownload object is instantiated whenever a Unit download is started, using its UnitDownloadStatus object, and is only kept in memory for as long as that download is active. It is responsible for fetching the Unit’s metadata, determining the URL of the downloadable content and performing any additional required pre and post-processing based on the Unit’s type.
Each ActiveDownload object uses an internal FileDownloader object that handles the actual download process. Much like the FileDownloader class, the ActiveDownload exposes methods for starting, stopping and progress reporting (via its own hot observable) as well as a number of utility methods related to the rest of its tasks.
ActiveDownload objects do not emit all progress updates from their FileDownloaders. Instead, they only emit critical status changes due to errors or when processing or downloading tasks have been completed. If detailed progress updates regarding a file download are required, ActiveDownloads expose a getter for their FileDownload object.
Content Downloader
This is the actual class which orchestrates all content downloading. It is a singleton that is initialized as soon as the application is launched and exposes a number of public methods that the app uses to enqueue and stop downloads, alter the overall downloading behavior and handle already available offline content.
It manages individual downloads via a collection of ActiveDownload objects and subscribes to their observables to receive progress updates. Additionally, it uses an observable to continuously retrieve updates from an asynchronous Realm query for UnitDownloadStatus objects of Units queued, but not yet downloaded.
In essence, after the Content Downloader has been started, it only responds to updates from its ActiveDownload objects and UnitDownloadStatus query observables. Since ActiveDownloads only emit status updates when critical events happen and because there are never more than a handful of concurrent downloads, the ContentDownloader is very lightweight.
This is the end of part 3 of this series. Part 4 is coming soon and will contain code examples for certain parts of the implementation that I could not find much information about online.
| Tags: TalentLMS Android App,Team
Leave a Reply