Incremental File System by Google
Google has been working on a new kind of File System that allows the execution of binaries before a complete bundle has been loaded. Taking the development of Android apps as an example, you currently have to wait for the whole binary to be transmitted to either the emulator or your real testing device for debugging. Incremental File System dramatically shortens this delay by allowing the execution of the binary before the whole package has been transmitted. This sounds immensely complex, and that’s what it is, but we can still take a look at it to better understand what will most likely be a default feature in Android 12 and later.
A Virtual File System called “incfs”
Google’s virtual file system solution, which in its first iteration will be used exclusively for APKs and AABs loaded from the Play Store, is a layer that sits on top of the real filesystem. To better illustrate the architecture, I’m using some slides Google created for a technical talk. You can find the link in the addendum at the end of this page.
Coming back to the topic, Google’s Incremental File System can be thought of as a cache, where the created entities directly reference files in the real filesystem. The code itself is deeply built into the operation system as a kernel module - which seems logical due to its low-level placement in the overall architecture.
Advantage of a virtual file system
The advantage of such a virtual FS is that you’re not bound to the limits of a real filesystem, where the read and subsequent execution of a program usually can only happen if the complete bundle is available - from first to last Byte. With the “Incremental Filesystem”, this limitation is technically cleared. Yet with such a powerful feature comes a large set of complex aspects to consider.
Taking it from a Google engineer, a big challenge is the correct handling of write-operations. The virtual filesystem has its own state regarding pending reads, but still has to maintain the data integrity of the loaded files.
The challenge is tackled by creating a custom index in the “incfs”. Each real file directly correlates to an entity in the virtual filesystem, even when the original file gets renamed.
One more abbreviation called “ioctl”
This abbreviation stands for “input/output control” and leads to the creation of such an index-file in Google’s Incremental File System. “ioctl” comes from the operation system and its technical explanation is out of scope for this article. If you want to read more, there’s a link in the addendum as well.
Also quite interesting: “incfs” has a built-in logging mechanism. This feature will be used to determine the real-world performance of its implementation for further enhancements.
Practical usage
The start of this feature was in May of 2019, when some of Google’s engineers started to commit changes to the Linux kernel. Placing it there allowed them to simply use it in the Android Open Source Project (AOSP), the open source part of Android and the main foundation of this operating system, as AOSP is built on the Linux Kernel as well.
Linux contributors weren’t all that happy, as similar approaches to enhancing a file system had already been made, such as FUSE or OverlayFS. Yet from Google’s perspective, those implementations weren’t nearly performant enough on low-end Android devices. This led to the development of the Incremental File System.
For the first iteration, it will be used for games downloaded from the Play Store starting with Android 12 to shorten the time it takes to actually launch the game. This really great feature presuppose the Incremental File System, but also some logic built into Android to determine how to acutally start those games early - without any changes by the developer, according to Google.
