I just had an epiphany. Well, not really, but an interesting thought, at least.
Computers are getting faster, and more capable. In particular, the amount of RAM available to computers has increased in leaps and bounds. My first computer (and IBM PCjr) had 64K of RAM, and that was part of an expensive upgrade from the default. The laptop I’m typing this on has 2G of RAM, and the workstations sitting next to it have 4G and 6G respectively. That’s a huge increase, over the past 25 years.
However, in my daily use of my laptop, 2G doesn’t seem to be enough. After a few days of uptime, my free RAM (defined on Linux as the amount used by disk cache, rather than the amount actually free, since RAM is basically never actually free) drops below 100M. This is not an acceptible amount of disk cache, and it causes my machine to slow way down, and even to go into swap storms sometimes. If I then go and emerge something written in C++, I trigger the OOM killer. On a 2G box. How does this happen?
One posited (and believeable) reason is the Developer-User machine divide. The reasoning goes thus:
Software development is a resource intensive operation. Compilers, debuggers, emulators, and so on, take lots of CPU and RAM to run fast. Developers are rather high paid individuals. Thus, most companies maximize ROI by getting developers large, fast development machines, so they don’t spend all their time waiting for tools to execute. Similarly, FOSS developers tend to spring for decent development boxes if they possibly can, because software development quickely becomes painful on low end boxes. Naturally, if a developer has a large, fast development box, they’re going to use it whenever they can, for normal, day-to-day tasks.
Typical users, on the other hand, have (by definition) average machines. They tend to be middle-to-low end, with much less CPU, RAM, and disk than a typical developer machine. (Unless it’s a gaming rig, of course, which ends up way in the top end for similar reasons to developers having good boxes). This means that the box a developer typically uses for day-to-day activities (mail, browsing, chatting, office, and so on) tends to be faster and have more RAM than the box a typical end user has. This is the Developer-User machine divide.
I’ve read a number of posited solutions to this. The most common is that developers should use “typical” boxes for day-to-day activities, rather than their development machines. This is not going to happen. First off, if you have a choice of 2 computers, one of which is much faster, you’re going to use the fast one, right? Second, especially for FOSS, developers typically don’t have the funds to have multiple machines like this. Any extra machines I have, for example, are typically either old dev machines that are below even the average curve and run my network, or specialized machines (say my elisa box) which cannot be used day-to-day. I’m going to use my laptop (which, granted, is not a beefy as my workstations at work, but is still, even 1.5 years later, beefier than anything any of my relatives have).
So, my reasoning went like this: We need a little daemon that lurks in the background, sucking up RAM and keeping your CPU at it’s lowest frequency, until it detects software development in progress. Then, it should free up RAM (temoprarily, of course) and jack up the CPU frequency, so that development happens fast. When development is done, it lowers the CPU again, and sucks RAM back up. This, of course, is problematic, difficult, and easily open to abuse (say a tight-loop script of gcc niced).
Then it hit me: a VM is the perfect solution. It’s slower by it’s very nature, and it can have strictly limited RAM. Developers should run their day-to-day programs in a VM! Then, they can allocate, say 512M of RAM to the VM, and leave the rest outside for normal development activities. Granted, this leaves it open for cheating, but at least developers would start to feel the pain of normal users, start to bridge the machine divide.
Will this happen? Probably not, at least not on a large scale. Many developers, of course, already develop in a virtual machine of some variety, for other reasons (need Windows on Linux or vice versa, for example). But in general, the first reason why developers don’t already use slow machines applies, and many developers (myself included) don’t have the time to track down complex time/space trade offs in complex software (I’m looking at you, Evolution!) to make their VM’d selves happy.
Will I do it? Again, probably not. This is because my primary development is Gentoo maintenance. This means that I can’t isolate my development to outside my VM, since I’d need to keep the VM itself up-to-date with ~arch software. I’ll think about solutions, but there may not be a good one (maybe distcc to the host box?). Additionally, the majority of my development work for FOSS is packaging software for Gentoo, which means I don’t usually do much actual development on that software, and what I generally do is bug fixing, not in-depth development. Then again, maybe I will do it, just to force myself to dig into these issues. It certainly could be a useful experiment.
Now, the question is: how do I get my VM to gracefully recover when my video doesn’t resume? 
Question for the Lazy-Web:
Has this been done? Are there tools for syncing, switching, backing up, and so on? Does, for example, VMware do this internally?
Tags: Development by Daniel
No Comments »