Wednesday, September 12, 2012

How is Qubes OS different from...

Many people ask how does Qubes OS differ from other approaches to desktop security. Today I'm trying to answer the most popular questions.
  • Why bother with Qubes OS, if any Linux/BSD already allows to setup different user accounts, or some form of light-weight containers or sandboxes, such as chroot, LXC, SELinux?
First, if you use Xorg or similar X-based server as your GUI server, and this is what nearly all Linux, and most of the other non-Windows OSes use, then you don't have any form of GUI-level isolation, which is essential for a desktop system. I wrote more about this surprising problem some time ago. Proper GUI-level isolation was one of the main goals for Qubes.

Second, all mainstream desktop OSes, such as Windows, Linux, BSD, even OSX, are all based on a monolithic kernels, which present a significant security problem. This is because a typical monolithic kernel of a contemporary desktop OS contains tens of millions of lines of code, and to make it worse, most of this code is reachable from (untrusted) applications via all sorts of APIs, making the attack surface on the kernel huge. And it requires just one successful kernel exploit to own the whole system, bypassing any security mechanisms that might have been built on top of it, such as SELinux, LXC, etc.

Additionally, all the various drivers, networking and USB stacks, are also hosted in the kernel, making attacks via buggy networking (e.g. via buggy 802.11 stacks or buggy firmware) or USB stacks a practical possibility. And there is essentially nothing one can do about it, when using an OS based on a monolithic kernel.

In Qubes, on the other hand, we use Xen hypervisor to provide security isolation between domains, and Xen is just a few hundred of thousands lines of code. It also doesn't need to provide all sorts of APIs to applications, because the Xen hypervisor is essentially only interested in CPU scheduling, memory management and power management, and very few things beyond that. Most notably, the Xen hypervisor knows nothing about networking, disk storage, filesystems, USB stacks, etc, as all those tasks are delegated to (often untrusted) service VMs.
  • How is Qubes better than just running a bunch of VMs in VMWare or Virtual Box?

First, products such as VMWare Workstation or Fusion, or Virtual Box, are all examples of type II hypervisors (sometimes called “hosted VMMs”), which means that they run inside a normal OS, such as Windows, as ordinary processes and/or kernel modules. This means that they use the OS-provided services for all sorts of things, from networking, USB stacks, to graphics output and keyboard and mouse input, which in turn implies they can be only as secure as the hosting OS is. If the hosting OS got compromised, perhaps via a bug in its DHCP client, or USB driver, then it is a game over, also for all your VMs.

Second, those popular consumer type II VMM systems have not been designed with security as a primary goal. Instead, their main focus has been on easy of use, performance, and providing seamless integration of the guest OS(es) with the host OS. Especially the latter, which involves lack of good method to identify which domain a given application belongs to (so, lack of trusted Window Manager), support for shared clipboards which every other VM can steal, insecure file sharing methods, and others, all make it not a very desirable solution when strong domain isolation is important. (This is not to imply that Qubes doesn't support clipboard or file sharing between domains, it does – it's just that we do it in a secure way, at least so we believe). On the other hand, there are many usability improvements in Qubes that are specific to multi-domain system, and which you won't find in the above mentioned products, such as trusted Window Manager that, while maintaining great seamless integration of all the applications onto a common desktop, still allows the user to always know which domain owns which window, support for advanced networking setups, per-domain policies, the just mentioned secure mechanisms for clipboard and filesystem sharing, and many other. Qubes also focuses on making the VMs light-weight so that it was possible to run really a lot of them at the same time, and also on mechanism to allow for secure filesystem sharing between domains (templates).

Finally, the commercial hosted VMMs are really bloated pieces of code. They support everything and the kitchen sink (e.g. Open GL exposed to VMs, and various additional interfaces to allow e.g. drag and drop of files to/from the VM), and so, the attack surface on such a VMM system is orders of magnitude bigger than in case of Qubes OS.
  • How does Qubes compare to [your favourite academic microkernel]? 
While the Xen hypervisor can indeed be considered a microkernel if you're not a strict terminology freak, Qubes itself is much more than just the hypervisor. Qubes is everything that is needed to build a reasonably secure desktop OS on top of a baremetal hypervisor (or microkernel). Theoretically, with just a few cosmetic changes (at least architecture-wise), Qubes could perhaps swap the Xen hypervisor for some other hypervisor or microkernel, such as perhaps Hyper-V, KVM, or some more exotic one. Thus, it makes little sense to compare Qubes with a hypervisor or microkernel project. What makes sense is to compare the Xen hypervisor, as used in Qubes, with some other hypervisor or microkernel.

Ok, so how does Xen compare with other hypervisors or microkernels out there? We think Xen is unique because it combines an elegant architecture (type I, baremetal, hypervisor) with a number of practical features, such as power management, support for Intel VT-d and driver domains, support for both para-virtualizaed, and fully-virtualized VMs, and many more, not found in e.g. academic microkernels/hypervisor projects, that otherwise seem attractive from the architecture point of view.
  • How is Qubes better than Google Chrome OS?
 First, Chrome OS is not a general purpose OS. Second, it's based on Linux with all its security limitation that are a result of using a monolithic kernel described above (e.g. all the networking and USB stacks in the kernel without a possibility to deprivilige them). Not being a traditional general purpose OS, Chrome is able to avoid many of the challenges of desktop computing, such as the need to define security domains, inter-domain file exchange (as there is essentially no filesystem visible to the user), and others, which is good, of course. But then again, Chrome OS is essentially just an environment to run the Chrome Browser, so the comparison to Qubes is a bit of a misunderstanding.

Technical aspects aside, there is always the privacy concern associated with running everything in a browser – why would all my private data be managed and accessible to some 3rd party organizations and their administrators?
  • How is Qubes better than [your favorite commercial military-grade certified secure OS]?
You must have heard about the super secure military-grade, formally verified, 100% certified, and generally “unbreakable” operating systems made by companies such as Green Hills, Lynx Works, and others. How do they compare to Qubes OS?

Really, I have no idea. For a mere mortal like myself (and perhaps not a US citizen), it seems impossible to get any more technical documentation of those systems (anything beyond the marketing pseudo-technical gibberish), not to mention a trial copy to play with...

Thus, from my point of view, those systems are just a vaporware. If you, my dear reader, are privileged enough to have access to such system, then good for you, but don't expect me to treat seriously a security product that is not available for a wider audience to touch and play with... (And the Chineese surely have the copies already to play with ;)
  • How is Qubes different than Bromium's “micro virtualization” solution?
Many people asked recently about the Bromium's upcoming product and how it differs from Qubes OS. Unfortunately there are few public information available on this product – essentially there is one not-very-technical whitepaper and there are Ian Pratt's presentation slides from the recent XenSummit about u-Xen, apparently a hypervisor that is to be ultimately used in their upcoming product.

The whitepaper suggests that Bromium is based on a hosted (type II) hypervisor running within a normal Window OS, and that this hypervisor is used to spawn a new “micro VM” for each new “task”, where apparently the task might be something as granular as opening a new tab in a Web browser, which makes it somehow similar to Google Chrome's approach. Clearly, the Bromium's main goal seem to be to automate the process of creating separation domains, which is in contrast with what we do on Qubes OS, where the user is required to define the domains explicitly.

The Pratt's slides provide also some technical insight into how Bromium intends to secure their hypervisor. As just discussed above, a hosted hypervisor must normally trust the hosting OS, in this case Windows, which, for obvious reasons, is not a good idea from the security standpoint. Pratt, however, clearly states that “host (...) can not interfere with the privacy or integrity of the hypervisor or other guests” (slide #8). This is a strong statement, so let's take a closer look at their approach to this problem.

The Bromium's idea of how to make their hypervisor (and the VMs) protected from a potentially malicious host OS is not really breakthrough: the slides suggest to “deprivilege the host into a VT-container” (I think the verb to bluepill is now an accepted term for such action ;), and to remove the host's access to the hypervisor pages (via EPT), as well as protect DMA access from devices via VT-d, plus to make this all sensible, use DRTM scheme such as Intel TXT, to load such a hypervisor from within a potentially untrusted OS.

So, what's wrong with the idea of a load-on-the-fly-secure-VMM-system? Isn't Ian Pratt correct that one could protect its memory and execution from the interference of the host? Actually that is possible – Intel TXT, VT-x, VT-d, and EPT give us means to achieve that (although there are a number of catches here). But he's missing one important point: it's the untrusted OS that still owns and manages the input devices (e.g. via USB stacks and drivers) and, most importantly, the output (via the GUI subsystem and drivers). Ensuring that the host OS cannot interfere (e.g. sniff the screen of trusted applications) might be very difficult, or even impossible, in practice.

If I ever was to break the security of such a system, I would just follow the simple way:
1) Infect the host e.g. via one of the many USB attacks (remember they cannot have sandboxed USB driver domain, as they have only a type II hosted hypervisor),
2) Hook somewhere into the GUI subsystem and keep recoding all the interesting data from the screen...
... or something like that ;)

There are also many other things that needs to be answered and which the publicly available documents are silent about, such as e.g. how does the system handle installation of new applications? How is clipboard and file exchange between (micro)VMs handled? How large are the interfaces exposed to each (micro)VM? For now, without a solid documentation available, and without any code to play with, it is just another vaporware for me. (Interestingly there seem to be Bromium's Beta program, which however doesn't seem to be working, at least not for me -- I tried to signup twice, but never got any confirmation or response...?)
  • How is Qubes different from Xen Client?
In many aspects, Xen Client might be the most similar product to Qubes OS. Like Qubes, it is based on the Xen hypervisor and so it is also a standalone OS, that one must install instead of one's favorite system, and also, like Qubes, it is targeted for desktop systems, and also offers a possibility to run a few VMs at a time.

However, XenClient has been designed with a different goal in mind, namely as a “Virtual Desktops To Go” solution, while Qubes has been designed to provide seamless experience for secure multi-domain desktop system. As a result, lots of focus in Qubes has been put on creating trusted GUI subsystem, support for advanced networking configurations, secure inter-VM clipboard and file sharing, secure method to reuse the same filesystem as a basis for the AppVMs, and also to optimize the AppVMs so they start almost instantly and take little memory, so that one could easily run many of them at the same time. All those things seem to be missing from Xen Client (as well as solid technical documentation about its design).

***
I surely have missed a few other products or approaches -- feel free to point them out in the comments, and I might write a continuation post one day.

20 comments:

sgr said...

Hi there,
I have one stupid question after all.
For whom is/are Qubes dedicated?
You do not expect average user to go and play with it do you?
Whats more, seems you do not care to much of linux users as they probably every day play with many different distros yet they have an option to give them a try with no installation.
Seriously,with no Live USB/DVD option this interesting project has zero,null,nil chance to survive.
Are you really willing to waste those three years of hard work because of this?

Best Regards.

Joanna Rutkowska said...

@sgr: installation on a USB disk is fully supported for Qubes (as indicated on the installation page). Interesting you decided to spend more time writing this comment, than reading the very installation page ;)

sgr said...

Well,Usb disk you say?actually i did notice this one worry not :)but I have a nice 16G key in my pocket may I? I think you expect to much for just to be able to give it a try.
Yet look I'm nobody just flush it to the spam if you like.


Cheers.

Anonymous said...

And Trusted Solaris ?

Joanna Rutkowska said...

Regarding Trusted Solaris Security Extensions:

As note e.g. here:
http://docs.oracle.com/cd/E19082-01/819-2252/trusted-extensions-5/index.html

"Solaris Trusted Extensions (Trusted Extensions) provides labels for local objects and processes, for the desktop and windowing system, for zones and file systems, and for network communications. These labels are used to implement a Multilevel Security (MLS) policy that restricts the flow of information based on label relationships. In contrast to Discretionary Access Control (DAC) based on ownership, the MLS policy enforced by Trusted Extensions is an example of Mandatory Access Control (MAC)."

In other words it's a bit like SELinux with the exception that they did _not_ forget about GUI-level isolation.

Nevertheless, it's still a traditional monolithic kernel model, without ability to isolate e.g. networking and USB stacks into untrusted domains, etc. Plus, all the "sandboxed" apps still have all the reach kernel APIs available for them to "play with", which further makes the attack surface very large.

Also, from the user's point of view, a Mandatory Access Policy as used there, might perhaps be well suited for some military environments, where strict ordering of security levels is easy to determine, but are rather useless for "normal" desktop use, where one wants to combine various domains usually not bound together by any > or < trust relationship (e.g. how do my "personal", "banking", and "work-email" domains compare in terms of security-level -- which one is more important? Not necessarily my 'work-email'!)

pepe said...

So how about some non-academic microkernel OS, like Genode?

The fundamental problem of Xen is that its not a microkernel. Not in the terminology freak sense, but in terms of IPC performance. And also in the way that Xen implements all the precious features you name, like power management. You can spend a lot of effort on making Xen and Dom0 less monolithic, or you just start implementing the same stuff in Genode, where people understand what you're doing and why.

Anonymous said...

How does Qubes OS prevent keyboard loggers from being installed? How can security be guaranteed that key strokes cannot be logged? Thanks

Unknown said...

I am convinced! I would love to give it a try, but my shiny new Dell laptop is built with an Ivy Bridge chip, and I'm too lazy (well, and resource constrained at the moment) to go build a custom distribution. Any chance of finding a (unsupported of course) pre-built .ISO based on the development version for us Ivy Bridge users?

Bruce Downs said...

I have also not heard back on the bromium beta program. Maybe they're scrambling to get something out there.

Note, the 'USB attacks' link incorrectly points to Ian Pratt's slide show.

pxeboot said...

> Nevertheless, it's still a traditional monolithic kernel model, without ability to isolate e.g. networking and USB stacks into untrusted domains, etc. Plus, all the "sandboxed" apps still have all the reach kernel APIs available for them to "play with", which further makes the attack surface very large.

Well, that's not entirely true. Many device drivers entrypoints are simply not visible within a zone (and can actually be configured what is visible and what not) and through all the kernel there is a conscious effort in making paths 'zone-aware' (which leads in reducing the attack surface, again).

Said that, it's MLS - it's designed to do one thing (separation) which is to some extent orthogonal to defense from a weaponized attacker. This is something that is true for Qubes as well. While isolation provides benefits (like any conscious design), it's not strictly defending from an attacker who has planted an iframe into your bank account website. (and still requires a conscious effort in configuring properly the system - a.k.a. the user has to think ahead about what he wants separated and what he doesn't, kinda like poor-man or more paranoid people does with two separate laptops :) ).

Also, getting back to the in-kernel vs hypervisor-kernel separation, it's not automatic that an in-kernel solution has more attack surface of an hypervisor based one. It depends on the implementation.

Joanna Rutkowska said...

@pxeboot: if there is a bug in your, say, wifi stack or usb stack, that could be triggered, say by a malformed 802.11 packet or by a USB device presenting some malformed header or something, then it doesn't matter if some "paths are not visible" within a zone or not -- the game is over anyway, because such bugs might lead to code execution in ring0.

pxeboot said...

> if there is a bug in your, say, wifi stack or usb stack, that could be triggered, say by a malformed 802.11 packet or by a USB device presenting some malformed header

I was focusing more on the second part of your sentence: "Plus, all the "sandboxed" apps still have all the reach kernel APIs available for them to "play with", which further makes the attack surface very large."

Confinement of the OS API is definitely possible (and done for Trusted Solaris/Zones) which leaves us with a smaller attack surface, which, though, as you point out, still includes USB and device/hardware based attacks. (I'd still argue these are a smaller part of the pie compared to browser/app exploitation/attacks, but in this I don't want to minimize at all all the good stuff Qubes does there)

Speaking of USB, though, are really all issues solved there by isolation? Given the current IOMMU implementation there is no real way to fully protect from certain type attacks, short of declaring everything that comes from USB "untrusted". While this may work in some very specific laptop configuration, I guess a lot of users still want to trust reading from some USB devices (the data that comes out and goes there) and the vast majority of desktop users probably live off their USB keyboard.

At the end of the day, imho, isolation only gets us up to some point (and, again, don't get me wrong, it has huge benefits, including fault-tolerance -- panic the usb domain rather than the entire system), but it's really on the ability of the entity (OS, Hypervisor, etc) of self-defend itself that you have to rely, more that the isolation principle in itself.

That's what I was trying to point out: it's hard to offload all your security design to Isolation -- some effort (equal or maybe even more) needs to go also on the self-defense side (which is the lesson taken from 10+ years of MLS vs, e.g., anti-exploitation/mitigation techniques).

Anonymous said...

First of all, let me thank you for all the good work, and I have been successfully using the Qubes for last few weeks (after 1.0 release). Some questions that I have unresolved:

1.I want easy access to USB pen drives; the current approach of qvm-block is not user friendly. Instead, I would like to mount it on one dedicated 'USB-VM', which only does one task: Mount it and then export it automatically using Samba. The destination VM can them use Samba to mount it easily using networking tab. (I still haven't tried out Samba, but I hope there is no security issue in that...)

2. I am not able to use USB-drive based broadband connection. I think it is a serial device (/dev/sr0). I wanted configuration details to be documented for this configuration.

3. How can I use independent VMs from different partitions, on top of Qube's xen, and totally isolated and not even sharing GUI. I could always use remote X windows, or terminal client. I badly miss my Ubuntu and Windows installations and am forced to dual-boot when required.

Thanks!

Joanna Rutkowska said...

Please ask technical questions about using Qubes on the qubes-devel mailing list:

https://groups.google.com/group/qubes-devel

michael said...

@Joanna Rutkowska How would you compare the approach taken with Qubes to the approach taken by the Genode OS Framework?

Joanna Rutkowska said...

@michael: As I understand, the Genode project focuses on building a whole new microkernel-based OS from scratch. They create their own device drivers, stacks, APIs for apps and even apps.

They apparently support a number of different microkernels, such as NOVA, L4*, etc, which can be used as alternatives. This is, of course, a a very nice approach in theory, but the price to pay is just too high IMHO -- e.g. the need to write drivers by hand for all supported platforms is alone something that makes this project impractical for anything except custom embedded platforms.

I think this the reason why there is no ISO with Genode OS that one could *install* and actually use as a primary OS, is there?

michael said...

@Joanna Rutkowska there is no installable ISO available now as the project is at a very early stage. It was less than a year ago that they made public intentions to create a general purpose OS using the framework(Genode in its current incarnation I believe is not targeting consumer class users but rather system builders and integrators). A live CD is to be made available around November or December. It is not entirely true that they create their own device drivers and stacks. They are currently using the Linux USB stack and network drivers from the ipxe project. These stacks and drivers can be used with little to no modifications. The drivers they write by hand are mostly for ARM platforms.

With regards to API for apps, Qt is available. Qt is the recommended way of developing GUI apps on Genode for now and the foreseeable future.

Security is one of the fundamental motivations for Genode. They aim to reduce the the trusted computing base as much as is reasonably possible therefore reducing the attack surface.

In theory, building Qubes on Genode could yield increased performance benefits and reduce complexity. You would run a single instance of the OS with isolated subsystems rather than running multiple VMs.

What do you think?

v0xCAB said...

Joanna,
I must admit, I'm stunned by your common sense approach to security and the breadth of your understanding of the subject. Your command of English isn't bad either considering that's not your native language. Sorry for the flattery but if I could get my programmers to have the least inkling of a clue then I wouldn't have to constantly clean up their messes (I'm the Systems Architect for an e-Commerce Company).

So, I want to try building secure application servers on top of Qubes. Is this a use case that Qubes can cover? I realize that Qubes is focused on reasonably secure desktop systems, but many of the same problems exist on the command line as well and as a target of hackers everywhere, I need to limit the damage should a system be compromised -- the very definitive case of isolation your system was built to provide.

Joanna Rutkowska said...

v0xCAB: A bit of flattery never hurts ;) Anyway, it's hard to answer your question without knowing something more about your use model (and threat model). While we envision Qubes OS mainly for desktop systems, I think that we also came up with a few mechanisms in Qubes OS that might benefit server users too.

I would suggest that you post your question to the qubes-devel mailing list, where we could discuss it more thoroughly. Or, if you have some serious _commercial_ plans for this, you can also write to me personally.

atx said...

Foundations for new Kaspersky OS mentioned recently in this interview:

https://threatpost.com/en_us/blogs/eugene-kaspersky-unveils-plans-new-secure-scada-os-101612

Few quotes:

Threatpost: What are the most important features for the new OS?

Eugene Kaspersky: Alas, I cannot disclose many details about it. The main thing is the OS is based on a software architecture model that allows developing an application that by default is not able to run any undeclared functionality and guarantees delivery of trustworthy data between different nodes.

Threatpost: How is it possible for Kaspersky Lab to develop this kind of secure OS while no one else could manage it?

Eugene Kaspersky: It's true no one else ever tried to make a secure operating system. This may sound weird because of the many efforts Microsoft, Apple and the open source community have made to make their platforms as secure as possible. With all respect, we should admit they were developing a universal solution for a wide range of application and various kinds of users. And security and usability is always a matter of compromise! With a universal OS a developer inevitably sacrifices security for usability. We aim to develop a highly tailored OS specifically for ICS without any compromise in usability. As a matter of fact, we are somewhat lucky here as usability was never a point in the industrial control systems. What is really valued in this market is a guarantee and our business model will include such guarantees.


Joanna, kindly ask for your comment on this.