Something similar happened to me, but with a different scenario. I hibernated my Arch Linux system and then booted into Windows (I unmount all of my NTFS partitions beforehand), did some work, and then rebooted into Linux again. The GNOME lock screen doesn't play nice with my NVIDIA drivers (closed-source), so it doesn't reinitialize the lock screen texture when it wakes up from suspend-to-RAM or suspend-to-disk. The lock screen now sported a portion of the desktop from the Windows system which was shut down before. It was a really interesting thing to see, but a bit unsettling too. Lots of surface for information leakage through VRAM, it seems.
An old Dell laptop, which originally came with Windows, repurposed as a Linux laptop. It had been used exclusively as a Linux laptop (the Windows partitions had been overwritten by the Linux install) for several months if not years, when one day, its lock screen displayed a Windows desktop (I don't recall if the laptop had just returned from suspend or not). Moving the mouse dispelled the "ghost" screen and showed the normal lock screen.
The only explanation I could come up with for that was that, somehow, that particular screen had survived intact in a corner of the video RAM, for months, until a bug in the Linux video driver made it appear.
Makes one wonder how long can information survive in a laptop's video RAM. The laptop had never completely lost power (it has a battery, after all), but it had been powered off daily; it wasn't left on suspend all the time.
Just to clarify ... the laptop you describe was powered off daily? Even Once? Powered off?
Then what you saw was NOT a VRAM artifact. Aside from OP clearly stating it is a re-boot issue, powering off the laptop would have removed power to the video RAM chips -- thus they would have lost their stored data.
What is likely you saw was a screen saver program displaying a random image which it found on the laptop hard drive. How this came to be found (assuming it is what you saw, just to give you the benefit of the doubt) would be very hard to guess without the laptop and it's configuration.
That was a long time ago, so my memory is a bit hazy on the details. If I recall it correctly, it was the laptop I was using at the time, and I almost always powered it off when leaving work. I still have another laptop of the same set at work, so I could look up the exact model tomorrow if you want (it's an old Dell Latitude IIRC).
But I can clearly recall that I thought at the time precisely that: "how is that even possible, powering off the laptop should have cleared the video memory, and this laptop didn't have a Windows install for years!" Yet, there it was, a Windows desktop with one window (or dialog, can't recall precisely) open. (I don't recall how I knew it was from its old Windows install; it might have been showing some work-related information.)
It wasn't the screen saver; IIRC, I used the "blank screen" screensaver (or some other equally simple screensaver), not a "slideshow" or "random" screensaver.
Yeah, it's hard to believe. I myself would have found it hard to believe if I hadn't seen it. I even took a photo, but it was several mobile phones ago, so I'd have to search through my old backups to see if I can find it.
The only explanation I could come up with was that the video memory on that laptop for some reason retained its contents as long as the laptop battery wasn't removed (while the laptop had been powered off many times, its battery had not been removed), and the Windows driver used the video memory in a different way than the Linux driver, so that desktop snapshot never got overwritten. Then one day a glitch made the "read the screen contents from here" pointer point to it, and so it showed in the screen. It's ridiculously unlikely, but I couldn't come up with a better explanation; it's even more bizarre in that the "ghost" image was perfect, with no visual glitches.
These stories, and the hdd full fledge arm controllers[1] makes me thing we need to correctly map and manage every bit of memory and processing unit in our machines.
I've seen this before as well, when going from Windows to Slackware. It's only happened to me with certain video cards, but notably with Intel's 945 chipset (the article mentions that, with the VirtualBox bug at least, Intel is not affected).
On a laptop with an Intel HD4000 and a dedicated nVidia card where the OS can switch between cards, one has to force the usage of the dedicated card to read from VRAM. The internal graphics card seems to be unaffected at the moment.
Makes sense; integrated GPUs (UMA) share the same RAM as the rest of the system, which will get cleared as the BIOS does its RAM test at POST (this might not always be the case if 'fastboot' or similar features are enabled.)
Another observation from this would be that GPUs do not do any memory tests on their VRAM, which also agrees with the fact that most of the time failed VRAM just causes artifacts to show up and no message/etc. upon boot. Models designed for GPGPU use may behave differently.
I recently wrote a library that rasterizes a PDF page to an image and was observing a very similar behavior. It was also highly system dependent and erratic as well. I would randomly see a distorted image of my own screen when I run my program on any PDF. Then figured out I was not initializing the buffer I was using to render the PDF page to image. After I memset'ed it to zero, the problem went away. I still couldn't quite comprehend what was exactly happening in the background.
Yes, this is the same way I found this - In the course of another project I had the same "Huh, this is weird" moment. Then I started experimenting with different setups (Linux, Windows, VMs) and was surprised that I apparently almost always succeeded in recovering stuff.
First, tails scrubs memory (or something like that) when you shut down - should they be scrubbing vram as well ?
Second, wouldn't it be quick and simple to "scrub" vram by filling it up with intensive usage after working on sensitive information ? What happens if I watch 2 minutes of 1080p video - wouldn't we expect all previous framebuffer data to be flushed at that point ?
Finally, I would be very curious to see how this changes as you work with multiple graphics cards in a multi-monitor setup. I tend to have all of my VMs in windows confined to a particular monitor that is driven by its own graphics card - presumably those guest operating systems are only accessing that framebuffer ?
Yes, but VRAM is a tricky beast. Modern GPUs actually have MMUs and the VRAM address layout the CPU sees may not cover the whole of the memory of the graphics card.
> What happens if I watch 2 minutes of 1080p video - wouldn't we expect all previous framebuffer data to be flushed at that point ?
No, because the video frames will be queued in a circular buffer and anything outside of that doesn't get touched.
The only way I can see to reliably scrub VRAM would be to use the GPU as a "dumb framebuffer" (without video acceleration), like in the good old days. Then to scrub what was used in the session it should be enough to draw a black screen, do a page flip, and draw a black screen again.
Yes, that is the exact same issue. While searching for any prior art, i did not find your blog post, but the problem seemed far too obvious for me to be the only one to have ever stumbled upon it.
I do think this issue needs some exposure, there seem to be a lot of potential security ramifications.
Just in components that claim to provide security boundaries but then do OpenGL passthrough without precautions. Web browsers handle this this properly in WebGL. VirtualBox is at fault here.
Your linked article is about just running two native desktop apps side by side under the same user account, there isn't any supposed security boundary between them so it's a different case.
I've noticed something similar playing Mini Metro last month.
I was running a Windows 7 VM on an OS X 10.9.5 host with Virtualbox 4.2.28 using my mid 2013 Macbook Air (Intel HD Graphics 5000 1024 MB). Instead of blurring the game for the weekly upgrade menu, it showed the center of my host desktop wallpaper, upside-down and flipped (other than that, there was distortion whatsoever). It really creeped me out.
The webgl standard has rather strong memory safety considerations:
WebGL resources such as textures and vertex buffer objects (VBOs) must always contain initialized data, even if they were created without initial user data values.[0]
As long as browser vendors implement this (and chrome and firefox seem to), this should not be an issue.
This was a problem with WebGL a few years ago on some OS/GPU/Browser combinations. It was possible to write a WebGL app that created uninitialized textures, drew them on screen, grabbed a screenshot and then uploaded that via HTTP(s) to a server.
This issue should now be fixed, but bugs and/or old unpatched browsers may still be out there.
Oh sure, but in this particular case it seemed like no-one had thought about the issue that uninitialized textures can be used for snooping screen content, this wasn't mandated by the spec (IIRC).
Howeverm we did not test on any other browser technologies like flash or java. I cannot guarantee that there is no possibility to access any VRAM from within a browser.
That's very interesting. I guess VRAM doesn't get powered off during reboot and (today) it is huge, so the chance of interesting information remaining is big.
Mitigation strategy: Ctrl-Alt-F1 and work in text mode. (or just fill up video ram doing stuff in your machine after doing the sensitive work)
I don't specifically know the answer to your first question if the VRAM is not DRAM (shared memory), but certainly rebooting will not simply automatically clear out DRAM.
This is actually one thing that bugs me about OpenGL, DirectX? and Vulkan.
Basically all the insentive so far is best speed. That means there's almost zero insentive for security in GPU drivers. You can try to implement security on top of the GPU. WebGL does this. Chrome also does this for most (all?) of its GPU access meaning even its page rendering is going through Chrome's secured GPU system that clears buffers and makes sure one process can't read another's GPU resources.
But, it seems to me this should really be at an OS level for both CPU and GPU memory. Otherwise it requires all programs to be vigilent. Clear every buffer before deletion and hope there aren't handles behind the scenes leaving older copies around.
I think it's not really that simple. Clearing buffers certainly isn't free if you're doing it all the time, and considering that high-end gaming already pushes GPU's to their max output, the difference between clearing buffers and not clearing them could be noticeable in the end. Of course, it depends completely on how you implement this and when you go about clearing. For example, if a game allocates a pool of GPU memory at the start and then grabs memory for this pool as it needs it, then the OS clearing the buffers hardly matters. If it's constantly asking and releasing memory though, then the buffer clearing is important. I honestly don't know how the bigger titles handles this, mostly because I don't know exactly how different GPU memory is from CPU memory (And if the pool approach is possible).
What could be an interesting idea is having a 'GPU security switch' on the OS level. If it is set on, then GPU memory buffers are cleared at every release. If it's off, then they're not cleared. My thinking would be that you setup a system in the kernel, somewhat like root access, where by default this switch is on, but a program like a game could suggest that it be turned off for that game to get better performance (And a program like sudo could pop-up with a request box and ask if you want to turn it off, and then ask for the root password to do so). What would be even better is if it could be accomplished on a per-program basis, so only that game's memory isn't zero'd (Which probably isn't that important of memory anyway) and the rest of your program's using the GPU are still zero'd.
In these tests we were able to recover screenshot-like views of entire websites after viewing them in the newest version of chromium, then running the poc on the local system.
Chrome isn't trying to not let other (non-chrome) processes read info. It's trying not to let other chrome processes (other webpages, native client, etc) not read info. In other words, stuff running inside Chrome can't read stuff it's not supposed to be able to since nearly all that code is untrusted.
Chrome saves its VRAM when using WebGL, but it does not protect the hardware accelerated website rendering. thats why one could read out VRAM with chrome content as well.
I wonder why the drivers don't do this on startup. Judging by the specs, most GPUs have enough memory bandwidth to clear all of their memory in less than one second.
