Right-sizing RAM is certainly a common requirement, as we know VM tend to be oversized. However, we need to be careful when reducing RAM. Reducing RAM on VM based on information at the hypervisor level can result in poor performance. You need to have in-guest level data.
I did a set of test on a physical desktop to see Windows memory behaviour. I did it on physical so we know for sure there is no hypervisor impacting any reading here.
I have 64 bit Windows 7 on a physical desktop with 4 GB RAM. For a test on 8 GB RAM, see this as the result is not the same, even though the workload is similar.
- I boot up the machine, and let it idle for a few minutes to ensure all start up programs have finished running. I want them to “settle down”…
- Modified (orange), means memory that is no longer in the Working Set of a process, but it has been modified. Windows needs to write to the page file before releasing it to the Standby pool. See this for more details.
- The screenshot shows that Windows 7 64 bit uses 1296 MB of RAM (162 + 1055 + 79). The Standby Memory means it contains cached data that is no longer actively in used. How does Windows define “actively in used” is not clear. If a block of RAM is not used in the past 1 minute, is it active? How about 10 minutes? 1 hour? The next few screenshots will clarify that.
- We have 674 MB of Standby and 2126 of Free as you can see above.
- I then launched a lot of common apps, and open lots of files. I opened 13 large PowerPoint files and 10 Adobe PDF files. You can see in the screenshot, the task bar is full of documents.
- Naturally, CPU and Disk would spike, so I let them settle down first. My focus here is RAM.
- The screenshot is taken after Windows settles down.
- You notice CPU has gone down. My desktop has 4 cores, and 2 cores are actually parked. You can see that the only CPU-consuming process is the Resource & Performance Monitor itself, and the Desktop Window Manager, as it has to redraw the screen.
- Both PowerPoint and Adobe has gone down to 0%, as they are done opening files.
- After verifying that CPU has settled down, I verify the RAM.
- Unlike CPU, we have a very different situation here for RAM. This is key to explaining why you cannot just downsize RAM without consulting application team.
- Windows now use 2553 MB. Much higher than 1296. Obviously the executables of PowerPoint, Adobe, Excel are in the RAM.
- The Free Memory has gone down from 2126 to just 469 MB.
- The Standby Memory increases from 674 to 1074. This is the 23 ppt/pdf files that were opened and after that not touched by. All I did was just opened them. So from here we can tell that “active” is a relatively short time, it’s in seconds or minutes. In my subsequent observation, it is actually in seconds.
- Based on screenshot #3 (not #4), I could conclude whether PowerPoint or Adobe read the entire files. I think it reads more than the first page or the first slide, but probably not all. I knew from other tests that 1 MB of powerpoint file translates to 15 MB in RAM. I still have 469 MB of Free RAM, while I think it should be lower.
- So I forced PowerPoint to read the entire content of almost every files. What I did what go to each file, change to “Slide Sorter” view, wait until PowerPoint redraws all slides, scroll down, wait until it redraws, repeat until the last slide is drawn. This ensures PowerPoint read every slide, and brought it to RAM. I did the same thing some of the Adobe file.
- The Free RAM dropped from 469 MB to just 9 MB. It’s interesting it does not go down to 0. This is showing Windows memory management. The Standby remains high, from 1074 to 953. This is a proof that “active” is a relatively short time.
- I let my desktop idle for 40 minutes. Naturally, Windows went to sleep mode after a while. I wanted to see the effect of sleep mode on RAM.
- When I brought up Windows from Sleep mode, I checked the RAM. Screenshot #5 shows that it’s hardly change. The Free RAM remains low, Standby is about the same, and In Use is about the same.
- It looks like Windows treat executables differently than it does data. I think the In Use Memory is the apps, and not the data. However, I’m not 100% sure here.
- It is confirmed that Sleep does not flush the RAM content.
- I am not able to check page fault as Resource Monitor does not have it. But when I use PerfMon, I can see paging.
- You can see paging at vCenter if you separate the page file into its own vmdk file. This is something you should consider as standard. You can then use vRealize Operations to monitor for abnormal behaviour. For example, the VM owner complains that the VM became slower since last weekend change. You do not see any significant difference in memory, CPU, Disk or network utilisation. Your ESXi host was healthy too, delivering very low contention. However, vRealize Operations shows abnormal behaviour in the vmdk file that hosts the page file (and only the page file). It has much higher read and write. From here you can at least say that there is excessive paging. This topic is covered in depth here.
- I closed all apps. As expected, the memory got back to near the original situation, when I had no apps open.
- I launch PowerPoint again, but this time only open 1 file (8 MB). I do not launch other apps.
- Notice the Standby Memory remains the same. So Windows keeps it, and uses the Free Memory instead. This makes sense.
- I launched Visio installer and OpenOffice installer. Total 560 MB file.
- I also launched Word, Adobe, etc.
- I notice Windows Free Memory did touch 0 MB for 1 second, and then Windows keeps it above it. You notice in the screenshot that the Standby Memory actually goes up by around 500 MB.
- This confirms that Standby is adjusted in seconds, not minutes. This means Active is in seconds too.
- I closed both the installer programs.
- The Active memory (In Use) remains the same.
- The Free Memory goes up by around 500 MB, while Standby drops by 500 MB.
- This means we know the installer was in the Standby Memory.
- I closed 25 MB worth of PowerPoint files.
- The Standby remains the same, so we know the files were not in Standby. They were in “In Use”, as it drops and the Free Memory goes up by 380 MB.
- Memory tends to stay even though it is not used. It is much more stable than CPU. This is because Windows does not know when it will be used again, so it just keeps the pages there, just in case. This makes sense. So the RAM remains idle.
- Because they are idle, the Hypervisor (VMkernel) will think they are not used. Remember the hypervisor has no visibility inside the Guest OS memory list. The VM Memory Usage counter in vCenter will give you a low number. It is based on what the hypervisor estimates as active. If you are sizing based on this, it can impact performance, as it can force paging inside Windows.
- For Windows, you need to use the Free Memory counter, In Use counter, and Standby counter to give a more accurate picture.
For performance, include Standby as that's serving as cache. For cost, exclude Standby.