windows – Fixing things

December 29, 2020

log rotate and archive with windows

I support a few applications which run on windows and there isn’t really any great way of rotating the logs like there is with linux. While the logs are all configured to go to off host log solutions such as Azure logs or elasticsearch I like to keep the local logs around for a period of time as well. Depending on your rules some fidelity or information may be lost when the logs are ingested into these solutions. I have tried using some of the built-in compression functionality with windows but some of the log files are so large they generate out of memory errors. I have found 7zip to be the best tool for these large files. The compression rate and performance is better.

One such application where I want to compress all logs older than 7 days and then keep for 90 days is bitvise.

You must install 7zip on the computer first, but then you can run the script below on demand or on a schedule. e.g. every night or weekend.

#Get all log files older than 7 days and compress to archive then delete source
cd “C:\Program Files\Bitvise SSH Server\Logs”
foreach ($file in Get-ChildItem -Filter *.log -Path “C:\Program Files\Bitvise SSH Server\Logs” | Where-Object { $_.LastWriteTime -lt (get-date).AddDays(-7)})
{
& “C:\Program Files\7-Zip\7z.exe” -sdel -mx=5 a “$file`.7z” “$file”
}

#Get all old 7zip archives and delete when older than 90 days
Get-ChildItem -Filter *.7z -Path “C:\Program Files\Bitvise SSH Server\Logs” | Where-Object { $_.LastWriteTime -lt (get-date).AddDays(-90)} | Remove-Item -Force -Verbose

June 13, 2016January 20, 2017

Windows 2012 Dedupe – huge chunk store and 0%

One of the best new features in 2012 was the file de-duplication. That said it does sometimes behave a bit strangely under some workloads. I recently faced an issue where a a 40TB volume with de-duplication enabled resulted in a huge chunk store that was using more space than the original data!

At a glance it looks like the best thing to do is turn off dedupe for this volume, but all this seems to do is disable further dedup work, anything that is already deduped will remain so. I found the best/fasted way to “re-hydrate” your data and get rid of the chunkstore (You could just format the volume if you don’t need the data) is to leave the dedupe enabled, but set an exclusion on the root.

Then run the commands below in power-shell (Assuming drive letter F:):

Start-DedupJob -Volume “F:” -Type unoptimization -Memory 50

Then run:

Start-DedupJob -Volume “F:” -Type GarbageCollection -Memory 50

You can then monitor the size of the chunkstore and/or run this command to see the progress of any “dedupjobs” with this command:

Get-dedupejob

Do bare in mind the increased IO and server load while this runs, it maybe best to start this out of hours. Please also note that this command will only actually re-hydrate your files if dedupe is still enabled.

October 3, 2015October 2, 2015

Compacting / shrinking a VHDX

Sometimes you over provision a server and need the space back on your cluster shared volume or use a full fixed size disc and later want a dynamic one and then need to shrink it. Or it could simply be you are making a template and want to compact the disk as much as possible for faster deployments and less disk space consumed on your VMM server.

Some of the steps here are also useful if you just want to find additional space on a windows server. Step 1 outlines how to cleanup the component store and remove installation files for unused windows features. (e.g. why keep the source files for the Remote desktop session host server role on a web server?) You can skip step 4 if you are just looking to shrink the file temporarily, if the VHDX is for a VMM template or if you are just cleaning up a working server for more free space.

Here is the process:

1. Clean up the disk first and remove any files or data you don’t need so that you can compact the disk as much as possible.

Here are a few options:

You can remove the source installation files for any roles and features you don’t need to save on disk space. This can make it difficult or sometimes impossible to enable the features later though so proceed only if you are sure you don’t need other roles to be added later. Make sure to run this in an Admin powershell window.

Get-WindowsFeature | Where-Object {$_.Installed -match “False”} | Uninstall-WindowsFeature –Remove

For 2012 R2 and other later windows versions you can also clean up the component store with DISM. This does mean you won’t be able to remove existing service packs and updates however future updates will be removable.

Dism.exe /online /Cleanup-Image /StartComponentCleanup /ResetBase

Clean up any temporary files, old profiles and all the other places you would usually look to free up more space. 2. Grab a copy of the sysinternal Sdelete tool and use it to zero the deleted files on the disk. This step is not essential but you will get better results if you do it. I would recommend skipping this step if you are using an SSD on a physical machine.

https://technet.microsoft.com/en-us/sysinternals/bb897443.aspx

This command assumes you are on the machine you want to shrink:

C:\pathtofile\sdelete.exe -s -z c:

3. Defragment the disk. You can run this from within the guest OS but you may get better results shutting down the VM and mounting the VHDX file on another system. This way all the files can be defragmented.

defrag C: -w –v

4. Consider shrinking the partition. It maybe if you are making a template you want to keep a large partition as you expect and want the VHDX to grow after it is deployed. (If that is the case skip this step) If you are looking for a more permanent shrink then you should now shrink the partition to the desired size. This is best accomplished in the disk management GUI or with PowerShell.

This assumes you only have one local disk and know the size you want to resize to, if this isn’t the case use get-partition to work out the correct numbers for your situation and use Get-PartitionSupportedSize to view the possible sizes you could shrink to.

Resize-Partition -DiskNumber 1 -PartitionNumber 1 –size 30GB 5. Now use the Hyper-V, VMM or failover cluster manager GUI to compact the VHD.

5. Compact the VHDX. The quickest method is to use the Hyper-V, VMM or failover cluster manager GUI to compact the VHD. e.g. right click on the VM>Select settings>Left click on the disk>click “Edit”>Click “Next”>Select “Compact”>Click “Next”>Review the change summary screen and click on “Finish.” This might take some time if there is a lot to shrink. The VM also needs to be powered off for this final step.

Happy shrinking!

September 22, 2015September 22, 2015

List windows updates or Hotfixes installed after a certain date

Ever found something broken and wondered what updates have been installed since a certain point in the past?

PS C:\Users\robin> Get-HotFix | ? installedon -gt 06/09/2015

Source        Description      HotFixID      InstalledBy          InstalledOn
——        ———–      ——–      ———–          ———–
Mxxxxx    Update           KB2693643     NT AUTHORITY\SYSTEM 09/07/2015…

May 11, 2013May 13, 2013

Windows Server 2012: Thoughts so far

When I first booted up into windows server 2012 I genuinely couldn’t believe my eyes. The user interface formerly known as metro? On a server? Who is going to have a touchscreen on a server? But slowly, it starts to make sense. When you open the start menu it is usually because you are looking to start an application or configuration console from a shortcut, and with the old start menu the rest of the screen is somewhat redundant. Not any more; every icon in the menu now fills the entire screen and the Win key + search term combo stills works so I am happy. The only thing I miss is the ability to shift and right click on an item to run it as another user; now I have to pin it to the task bar and go to the desktop to do so. A minor inconvenience, but an inconvenience none the less.

Now on to some of the amazing new features of 2012: I love the ability to team network interfaces at the OS level. Previously, you were at the whim of your network card drivers to achieve any kind of teaming, whereas now you can use whatever network interfaces you like to create a bit of redundancy and/or failover. I can’t say enough good things about the new server manager either; it makes adding roles and features a breeze, particularly when compared to previous versions. You can quickly and easily add a role or make a change to entire clusters of servers from one server manager console.

The new resilient file system, and in particular the Deduplication feature of 2012, look very exciting and I suggest everyone tries building a test 2012 server and moving their file shares to it, just to see how much space you could save with deduplication. Actually using it in production could be a little trickier as it requires backup solutions that are Deduplication aware ,or else on a restore you may find yourself rapidly running out of space or encountering other issues. I don’t imagine it will be long before vendors include support for this feature. Another great new feature of the resilient file system is that you can now run check disk online; never again will you have to restart and wait while check disk trawls tediously through a volume before the operating system starts. The new resilient file system also does not re-use the same disk blocks during a write, so if there is a power outage or other failure, the original data will still be readable.

PowerShell 3 is touted to have over 2,400 command-lets and to be honest I am only starting to scratch the surface of what is now available, but it is safe to say that if you liked PowerShell in 2008 R2, you will love it in 2012. A useful trick I use to learn more about PowerShell is to first configure something in the GUI and then hunt through the PowerShell logs in event viewer to see all the actual commands that were run. Also dont forget to check out the new PowerShell 3.0 ISE.

There are other less tangible improvements such as boot time; it certainly feels a lot quicker to be up and running than previous versions.

There are a few gotchas. For example, while deploying a new Lync 2013 environment I discovered that 2012 has much tougher certificate requirements, and even a single non self-signed certificate in the “Trusted CA” certificates folder was enough to upset the reset of the certificates in the personal store. So if you are planning to move to 2012 any time soon, now is a great time to think about cleaning up your certificates and rationalising any you have pushed out via group policy. Another issue I faced was with a core edition server which had many updates applied. I then tried to install the server GUI and found myself unable to do so, I would recommend that you build all servers with the GUI, update them and then uninstall the GUI so that you have the option of re-adding it later should you so desire. The new “Minimalism” interface offers a reasonable compromise if the core is a little to extreme for you but you want to realize the benefits of a lighter footprint.

Hyper-v is now in its 3^rd generation and each new version feels a little more mature and stable, and if you are already paying for datacentre licenses for your hosts this new version makes it harder than ever to justify paying for a competitor’s Hyper-visor when this is already included in your datacentre licenses. Unfortunately I have not yet built a 2012 Hyper-v cluster, but even running it on single hosts I can see improvements. Additionally, running native hyper-v guests means that you can always export them to Azure, either for a bit of extra capacity or as a backup/DR solution. My only gripe is that the new Hyper-v management tools can’t manage older 2008 R2 Hyper-v hosts, but I guess that is one of the prices of progress.