Оффлайн
- Регистрация
- 1 Мар 2015
- Сообщения
- 1,481
- Баллы
- 155
The Mystery of the Eating Disk Space
If you're a Docker user on Windows and using WSL 2, you might have found yourself in the same shoes as me, where suddenly all the free space on my Windows (C: drive) vanished out of nowhere. You would think, maybe some cache, maybe Windows updates (No surprise there), or I might have malware or something?
While the answer might be true for all of the above, this time it was false for all of them above. Using a Disk Analyser tool, I found out that two files, ext4.vhdx and docker_data.vhdx, are consuming over 500 GB, and they both belong to the culprit, which is Docker.
Why Docker's VHDX Files Grow So Large
Docker Desktop on Windows, when using the Windows Subsystem for Linux 2 (WSL 2) backend, stores your Linux environment, Docker images, containers, and volumes within virtual hard disk files (VHDX). These VHDX files are typically "dynamically expanding." This means they grow in size on your Windows file system, as Docker needs more space.
Now, when you delete Docker images, containers, or data inside these VHDX files, the VHDX itself often (which was my case) doesn't automatically shrink on your Windows host. The space is marked as free within the virtual disk (inside the Linux itself), but the file on your C: drive remains at its largest allocated size.
In other words, like a physical disk, when you delete something, it doesn't really delete it from the disk but removes your access to it and makes the space available for rewriting.
The Usual Docker Pruning (Necessary, but Not Always Enough)
The first step is to use Docker's built-in pruning commands to clean up unused resources:
docker system prune -a --volumes
# Or more specific commands:
docker image prune -a
docker container prune
docker volume prune
In my case, running these commands reclaimed some space – about 14.7 GB. Helpful, yes, but nothing compared to the 500 GB+ the VHDX files were occupying. My Disk Analyser still showed ext4.vhdx at a whopping 377.7 GB and another docker_data.vhdx at 145.2 GB.
What Was Actually Inside Docker?
To ensure active data wasn't the main issue, I checked:
So, active Docker data (images, container layers, volumes) accounted for roughly 30- 35 GB. Clearly, this wasn't explaining the enormous VHDX sizes. The rest was "phantom" space.
Compacting Your VHDX Files
The real solution lies in manually compacting the VHDX files. This process forces the virtual disk to release the internally freed space back to your Windows operating system.
Important Note: While this process is generally safe, it's always a good practice to back up any absolutely critical data from within your Docker volumes or WSL distributions before performing disk operations.
Optimize-VHD -Path "PATH TO VHDX FILE" -Mode Full
The Sweet Relief of Reclaimed Space
After running Optimize-VHD on both my main Docker VHDX and the one associated with my Ubuntu WSL instance, the results were astounding!
The ext4.vhdx (Docker main data) shrank from 377.7 GB down to 117.2 GB.
The docker_data.vhdx (likely my Ubuntu WSL) shrank from 145.2 GB down to 52.0 GB.
That’s a total of over 353 GB reclaimed on my C: drive! The VHDX files now accurately reflect the space actually used by Docker and my WSL environments.
If you're a Docker user on Windows and using WSL 2, you might have found yourself in the same shoes as me, where suddenly all the free space on my Windows (C: drive) vanished out of nowhere. You would think, maybe some cache, maybe Windows updates (No surprise there), or I might have malware or something?
While the answer might be true for all of the above, this time it was false for all of them above. Using a Disk Analyser tool, I found out that two files, ext4.vhdx and docker_data.vhdx, are consuming over 500 GB, and they both belong to the culprit, which is Docker.
Why Docker's VHDX Files Grow So Large
Docker Desktop on Windows, when using the Windows Subsystem for Linux 2 (WSL 2) backend, stores your Linux environment, Docker images, containers, and volumes within virtual hard disk files (VHDX). These VHDX files are typically "dynamically expanding." This means they grow in size on your Windows file system, as Docker needs more space.
Now, when you delete Docker images, containers, or data inside these VHDX files, the VHDX itself often (which was my case) doesn't automatically shrink on your Windows host. The space is marked as free within the virtual disk (inside the Linux itself), but the file on your C: drive remains at its largest allocated size.
In other words, like a physical disk, when you delete something, it doesn't really delete it from the disk but removes your access to it and makes the space available for rewriting.
The Usual Docker Pruning (Necessary, but Not Always Enough)
The first step is to use Docker's built-in pruning commands to clean up unused resources:
docker system prune -a --volumes
# Or more specific commands:
docker image prune -a
docker container prune
docker volume prune
In my case, running these commands reclaimed some space – about 14.7 GB. Helpful, yes, but nothing compared to the 500 GB+ the VHDX files were occupying. My Disk Analyser still showed ext4.vhdx at a whopping 377.7 GB and another docker_data.vhdx at 145.2 GB.
What Was Actually Inside Docker?
To ensure active data wasn't the main issue, I checked:
- Docker Images (docker images): I had a number of images, some large, but the total was around 25 - 30 GB.
- Container Writable Layers (docker ps -as): The actual data added by running containers was minimal for most, though one container had a 1.34GB writable layer. The total for all containers was only a couple of GBs.
- Docker Volumes (docker volume ls): This was a key suspect. However, after inspecting the few anonymous and named volumes I had (like portainer_data), their combined actual disk usage within WSL was only about 1.5 GB.
So, active Docker data (images, container layers, volumes) accounted for roughly 30- 35 GB. Clearly, this wasn't explaining the enormous VHDX sizes. The rest was "phantom" space.
Compacting Your VHDX Files
The real solution lies in manually compacting the VHDX files. This process forces the virtual disk to release the internally freed space back to your Windows operating system.
Important Note: While this process is generally safe, it's always a good practice to back up any absolutely critical data from within your Docker volumes or WSL distributions before performing disk operations.
- Identify the Exact Paths of Your VHDX Files. Find where ext4.vhdx and/or docker_data.vhdx are.
- Shut Down Docker and WSL:
- Quit Docker Desktop: Right-click the Docker icon in your system tray and select "Quit Docker Desktop."
- Shut Down All WSL Instances: Open PowerShell as an Administrator and run: wsl --shutdown
- Compact the VHDX using Optimize-VHD
Optimize-VHD -Path "PATH TO VHDX FILE" -Mode Full
The Sweet Relief of Reclaimed Space
After running Optimize-VHD on both my main Docker VHDX and the one associated with my Ubuntu WSL instance, the results were astounding!
The ext4.vhdx (Docker main data) shrank from 377.7 GB down to 117.2 GB.
The docker_data.vhdx (likely my Ubuntu WSL) shrank from 145.2 GB down to 52.0 GB.
That’s a total of over 353 GB reclaimed on my C: drive! The VHDX files now accurately reflect the space actually used by Docker and my WSL environments.