If you are here just for the steps to follow, jump here!

Background

When you start to have many data science projects on your computer, the hard disk gets full of:

  • CSV and parquet files
  • images (for well, image recognition)
  • memes
  • docker images of the model you want to deploy

On my Windows laptop, I got to the point of not having enough disk space for building a single image, since I got under 2 GB of space left on a 512 hard drive.

For making up space I started to archive old projects and data on another drive but the problem persisted.

When I used docker system prune --all to remove unused images and containers, Docker stated several GB were reclaimed. Unfortunately, when I checked the disk space I did not see any improvement. Where was the problem?

In the quest to delete large folders and files, I was using WinDirStat to hunt the biggest items.

At first, I removed files and folders of which I knew the content, but then I realized that 106 GB (20% of my disk!) were occupied by the Docker WSL (Windows Subsystem Linux).

So apparently, Docker was freeing the space but the WSL would be still occupied.

After some googling, and attempts with wsl --manage docker-desktop --set-sparse true, I found How to Shrink a WSL2 Virtual Disk that helped me solve the issue. In the following section I summarized how to use diskpart to reduce the space occupied on Windows by Docker.

Procedure

  1. Remove all stopped containers, dangling images, unused build cache and networks with

    docker system prune --all

    if you want to remove also volumes add flag --volumes

  2. (Optional) if you tried to set the distribution to sparse before this procedure set sparse property to false by running

    wsl --manage <distribution> --set-sparse false

  3. Run

    wsl --list --verbose

you should see listed docker-desktop-data and docker-desktop running.

  1. If they are running

    wsl --terminate <distribution>

  2. (Recommended) back up WSL2 installation using

    wsl --export <distribution> <path_backupfile.tar

    (in case something goes wrong you can run cmd wsl --import <distribution> <path_backupfile.tar>)

  3. Run

    diskpart

    a shell with DISKPART in the prompt should open

  4. In the diskpart shell

    select vdisk file=<FULL_PATH_TO ext4.vhdx>

    It should be %USERPROFILE%\AppData\Local\Docker\wsl\data\ext4.vhdx

  5. and then

    compact vdisk

    After waiting a couple of minutes, you can enjoy a bunch of free space!

  6. Just to be sure, restart the laptop (after all we are on Windows).

The first time I used this procedure I nearly halved the space uselessly occupied going from 106 GB to 54 GB. A bunch of space for new memes datasets!