My Windows Subsystem for Linux setup

My trusty 2011 Macbook Air had started to show its age – a couple of missing keys, the fan being on pretty much all the time – so I looked for a replacement. After some tedious investigation, I settled upon a Dell XPS 9370 from their Outlet store. Windows 10 can run Linux now, via the Windows Subsystem for Linux (WSL), so it seemed like an interesting thing to spend some time with, and if it all went sour I could always fully install Linux instead.

I’m sure some people have a reason to prefer one OS over another, but I have never really understood it, at least not since my last RISC OS machine :) They all have awful file management (as do I), and they’re all much of a muchness in every way I seem to use them; I can install most things I want to use on anything, and most of the rest is in a web browser.

I’ve had it about a month now, and I thought it worth writing up some of the things I’ve done to aid shared development between Windows and WSL.

Applications

So firstly, let us rattle off the list of apps I installed. Many were identical to what it would be on a Mac: Firefox (with Containers), Thunderbird, Slack, Steam, VSCode, Licecap, Spotify, Git, KeePassXC, Chrome, and OpenVPN. As replacements for some small Mac programs, I have installed HexChat (IRC), KeyFerret (special character input), Ditto (clipboard memory), and NVDA (screen reader).

I also newly installed LDraw because my four year old has made her own Lego dinosaur and wanted to generate some instructions :)

Setting up WSL

To install WSL and Ubuntu, I followed Microsoft’s instructions (except in more stereotypical Windows behaviour, I activated WSL by going to Control Panel → Programs and Features → Turn Windows features on or off, rather than using the command line). Ubuntu was indeed a one click install and initial set up worked fine. I’m not sure this is something I’d ever have predicted years ago!

Shell and terminal

fish is my shell of choice (saves so many typos). If you start WSL with wsl.exe rather than bash.exe, then it uses your default shell. I tried a few terminals and settled on Hyper with the Fira Code font, though it is somewhat annoying (copy and paste use Ctrl-Shift, new tabs don’t keep the same directory, etc). Thankfully I have always loved autojump, which alleviates the latter annoyance, making project navigation trivial (j f takes me directly to my main FixMyStreet git checkout).

File permissions

In order for WSL to maintain Linux file permissions, you need to enable metadata on the mounted C drive. wsl-config documentation tells you how, but not why, which can be found at this blog entry. My wsl.conf looks like this: 

[automount]
enabled = true
options = "metadata,umask=002"
mountFsTab = true

Database server

postgres installed and ran fine in WSL, but I’d like it to start on computer start. I found the instructions at https://dev.to/ironfroggy/wsl-tips-starting-linux-background-services-on-windows-login-3o98 helpful, though I made a few changes (e.g. I added a file to /etc/sudoers.d).

Sharing ssh keys between Windows and Linux

So now I was at a point where I had a working Linux installation inside my Windows laptop. But I didn’t want to keep it totally separate, because that would make development harder. I wanted them to work together where that would be useful.

For example, KeePassXC lets me store my SSH key passphrase within it, and automatically add/remove it from the SSH agent in Windows while unlocked. I didn’t want to have to enter my passphrase within WSL when running SSH, in order to add the key to the Linux SSH agent.

Firstly, I made the /home/matthew/.ssh directory in WSL a symlink to /mnt/C/Users/Matthew/.ssh so my SSH keys could be shared. I made the SSH agent run automatically on Windows (it’s not by default), and then installed ssh-agent-wsl which can pipe the Windows SSH agent information through to WSL.

Everyone appears to have slightly different SSH agent script to run on login; here’s mine: 

~ $ cat .config/fish/functions/fish_ssh_agent.fish 
set SSH_ENV $HOME/.ssh/environment
set SSH_AGENT /mnt/c/Users/Matthew/bin/ssh-agent-wsl -c -b

function __ssh_agent_needs_starting -d "check if ssh agent is already started"
    if test -f $SSH_ENV
        source $SSH_ENV > /dev/null
    end

    ssh-add -l >/dev/null 2>&1
    if test $status -eq 2
        return 0
    end
    return 1
end

function __ssh_agent_start -d "start a new ssh agent"
    umask 077
    eval $SSH_AGENT > $SSH_ENV
    source $SSH_ENV > /dev/null
end

function fish_ssh_agent
    if __ssh_agent_needs_starting
        __ssh_agent_start
    end
end

As you can see I run ssh-agent-wsl with -b, as without that it started using runaway CPU after the last terminal window was closed.

Update August 2020: I have upgraded to WSL2, and the above is now a bit simpler. I installed npiperelay in ~/winhome/.wsl/ (winhome a symlink to my Windows homedir), installed socat in Ubuntu, then changed the above function to: 

set -x SSH_AUTH_SOCK $HOME/.ssh/agent.sock

function fish_ssh_agent
    ss -a | grep -q $SSH_AUTH_SOCK
    if [ $status -ne 0 ]
        rm -f $SSH_AUTH_SOCK
        begin
            setsid socat UNIX-LISTEN:$SSH_AUTH_SOCK,fork EXEC:"$HOME/winhome/.wsl/npiperelay.exe -ei -s //./pipe/openssh-ssh-agent",nofork &
        end >/dev/null 2>&1
    end
end

Sharing git repos between Windows and Linux

One very frequent thing I do is work with git repositories in both Windows (using VSCode) and WSL, and so want to run git commands in both. So I needed to make sure these interoperated okay.

In Windows git, I set core.autocrlf=false, as I don’t want git to do any cleverness – this repo will be accessible by two different OSs at once! I set core.filemode to false as otherwise it gets confused about executables

It took a while to track this down, but Windows git includes its own SSH commands, which meant I was getting asked for my passphrase even though I had my key loaded into the Windows SSH agent. Thankfully, recent git lets you set core.sshCommand to the SSH you wish to use, so I set it to C:\Windows\System32\OpenSSH\ssh.exe (and ssh.variant to ssh) and this was resolved.

The only annoyance left was symlinks. I enabled symlink creation by turning on Developer Mode and adding my user to the correct policy group, as explained at https://github.com/git-for-windows/git/wiki/Symbolic-Links, which meant I could set core.symlinks to true. Symlinks then worked fine in both Windows and WSL, but sadly git status would say that the symlinks were modified in one OS or the other (ie. if I git reset them in Windows, WSL would say they were modified, and vice-versa). After asking on StackOverflow, it seems this is one thing I have to live with, and as mentioned there have marked them as assume-unchanged, as there are only a small number and they very rarely change.

‘open’ script

On a Mac, you can type "open [file]" to open that file with the default program for it. To recreate the same thing, I created a tiny script in my bin directory: 

#!/usr/bin/env fish

cmd.exe /C start "" $argv
For slightly more advanced behaviour, I have just discovered wsl-open which I should probably install.

Cypress

FixMyStreet has a lot of tests in its test suite – most of them are “back end” tests (so run on a server testing various parts of the server-side code do what they should, pretending to be a user fetching pages and submitting forms etc), but some are “front end” tests, which run in an actual browser (this can be done “headlessly” so no actual browser window opens up when they’re run), connecting to a running test server, and check that parts of the front end (mostly JavaScript) do what they should.

For this, we use a piece of software called Cypress which helps automate it all and aid development, and wrote a script to do the set up/tear down parts of creating a test database, running a test server for Cypress to connect to, and so on. Our test suite runs on Travis with each commit, and our front end tests running there push video recordings of the running output to a service called Cypress Dashboard, so you can see the output of the latest run. For developing locally, you can do the same as Travis and run the front end tests headlessly, only seeing the output text and videos, or you can run Cypress as a GUI and see the browser running the tests, inspect the DOM at any point of a test, easily rerun a particular test, and so on.

As Linux on Windows 10 is basically ‘native’, I can call Windows programs from inside Linux. I have Cypress installed natively on Windows (installed node and npm install cypress), and so I run the test script in WSL with a special argument (unsurprisingly, --wsl), which at the point it wants to run Cypress, calls out to it in Windows to run there. (I did have the option of installing Cypress in Linux, setting up a Linux window manager and running it entirely inside, but the GUI support for Linux on Windows isn’t their core focus and I thought this way would probably be easier.) This was a little tricky to find the precise combination of commands that worked (by default, running Windows Cypress from Linux either assumed you were in Linux and died because there was no Linux window manager, or assumed you were in Windows and couldn’t find the command to run because it tried to use Windows path names in Linux), but now that it’s done, it’s all working smoothly.

Update August 2020: With WSL2, I still have it working, but I have to have the code checked out under /mnt/c rather than in Linux, otherwise it is incredibly slow.

Conclusion

I always used MacPorts on my Mac, which worked nicely; however, it is also nice to have Linux directly installed. I’d also have been fine with a full Linux installation, but don’t really have the time at present to play about with that, so the presence of WSL means I am happy to use Windows until such a time as more free time becomes available(!)