From e37a817fe0652f5fab0b0e6926a716a7694779d4 Mon Sep 17 00:00:00 2001 From: Cassie Jones Date: Tue, 18 Feb 2020 18:03:39 +0100 Subject: [PATCH] Write a post about how I do my Git hosting --- posts/2020/hosting-my-own-git.md | 230 +++++++++++++++++++++++++++++++ 1 file changed, 230 insertions(+) create mode 100644 posts/2020/hosting-my-own-git.md diff --git a/posts/2020/hosting-my-own-git.md b/posts/2020/hosting-my-own-git.md new file mode 100644 index 0000000..9e801cd --- /dev/null +++ b/posts/2020/hosting-my-own-git.md @@ -0,0 +1,230 @@ +--- +title: "Hosting My Own Git" +date: 2020-02-18 +tags: +- git +- web +summary: "I don't want to have to use GitHub for all my personal projects while they still support ICE, so I figure out how to do my own hosting." +--- + +I've wanted to get off GitHub at least for new projects due to their [supporting ICE][], but I hadn't figured what I was going to do until recently. +Then, Jordan Rose made [a post about setting up personal Git hosting][gitweb shared] that I liked, and I learned how easy it was to set up, and about [GitWeb][]. +So I set up my own shared Git hosting that suited my needs, which you can take a look at at [git.witchoflight.com][]. + +[supporting ice]: https://github.com/drop-ice/dear-github-2.0 +[gitweb shared]: https://belkadan.com/blog/2020/01/Gitweb-on-Shared-Hosting/ +[gitweb]: https://git-scm.com/docs/gitweb +[git.witchoflight.com]: https://git.witchoflight.com + +### Hosting *Your* Own Git + +It turns out if you only care about SSH access, hosting Git is way easier than I believed it would be. +If you have a server with SSH access, as the user you want to use Git as, run `git init --bare` to create a repository. +On my server, I created a `git` user, and a `/repos/` folder owned by the `git` user where I could create all of my repos. +Then, it's just a matter of using `@:` as your remote. +With that, your git and ssh handles all of the connections and transfer, you don't need a special Git server running or anything. + +As a specific example of using this: +When I want a new repository on my server "Sunstone," I ssh there and create a new bare repository: + +```bash +$ ssh sunstone +$ su git +$ cd /repos/ +$ git init --bare fancy-new-repo.git +``` + +I include the `.git` suffix as a personal preference, it's just part of the path and doesn't do anything special, so you can leave it off if you prefer. +Now I can clone it as `git clone git@sunstone:/repos/fancy-new-repo.git`. +You can do all sorts of setup to make this work fine for letting multiple users push, including users that don't have access to most of the computer, but I'll leave those kinds of things to [the Git docs][]. +There are other fancy things you can do to make it nicer, I'll get to some of those later. + +[the git docs]: https://git-scm.com/book/en/v2/Git-on-the-Server-Setting-Up-the-Server + +### Sharing With The World + +One of the big things that GitHub offers is the ability for anybody to easily read and clone the code without any particular authorization. +It turns out that Git itself offers a solution for this, called [GitWeb][]. +It's a server that makes readable web pages for Git repositories, and as a side effect you can do plain HTTP(S) clones. + +For this, I followed the setup directions [in the docs][gitweb] and in [Jordan's post][gitweb shared], as well as searching around online for various setups. +In the end, I put a few pieces together which were different from what I found. +My setup is a bit different from those because I'm on nginx. +Like Jordan, I want people to be able to use the nice URLs, and also to clone over HTTPS. + +#### Configuring GitWeb + +First, `/etc/gitweb.conf`: + +```perl +our $projectroot = "/repos"; +our $site_name = 'Witch of Git'; +our @git_base_url_list = qw(https://git.witchoflight.com); +our $omit_owner = true; +$feature{'highlight'}{'default'} = [1]; +$feature{'pathinfo'}{'default'} = [1]; +$feature{'search'}{'default'} = [0]; + +our %highlight_ext = ( + %highlight_ext, + (map { $_ => $_ } qw(rs lua)), + toml => 'ini', # good enough for lazy :? +); + +@stylesheets = ("/static/gitweb.css"); +$logo = "/static/git-logo.png"; +$favicon = "/static/git-favicon.png"; +$javascript = "/static/gitweb.js"; + +$export_auth_hook = sub { not ($_[0] =~ /\.git\z/) }; +``` + +This does a few things. +First, some basic variables. +Where it should look for files, what the title should be, stuff like that. +I enable syntax highlighting, and importantly enable `pathinfo`, which changes URLs from messy query strings to readable URLs. +If you're not careful, this can cause trouble in the server config later, but I'll go over that. + +I need to modify `%highlight_ext`, which specifies what file extensions map to which syntax highlighting. +This is based on looking at `/usr/share/gitweb/gitweb.cgi` which sets up all the features that you can modify, and it has a comment: + +```perl +# see files in /usr/share/highlight/langDefs/ directory +``` + +Setting `pathinfo` requires that we specify the stylesheets, logo, favicon, and JS explicitly, for some reason. + +The very last line is my own idea here. +The `$export_auth_hook` is a callback that checks if a given repo should be exported. +I create two copies of all my repos with a symlink, a `repo.git`, and a plain `repo` linked to the first. +I create the former because I like having my remotes have `.git` on them, and I create the latter because it makes for nicer URLs in GitWeb. +I use the hook here to avoid exporting two copies of each. + +#### Nginx + +I happen to like Nginx as a static file server. +I planned to have it already running on my server, so it seemed like the best idea to get GitWeb running through there. +Unfortunately, GitWeb is a CGI script, and Nginx doesn't directly support them. +I searched around on three different fronts. +One, tutorials for Nginx CGI scripts. +Another, tutorials for setting up GitWeb with other servers. +And also in the middle of those, tutorials actually on GitWeb with Nginx. +They didn't completely solve my issues, but I used [a tutorial from the Arch wiki][] and [a tutorial in a gist][]. + +Synthesizing all this together, I made my `/etc/nginx/sites-available/gitweb`: +```nginx +server { + listen 80 default_server; + listen [::]:80 default_server; + server_name git.witchoflight.com; + root /repos/; + + location / { + try_files $uri @gitweb; + } + + location /static/ { + root /usr/share/gitweb/; + } + + location @gitweb { + include fastcgi_params; + gzip off; + fastcgi_param SCRIPT_FILENAME /usr/share/gitweb/gitweb.cgi; + fastcgi_param PATH_INFO $uri; + fastcgi_param GITWEB_CONFIG /etc/gitweb.conf; + fastcgi_pass unix:/var/run/fcgiwrap.socket; + } +} +``` + +This will try to route files at the root through GitWeb, and files under `/static/` to the filesystem. +GitWeb is set up as a fastcgi client, with fastcgi configured to run through `fcgiwrap.socket`. +I pass the request uri in the `PATH_INFO`, seems to be necessary to handle routing with the pretty paths. +As the absolute basics, you always have to pass the script filename, and the config. +I actually got stuck for about an hour here with a mismatch between `fcgiwrap.socket` and `fcgiwrap.sock`, so check your spellings carefully while you configure things! +With this setup, GitWeb will correctly format HTML pages when it should, and also correctly pass through files inside the repositories so that HTTP(S) cloning works fine. +Speaking of HTTPS... + +[a tutorial from the arch wiki]: https://wiki.archlinux.org/index.php/Gitweb#Nginx +[a tutorial in a gist]: https://gist.github.com/mcxiaoke/055af99e86f8e8d3176e + +#### Using HTTPS + +As my final step, I got a cert set up for HTTPS. +I've never set up my own HTTPS before, just used it if it was provided by my hosting, but setting one up with [Let's Encrypt][letsencrypt] ended up being quite easy. +I just directly followed the steps on their "Get Started" page and things worked first try. +I pointed it at my GitWeb Nginx config and it slightly rewrote it to add certs and things, which I left out above. + +[lensencrypt]: https://letsencrypt.org/ + +### Making It Nice + +I do *lots* of hobby projects, so I want it to be nice to set up repos. +In the `git` user's home directory, I have a `git-hooks/` folder that holds all of the "universal" base hooks I come up with, a `git-template/` folder that stores the template for new git repos, and a `make-repo` script which does the setup for new repositories. + +First, the `git-hooks/`. +I have a dedicated folder for them so the template can symlink that, so I don't need to edit all the different hooks. +Currently I just have one, the `post-update` hook. +It contains: + +```bash +#!/bin/sh +git update-server-info +git cat-file blob HEAD:README.md | pulldown-cmark > README.html +``` + +We need to `git update-server-info` to prepare the data that HTTP clones and things need. +Then, we prepare the `README.html` file that GitWeb displays on the main page of a project. +For this, `git cat-file blob HEAD:README.md` grabs the latest `README.md` from the main branch, and then we render it to HTML. + +In the `git-template/` folder, I make symlinks from the `~/git-hooks/` folder into the `git-template/hooks/` folder so they just need to be updated in one place. +I also change the default branch to `develop` since I prefer that instead of `master`. +You can set the template dir with `git config --global init.templatedir `. + +And then the `make-repo` script I just have set up so that it will create both `repo` and `repo.git` in the right place when I ask for it, and let me rewrite the description immediately. + +```bash +#!/bin/bash +set -e + +if [ -z $1 ]; then + echo "Usage: make-repo " + exit 0 +fi +# Strip .git from the name if you specify it, for uniformity +repo=$(basename -s.git $1) +# We don't want to work as root, but we login as root (oops?) +# so it's convenient to automatically switch to the git user +if [ "$EUID" -eq 0 ]; then exec su git -c "$0 $1"; fi +# Only allow running as the git user, for file permissions purposes +if [ $(whoami) != "git" ]; then + echo "Please run this tool as the 'git' user" + exit 1 +fi +# Don't overwrite existing repositories +if [ -d "/repos/$repo.git" ]; then + echo "Repository '$1' already exists" + exit 2 +fi + +cd /repos/ +# Make a bare repo for SSH cloning +git init --bare "$repo.git" +# We immediately edit the description file that will show up in GitWeb +sensible-editor "$repo.git/description" +# And have both repo and repo.git point to the same place for cloning +ln -s "$repo.git" "$repo" +``` + +### What Now? + +Well, now I have Git hosting! +And if you want some, you can probably figure it out too, it's less work than I thought it might be! +I've done some minimal styling, making it a narrower column and fixing some things, but I want to do more. +That's a project for another day, though. + +Now, go look at my projects. +Some of them are cool! +I'll try to write about them soon. :) +[git.witchoflight.com][]. -- 2.43.2