Git Mirroring to GitLab

Steps for setting up automatic mirroring cron jobs:

1. Add a new user:

   sudo adduser --disabled-login --system git-mirroring
   

2. Start a shell as the git-mirroring user:

   sudo -H -u git-mirroring /bin/bash
   

3. Switch to the home directory:

   cd
   

4. Create an executable file sync.sh (replace nano with the editor of your choice):

   touch sync.sh
   chmod +x sync.sh
   nano sync.sh
   

   1. Type in the following contents:

      #!/bin/sh
      for repo in ~/*.git; do
          cd "$repo" && git fetch -q && git push -q gitlab || echo "sync failed for $repo"
      done
      

   2. Save and exit the editor.

5. Create a ssh key:

   ssh-keygen
   

   Press enter for all prompts -- leaving the file names as default and with an empty password.

6. For each repo to be mirrored:

   In the following commands, replace $UPSTREAM_URL with the url for the upstream repo, $NAME with the name selected for the local directory.

   1. Create an empty repo using the GitLab website.

      Warning

      Following these steps will overwrite anything that is in the GitLab repo.

   In the following commands, replace $MIRROR_URL with the ssh url for the mirror, you can get it by clicking the Clone button for the empty repo you just created.

   2. Add the ssh key for the git-mirroring user to the GitLab repo as a deploy key with read/write access:

      Get the ssh public key for the git-mirroring user:

      cat ~/.ssh/id_rsa.pub
      

   3. Clone the upstream repo:

      git clone --mirror "$UPSTREAM_URL" ~/"$NAME".git
      

   4. Switch to the directory where you just cloned the repo:

      cd ~/"$NAME".git
      

   5. Add the mirror as a new remote to allow pushing to it:

      git remote add --mirror=push gitlab "$MIRROR_URL"
      

   6. Try pushing to the new remote to ensure everything is operating correctly:

      git push gitlab
      

7. Switch back to the home directory:

   cd
   

8. Ensure sync.sh works properly:

   ./sync.sh
   

9. Set up the cron job for running sync.sh:

   1. Edit the user's crontab:

      crontab -u git-mirroring -e
      

   2. Add the following line to the end, then save and exit:

      * * * * * exec ~/sync.sh >> ~/sync.log 2>&1
      

10. Wait for the cron job to run (2 minutes should be sufficient), then check the log file:

    less ~/sync.log
    

    The log file should exist and be empty, if it isn't empty, it contains error messages. If it doesn't exist, the cron job didn't run yet for some reason, try waiting a little longer.

11. Try pushing a commit to one of the upstream repos and seeing if it gets properly synced to the corresponding mirror repo. The sync job runs every minute, so wait 2 minutes and check.