Version control systems like subversion are important for software development. They keep the history of source code and allow management of code branches. Merging is important for collaboration and developing branches. Subversion has revisions with mergeinfo for each branch. Git allows cloning and merging of distributed repositories. There is good documentation in (manpages, svn-book, pro-git).

Submodules - Divide and conquer git

A git repository can add other git repositories as submodules. A submodule can be a existing repository inside the working tree or a repository outside of the working tree. If the submodule url is outside there will be a bare clone in ./git/modules of the super project. A submodules has a repository url and and working directory path.

Adding existing repo

Usually a submodule path and url are at different locations. But if the path of the submodule is the actual repository url there is no clone and no gitfile. Similar to symbolic links submodules use gitfiles to locate the git directory. A gitfile separates working-tree and git directory. It is a text file with the location of the gitdir. The submodule path and url can be same. In this case there is no gitfile. Only with gitfile its possible to deinit the submodule. The deinit command removes the submodule from worktree without losing the repository.

The deinit command would remove the complete repository. So deinit is for submodules with gitfile. This removes the gitfile and

The git-rm command removes a submodule configuration and migrates .git to .git/module of the super project. For submodules with .git directory only the configuration can be removed. There is a working example of submodules on https://www.x.org/wiki/Building_the_X_Window_System

The submodule to to a gitfile cannot be reinit. Submodules cannot be bare repositories. The push requires a detached checkout. srv/git/wiki already exists and is not a valid git repo

Difference between submodules with .git directory instead of a gitfiles

A submodule with [path] from url ( <repository> ) is registered in .git/config as path after init. With sync submodule config url is updated from .gitmodules to .git/config.

git submodule add ./tools/depot_tools/ tools/depot_tools/

A git repository can add other git repositories as submodules. A submodule can be a existing repository inside the working tree or a repository outside of the working tree. If the submodule url is outside there will be a bare clone in ./git/modules of the super project. A submodules has a repository url and and working directory path.

Adding existing repo

Usually a submodule path and url are at different locations. But if the path of the submodule is the actual repository url there is no clone and no gitfile. Similar to symbolic links submodules use gitfiles to locate the git directory. A gitfile separates working-tree and git directory. It is a text file with the location of the gitdir. The submodule path and url can be same. In this case there is no gitfile. Only with gitfile its possible to deinit the submodule. The deinit command removes the submodule from worktree without losing the repository.

The deinit command would remove the complete repository. So deinit is for submodules with gitfile. This removes the gitfile and

The git-rm command removes a submodule configuration and migrates .git to .git/module of the super project. For submodules with .git directory only the configuration can be removed. There is exampe for submodules on https://www.x.org/wiki/Building_the_X_Window_System

Remove submodule section from config

git config --remove-section submodule.m4
git config --file .gitmodules --remove-section submodule.m4

Repositoy urls

Submodule urls will be cloned. Thats why a public url is recommended in the superproject for clone. Inside the submodule the origin can be changed to a privat url.

Reinitalization

The hash of the submodule is stored in the index file. If the tremote repository is reinitialized there is a "warning: no common commits". It is possible to reuse the existing repository with a forced update. After a forced update the merge needs option "--allow-unrelated-histories".

GIT_DIR and GIT_WORK_TREE

Git has a conept of git-dir and work-tree. The environment variable GIT_DIR is mentioned in man pages. But it does not work for git submodule because work tree is required too. In the worktree is a file .gitmodules required with the submodule mapping. Git has an option -C to change working directory for this situation. Without this option the errors are "fatal: no submodule mapping found in .gitmodules for path" or "fatal: $program_name cannot be used without a working tree". This was found on git@vger.kernel.org mailing list. ( http://git.661346.n2.nabble.com/git-submodule-ignores-git-dir-tt7639301.html)

Example for GNU/Linux root file system with submodules

/etc repository for etckeeper /var/www gitfile to /srv/git/www/.git /var/mail repository /home repository /usr/share/cgit repository /srv/git/www bare repository /srv/git/wiki bare repository

Rebase and Merge

During merge this can be useful

git reset --soft
git stash push --
git commit --all --amend

Ignore untracked files

Add untracked files to ignore list.

echo download >> .gitignore

Untracked files will be removed with git-clean except files of other .git repositories. The folders with a valid .git directory are ignored from clean. This can be used to ignore downloads from versioning and prevent them from cleaning.

git init download/.git

Commands to push a submodule upstream git-push
[https://raw.githubusercontent.com/wiki/furrymcgee/furrymcgee.github.io/git/git-push.sh]

# Some useful git commands to maintain the wiki

## Clone github pages with wiki
git clone --recurse-submodules https://github.com/furrymcgee/furrymcgee.github.io.git

## Pull the master branch with wiki submodule
git checkout --orphan master
git remote add origin https://github.com/furrymcgee/furrymcgee.github.io.git
git remote set-branches origin master
git pull --recurse-submodules origin master
git submodule add -b origin/master https://github.com/furrymcgee/furrymcgee.github.io.wiki wiki
git submodule update --init wiki
git submodule foreach git pull --strategy-option=ours origin master
git pull --recurse-submodules --strategy-option=ours origin master

## Commit changes
git reset HEAD^
git commit --all --amend

## Push wiki submodule to production
git branch --set-upstream-to=origin/master
git push --recurse-submodules=on-demand
git submodule foreach git push --delete origin master
git submodule foreach git push --set-upstream origin master

## Check current remote url and upstream branch
git remote get-url origin
git config branch.$(git name-rev --name-only HEAD).remote

## History of current branch
git log --stat --oneline --graph

## Delete remote wiki branch
git submodule deinit wiki
git push --delete origin master
git remote prune origin

Import subversion into git git-svn
[https://raw.githubusercontent.com/wiki/furrymcgee/furrymcgee.github.io/git/git-svn.sh]
)

#!/bin/bash
# Join subversion and github repositories

# Checkout btonline-btech and btmux repositories
git svn clone svn://svn.code.sf.net/p/btonline-btech/code .
git remote add btmux http://github.com/gtaylor/btmux
git fetch btmux master
git checkout btmux/master -b btmux/master
git checkout master -b btonline-btech

# Rebase btmux trunk, tags and branches from btonline-btech onto master
git ls-tree --name-only btonline-btech -- btmux/trunk btmux/tags/ btmux/branches/ | \
while read; do

        # Split subtree from btonline-btech trunk, tags and branches
        git checkout btonline-btech
        git subtree -q split --prefix $REPLY | xargs git checkout -q

        # Find a commit prior initial git-svn-id for rebase onto btmux
        git log heads/btmux/master | grep '^commit\|git-svn-id:' | \
        grep -B1 -f <( git log heads/btmux/master | \
        perl -n -l <(cat \
<<-PERL
                                # Perl returns git-svn-id less then initial git-svn-id
                                /git-svn-id: / and
                                /(?<=@)[[:digit:]]+(?= )/ and
                                $& <= $(
                                        # Get initial git-svn-id of subtree
                                        git log HEAD | \
                                        grep -o -P '(?<=@)[[:digit:]]+(?= )' | \
                                        tail -n 1
                                )
                                ? print $&
                                : undef
PERL
                ) | sed -e 's/^/@/' -e 's/$/ /'
        ) | \
        grep commit | head -n1 | cut -d' ' -f2 | \
        xargs git rebase -q --strategy-option=theirs $(
                # Get initial commit of subtree for rebase
                git log HEAD | grep '^commit\|git-svn-id:' | tail -n2 | \
                grep -o -P '(?<=commit )[[:alnum:]]*$'
        ) HEAD --onto 2>&- && \
        git checkout -b $REPLY && \
        git checkout -q btonline-btech && \
        git rm -q -r $REPLY && \
        git stash || \
        git rebase --abort
done