Git development

Overview

Standard and suggested workflow for developers to contribute to project.

Workflow goals

• Keep full history of commits
• Group feature/fixes together so easy to see related commits
• Have an online/remote backup of coding activity, including feature/fixes etc
• Can roll back feature/fix merges easily

Reasons for chosing Forking flow

• Probably most robust
• common method for larger open source projects
• allows developers to start work without any authorization on main repo
• no need to clean up branches once finished
• cleaner commit history on central repo
• cleaner branch structure on central repo
• is more complex than branching with more steps involved

Protocol - Central repo

• has a master branch = release ready (some call stable or release branch)
• has a development branch = work preparing for next release (some call master branch)
• possible to have feature/fix branches, eg if large long term feature, although most changes will be done on forks and merged into development
• restricted developer access, maybe allow wider access for code reviewers only, pushing restricted to smaller number

Tasks to create new microservice repository on Bitbucket

1. Create new repository in correct Workspace/Project on Bitbucket
2. clone repository to local in correct folder matching stack/workspace
3. In the new local clone folder:
   1. copy [Empty Service Template], all files excluding .git folder but including hidden file .gitignore
   2. update files:
      1. ./README.md
      2. ./config/serverless.config.yml (iz_serviceName)
      3. ./app/package.json (name: "izara_{service name}")
   3. commit (eg: "Initial commit")
   4. push to remote
4. On Bitbucket
   1. Create new development branch, branching from master
   2. In Repository settings > Branching model set development branch to development, Production branch to Use main branch (will be master)
   3. Check user access is set correct
   4. Set any default reviewers
5. on local pull to get new branch, checkout development ready for changes

Tasks to create new npm module repository on Bitbucket/npm

1. Create new repository in correct Workspace/Project on Bitbucket Remote repository and workspace structure
2. clone repository to local in correct folder matching stack/workspace
3. copy .gitignore / README.md / package.json from another project
4. edit README.md to the new npm package name (two places)
5. edit package.json:
   1. name *this also sets the npm organization scope
   2. version: 1.0.1
   3. description
   4. repository url
   5. homepage
   6. dependencies may need to removed as needed
6. commit (eg: "Initial commit")
7. push to remote
8. in repositories root folder run: npm publish --access public
9. should be live at: https://www.npmjs.com/package/@izara_project/{package name}
10. On Bitbucket
    1. Create new development branch, branching from master
    2. In Repository settings > Branching model set development branch to development, Production branch to Use main branch (will be master)
    3. Check user access is set correct
    4. Set any default reviewers
11. on local pull to get new branch, checkout development ready for changes

Protocol - Each developer forks

• Fork the central repositories development branch
• Can share forked repository with multiple developers, or each have their own and can and use a similar forking flow to merge with each other

Protocol - Developers working on code

• Their fork is set as the remote push/pull repo
• This allows each developer to backup in progress work to online remote, without affecting central repo
• Clone fork to local com
• The central repo gets added by developers as a second remote repo on their local repo (called upstream), so when working locally they have both their fork repo and the central repo as remotes:

git remote add upstream git@bitbucket.org:stb_Working/{repository}.git

• Check remote repos:

git remote -v

• On local repo create a new feature branch, changes are made there:

git checkout -b [name_of_your_new_branch]

• Push feature branch to forked remote:

git push origin [name_of_new_feature_branch]

• Make regular commits to forked repo so work gets safely backed up
• Feature branch does not exist in central repo, only on developers forked remote repo
• Development branch on developers local computer never gets changed by developer, keep it as a clean copy of upstream repo's development branch. Changes to upstream are pulled into local development branch and can be merged into developers feature branch as needed

Protocol - Multiple people work on fork

If multiple people working in a fork must be careful about any rebasing/squashing that gets pushed to remote fork (forced push would probably be needed when updating remote). Pushing such changes to the history to the remote branch will mess with other developers who have cloned that branch, commits will be lost because rebase/squash commits are new commits that remove old commits, if other developers work is based on those deleted old commits their history will no longer exist in the commit history and their commit chain history gets broken.

example: when upstream changes we might want to rebase current work onto changes, which will delete commits in our feature branch, if we had previously pushed the deleted commits to fork remote and other developers were working from them, when we push the new history to fork remote those other developers work will be broken.

If current work already pushed to fork cannot change history if other people doing might be working on those pushed commits.

Idea 1

• Treat the forks feature branch as a public branch (no history changes), create a 3rd branch for personal development
• 3rd branch can be pushed to fork repo for backup but make sure no-one else works from it, other developers can have 4th/5th.. branches for themselves
• In the 3rd branch can change the history trusting no-one else is relying on it, once clean merge into public feature branch.
• Plan the commit history in feature branch to have clean history ready to merge into upstream when PR sent.

Idea 2

• Many people can work on forked feature branch with a rule no history changes to the forks remote branch
• Use merge commits to join in changes from upstream
• Once finished and preparing PR create a new branch that copies upstreams development branch
• Merge messy feature branch into the new branch cleaning it up. Maybe use rebase interactive, or squash
• eg see (Larger work section): https://sandofsky.com/workflow/git-workflow/
  • Not sure how rebase interactive would work if upstream changes were merged in during feature development, they would probably show as merge commits, but not sure how they get handled when eg squashing in rebase interactive
  • Also consider “Declaring Branch Bankruptcy” section

Idea 3 (use this)

• Combination of above ideas
• Main feature branch never change history
• Can have per developer branches if wanted/many developers
• Can work on the one feature branch (especially when only 1 developer)
• Use merge commits to merge in updates from upstream
• When finished squash all commits into 1 (or a few commits), this can be done on a new branch to protect the work on the feature branch
• This squashing will include merge commits from upstream, but should not matter because final squashed commits are what will be compared with upstream in pull request
• Create a new branch from squashed branch, rebase interactive with updated upstream, might have merge conflicts etc, but should only happen once (or a reduced few times) because new code was squashed
• This final rebased squashed commits are what is sent in pull request

Protocol - When central repo has updates

Step 1 - Get a copy of central repo to local com

• fetch into local upstream/development branch, then merges into local development branch
• Should try to fetch and merge into developers work as soon as possible so development and testing proceeds with most recent upstream code
• Developer pulls from central repo to their local repo's development branch, merging in changes:

git checkout development
git pull upstream development

• pull does a fetch + merge command, so no need to use rebase or no-ff, we are merging into local repo’s development branch which should be a clean copy of upstream development, no local changes made to it (local coding is done in feature branch/es), so local development branch will auto fast forward the merge

• alternative:

git fetch upstream
git checkout development (can do before fetch)
git merge upstream/development

Step 2 - Merge upstream changes into local feature branch

Option 1 - use if feature branch is public/shared with others (cannot change history)

• merge into feature branch, creating a merge commit
• no-ff prob does nothing here because feature branch will have changes already

git checkout feature-branch
git merge --no-ff development

• resolve any conflicts

Option 2 - If one person working on fork and can change feature branch history (normally use this)

Only do if comfortable changing history of feature branch - see notes If feature branch already pushed to fork>remote, probably need to create a new branch to rebase, or remote source control will not allow to push rebase changes (changed history) without forcing it

• Move to using local feature branch (maybe copy to new branch first to be safe)

git checkout feature-branch

• Update new upstream changes into feature-branch, using rebase to bump our new local work onto the end of the central repo changes:

git rebase -i development

• Can rebase interactive to allow joining and cleaning up the local feature branch commits when moving on top of upstream commits, ie: reduce the number of commits we have made locally into a cleaner picture, before pushing to fork remote repo and making a PR to upstream

If feature branch on fork is public and commits have been pushed, cannot change commit history

• Only rebase/squash local changes that have never been sent to a remote (risk used by other people)
• Cannot rebase, cannot squash, upstream merge will probably not fast forward because feature head is not a direct child of upstream head (changes have been made), so we are left with standard merge that creates a merge commit in the feature branch (a merge commit has two parents, the head of upstream and the old head of feature)

git checkout feature-branch
git merge --no-ff development

Step 3 - Update development branch on remote forked repo (optional)

• After pulling from upstream and merging into local development branch, can push to fork repo to keep its development branch up to date:

git checkout development
git push origin development

• this is not needed because when using forking model we do not use the development branch on remote, we always work from feature branch/es. We do use the development locally to keep our feature branch updated with upstream but no need to back that up

• feature branch can also be sent to remote after merging in upstream, depends on whether we are treating fork development as public/shared or single developer, and how we want to maintain the commits on the remote branches

Protocol - Finished feature, make pull request (PR)

Before deciding to do a PR

• If working with other developers in forked repo there might be changes made be them we need to pull to local and cleanup before submitting PR
• Make sure all tests work

Clean up feature before making PR

• Want to squash into one commit, or a few commits if really large change with multiple stages in coding
• squash in a new branch on local so keep a backup of the old commit history
• Use squash to compress all working commits into a small number to keep upstream commit history clean
• reference: https://blog.carbonfive.com/2017/08/28/always-squash-and-rebase-your-git-commits/
• Find SHA of the parent commit this feature branch first branched from, might be easy to see in BitBucket, or if only a few commits made can count using:

git log

• or:

git log --graph --decorate --pretty=oneline --abbrev-commit

• Perform the squash, this is on the single feature branch, not yet rebasing on top of upstream/development branch (although might be same if already merged in all upstream changes)
• if counted number of commits:

git rebase -i HEAD~[NUMBER OF COMMITS]

• Or if using SHA of start parent commit:

git rebase -i [SHA]

• Update development branch to match upstream

git checkout development
git pull upstream development

• rebase squashed feature branch commits onto upstream/development head
• No need to rebase interactive because we have already squashed as far as we want to

git checkout [name of new squashed feature branch]
git rebase development

Push to fork remote repo

• Double check still passes all tests after cleaning commit history
• push the cleaned up, squashed, rebased branch to fork repo remote

git checkout [name of new squashed feature branch]
git push origin [name of new squashed feature branch]

Submit PR

• Send PR from remote forked repo/[name of new squashed feature branch] >> central repo's development branch

Central repo maintainer

use services merge tool

• It seems the most common method to merge in pull requests is to use the source control’s (eg BitBucket's) Merge button, if trust the PR does not need to be tested can do this, merging into the development branch:
  • Merge strategy: Merge Commit
  • Use merge commit so there is always a merge commit, even if the PR was only one commit, which will be common. Helps show a feature was merged here and which commit/s belong to it

If want to review or change before merge

Seems to be no way to pull the PR directly from the central repo to local

Idea 1 - Clone the PR branch from the forked repo and test before accepting the PR

• ? Add the central repo development branch as upstream remote so can make sure PR branch is up to date

(... add commands here)

• ? Make a clone of central repo, create a new branch that tracks the forked repo PR branch, merge and test, if OK accept PR

(... add commands here)

Idea 2 - Merge and push from local rather than merge through source control

• ? clone/pull from forked repo PR branch into a new local branch
• Create new branch for PR branch

(... add commands here)

• Adding the forked repo as a remote for new branch

(... add commands here) create new branch

• pull into local PR branch

(... add commands here)

• merge into development branch

(... add commands here) -> prob same as above merge commands on developers local repo)

• Then can push to central repo > development branch

(... add commands here)

Notes

Merge PR using fast forward or merge no-ff

• fast forward will place the PR commit/s directly on the end of the branch with no marker this is a branch being merged in. no-ff will force a new merge commit
• Some people suggest should always fast forward so development branch is one clean line, others suggest you lose the grouping of commits in a feature doing this
• If the PR has been squashed into a single commit, can probably fast forward relatively cleanly, if there are multiple commits in the PR no-ff would group them together as a “branch and merge back in” in the development branch commit history
• Chosen plan:
  • Can always use merge commits (merge no-ff), even if one commit, so there is a clear branch in and out for each PR
  • But not too serious, if tiny little changes, a fast forward merge is acceptable

Commands to use if merging in a feature branch on local repo

• I don’t think works for PR’s directly
• fast forward merge (eg: single commit PR):

git checkout development
git merge [feature-branch]

• (should fast forward)
• no-ff merge (eg multi-commit PR):

git checkout development
git merge --no-ff [feature-branch]

• (maybe not if is a long lived feature branch) Should be no conflicts because PR from forked repo should be up to date with upstream (and maintainer’s local development branch should be a clean copy of central repo’s development branch)
• Once complete maintainer can push to central repo’s development branch

Protocol - After central repo updated

• All developers can pull in the central repos development branch changes #Protocol - When central repo has updates
• Developer who made the PR can delete that feature branch locally (after pulling upstream/development):

git branch -d <branch name>

• Delete fork repo branch on remote:

git push --delete origin <branch name>

• ? do we also need to delete the local reference to remote:

git branch -dr origin\<branch name>

• Update development in their forked repo if not already done (optional):

git push origin development

Protocol - Changes to PR before can merge into central repo

• Developer can fetch and merge any upstream changes to their local repo as normal, merge into their local feature branch, make changes and add new commits, then push to their forked repo
• Understand the original PR gets rejected, and a new PR is created with the changes? There might be a way to update PR, but maybe simpler to create a new one after cleaning up commit history like normal
• (test) - Danger of destructive/uncared for code changes if rebase central repo changes into feature branches over time?
• A rebase bumps the feature branches commits onto the end of new central repo changes merged into the feature branch during its development, that might change the context of the feature branches changes, causing issue?
• If we don’t rebase, do a normal merge (or no-ff merge) would that mess up merging the feature branch back into the central repo? Because there will be a mixed order of commits in the feature branch (central repo commits mixed into feature branch commits)
• My thoughts are it will be OK, the central repo branch will simply have a split chain of commits for the feature branch, joining into the final head as a merge commit (links to two parent commits)

Set VSCode as default editor

git config --global core.editor "code --wait"

See also

• https://www.atlassian.com/git/tutorials/comparing-workflows/forking-workflow
• https://www.atlassian.com/git/tutorials/git-forks-and-upstreams - Focus on making a single commit to the central repo when merging in a PR to reduce central repo commits, but that loses all history of commits made in development of the feature branch (squashed into one) - not sure I agree with this
• https://github.com/Kunena/Kunena-Forum/wiki/Create-a-new-branch-with-git-and-manage-branches
• https://github.com/KratosMultiphysics/Kratos/wiki/Style-Guide - Good overview of coding conventions
• https://github.blog/2015-06-08-how-to-undo-almost-anything-with-git/ - Good rundown of undo mistakes/commits