Loading...
 
Page actions
PDF Print Open in WebDAV

Git

1.1. Talks

2015-05-14 Xavier de Pedro - Enhance Team Work with DVCS - Git and Bazaar
http://ueb.vhir.org/SeminarDVCS

1.2. Cheatsheets

1.2.1. Git

Image Github Git Cheat Sheet

See also:
http://www.r-bloggers.com/rstudio-and-github/

1.3. My Online Repositories

1.3.1. Gitlab

https://gitlab.com/xavidp/

Gitlab allows public and private repos, plus self-hosting gitlab in your own server if you wish.

1.3.2. Github

https://github.com/xavidp/

Steps to add to Github.com some git-controlled code in your desktop.
First create a repository at Github.com with the same name of the local repository that you have in your local computer.
Then add the ssh public key from your computer at Github.com, if you haven't done so already.

Then these commands:

 
1
cd ~/scripts/TikiProfilesTester
2
git remote add origin https://github.com/xavidp/TikiProfilesTester.git
3
git remote set-url origin https://github.com/xavidp/TikiProfilesTester.git
4
git config remote.origin.url git@github.com:xavidp/TikiProfilesTester.git
5
git commit -m "Added a minor change" # optional, only if requried
6
git pull https://github.com/xavidp/TikiProfilesTester.git
7
git push --set-upstream origin master
8
git push -u
9
sed -i -e "s/url = https:\/\/github.com\/xavidp\//url = git@github.com:xavidp\//g" .git/config # this seems needed from work at VHIR, otherwise it doesn't succeed pushing through Firewall/Proxy.


1.4. Identify username and email

xxxxxxxxxx
 
git config --global user.name "Your Name"
git config --global user.email you@example.com

1.5. Rename some author name and email from all git history from a repo

Create a new bash file "git_rewrite_author.sh" with these contents:

​x
 
#!/bin/sh
​
git filter-branch --env-filter '
OLD_EMAIL="myoldemail@oldexample.com"
CORRECT_NAME="My real name"
CORRECT_EMAIL="myrealemail@newexample.com"
if [ "$GIT_COMMITTER_EMAIL" = "$OLD_EMAIL" ]
then
    export GIT_COMMITTER_NAME="$CORRECT_NAME"
    export GIT_COMMITTER_EMAIL="$CORRECT_EMAIL"
fi
if [ "$GIT_AUTHOR_EMAIL" = "$OLD_EMAIL" ]
then
    export GIT_AUTHOR_NAME="$CORRECT_NAME"
    export GIT_AUTHOR_EMAIL="$CORRECT_EMAIL"
fi
' --tag-name-filter cat -- --branches --tags


Then form a terminal at the git project root (the repository root) run this command: 

xxxxxxxxxx
 
bash ./git_rewrite_author.sh


If you are in a Windows computer where you don't have access to a terminal, you can run git commands from a bash terminal through the git-for-windows version of Git, run from a USB disk.
https://gitforwindows.org

1.6. Add your ssh public key to your gitlab/github accounts

Get a copy of the public ssh keys with commands like 

xxxxxxxxxx
 
# If you have rsa keys
cat ~/.ssh/id_rsa.pub
# If you have dsa keys
cat ~/.ssh/id_dsa.pub


And copy&paste in the right place of each website.

1.7. Get code from github

xxxxxxxxxx
 
# Get the url ending in .git from the repo you want to clone
git clone https://github.com/xavidp/bashscripts.git
git clone git@github.com:uebueb/bpmmirna.git
# edit your files
nano myfile.sh
# Commit your changes adding your editions to the new revision and providing a commit message
git commit -a -m "My changes info"
# Send your changes to the remote repo where you cloned your code from
git push

1.8. Publish on github your local git-controlled code

Steps to add to Github.com some git-controlled code in your desktop.
First create a repository at Github.com with the same name of the local repository that you have in your local computer.
Then add the ssh public key from your computer at Github.com, if you haven't done so already.

Then these commands: 

xxxxxxxxxx
11
 
1
cd ~/scripts/TikiProfilesTester
2
git remote add origin https://github.com/xavidp/TikiProfilesTester.git
3
git remote set-url origin https://github.com/xavidp/TikiProfilesTester.git
4
git config remote.origin.url git@github.com:xavidp/TikiProfilesTester.git
5
#git commit -m "Added a minor change" # optional, only if requried
6
git config --global push.default matching
7
git config --global user.name "xavidp"
8
sed -i -e "s/url = https:\/\/github.com\/xavidp\//url = git@github.com:xavidp\//g" .git/config # this seems needed from work at VHIR, otherwise it doesn't succeed pushing through Firewall/Proxy.
9
git pull https://github.com/xavidp/TikiProfilesTester.git
10
git push -u -f
11
git reset HEAD *

1.9. Bash script to fix these things for your user and repo

See:
https://github.com/xavidp/bashscripts/blob/master/fixgitueb.sh

1.10. Convert between Git and Bazaar

http://ueb.vhir.org/Bazaar#Convert_between_Git_and_Bazaar_and_others_

1.11. Use your own open-source github-like webapp for your code

See:

1.12. FLOSS Git GUIs

git-cola

xxxxxxxxxx
 
sudo apt-get install git-cola

See: http://git-cola.github.io/downloads.html

git-gui

xxxxxxxxxx
 
sudo apt-get install git-gui

See: https://www.maketecheasier.com/6-useful-graphical-git-client-for-linux/

GitKraken

FLOSS (?) Crossplatform Git GUI
See: https://www.gitkraken.com/

1.13. Git AjBCN (with portable apps)

At Aj.BCN you can install Git on Windows in a USB disk with portable apps:
https://portableapps.com

And install there also GitforWindows:
https://gitforwindows.org

1.13.1. GitForWindows

Git for Windows focuses on offering a lightweight, native set of tools that bring the full feature set of the Git SCM to Windows while providing appropriate user interfaces for experienced Git users and novices alike.

Git BASH

Git for Windows provides a BASH emulation used to run Git from the command line. *NIX users should feel right at home, as the BASH emulation behaves just like the "git" command in LINUX and UNIX environments.

Git GUI

As Windows users commonly expect graphical user interfaces, Git for Windows also provides the Git GUI, a powerful alternative to Git BASH, offering a graphical version of just about every Git command line function, as well as comprehensive visual diff tools.

Shell Integration

Simply right-click on a folder in Windows Explorer to access the BASH or GUI.

1.13.2. RStudio + Git

You can use Git from RStudio portable version also.

1.13.3. Rewrite author and email from all your commits locally

The computer from AjBCN uses by default your windows name (in my case, my Surnames in uppercase) and the InternalUserId@bcn.cat as your email, which is likely to not match at all with your github username and email.
Therefore, you might want to fix your user name and email so that they can show up in your github contributions graphs and stats, etc.

I fixed locally this information running the script below in a Git BASH window openend from the root folder of the git-controlled project in your computer (go with Windows explorer to that folder, right click and select Git Bash Here ).
Make a file with this contents in that folder:

Contents of gitrewriteauthor.sh
xxxxxxxxxx
17
 
1
#!/bin/sh
2
​
3
git filter-branch --env-filter '
4
OLD_EMAIL="myoldemail@oldexample.com"
5
CORRECT_NAME="Foo Bar"
6
CORRECT_EMAIL="mynewemail@oldexample.com"
7
if [ "$GIT_COMMITTER_EMAIL" = "$OLD_EMAIL" ]
8
then
9
    export GIT_COMMITTER_NAME="$CORRECT_NAME"
10
    export GIT_COMMITTER_EMAIL="$CORRECT_EMAIL"
11
fi
12
if [ "$GIT_AUTHOR_EMAIL" = "$OLD_EMAIL" ]
13
then
14
    export GIT_AUTHOR_NAME="$CORRECT_NAME"
15
    export GIT_AUTHOR_EMAIL="$CORRECT_EMAIL"
16
fi
17
' --tag-name-filter cat -- --branches --tags


Then run this gitrewriteauthor.sh bash script there

1.13.4. Generate your ssh key pair (if needed still)

Open a Git Bash terminal window and run the command to generate your new ssh Key pair.
Follow the instructions in github to generate your key pair.
See:


Copy the contents of your local public key into the corresponding place of your profile account in github.com
See:

1.13.5. Commit from Local to Github

Code to commit from local to Github, after you create the branch with equivalent name in github as usual, etc (see above for further generic instructions). 

xxxxxxxxxx
11
 
1
cd ~/yourpath/SAIER_Urgencies
2
git remote add origin https://github.com/xavidp/SAIER_Urgencies.git
3
git remote set-url origin https://github.com/xavidp/SAIER_Urgencies.git
4
git config remote.origin.url git@github.com:xavidp/SAIER_Urgencies.git
5
#git commit -m "Added a minor change" # optional, only if requried
6
git config --global push.default matching
7
git config --global user.name "xavidp"
8
sed -i -e "s/url = https:\/\/github.com\/xavidp\//url = git@github.com:xavidp\//g" .git/config # this seems needed from work at VHIR, otherwise it doesn't succeed pushing through Firewall/Proxy.
9
git pull https://github.com/xavidp/SAIER_Urgencies.git
10
git push -u -f
11
git reset HEAD *

1.13.6. Fix permanently your user and email for this project

You can add some new parameters to the .git/config file under your project folder. This will stick to these new user and email values for this specific folder, without touching the other git projects in your computer.
Therefore, open a new Git Bash terminal in your project folder, and then move the .git subfolder and open the config file with the nano editor: 

xxxxxxxxxx
 
cd .git
nano config

Click to expand
Click to expand


Then add this section at the top of the file: 

xxxxxxxxxx
4
 
1
[user]
2
        name = YourName And Surnames
3
        email = youremail@bcn.cat
4
        username = yourusername


You can copy from here or elsewhere and paste through right-cliking on the terminal window to select Paste (Shit + Ins, in Git Bash terminal windows), so that after that it looks like:

Click to expand
Click to expand


and press Ctr + x later on to save the file and exit the nano editor in the same go.

From now on, when you commit changes from RStudio (or other git based programs and workflows), you will get this name, email and usersname linked to those commits.

1.13.7. Push through Git BASH

Pushing from RStudio doesn't seem to succeed getting the right ssh keys to push your changes automagically to github. Therefore, you'd better open a Git Bash terminal window in your project root folder, and push from there with a standard command to do so: 

xxxxxxxxxx
 
git push

1.14. Gitlab prune big-files from project

Reference: 

xxxxxxxxxx
211
 
1
xavi@tricholoma:~/tmp$ git clone --mirror https://gitlab.com/radup/curs-r-introduccio.git
2
S'està clonant al dipòsit nu «curs-r-introduccio.git»...
3
remote: Enumerating objects: 311, done.
4
remote: Counting objects: 100% (311/311), done.
5
remote: Compressing objects: 100% (165/165), done.
6
remote: Total 854 (delta 174), reused 276 (delta 145)
7
S'estan rebent objectes: 100% (854/854), 383.42 MiB | 714.00 KiB/s, fet.
8
S'estan resolent les diferències: 100% (494/494), fet.
9
xavi@tricholoma:~/tmp$ java -jar ../bfg-1.13.0.jar --strip-blobs-bigger-than 22Mcurs-r-introduccio.git
10
Error: Unable to access jarfile ../bfg-1.13.0.jar
11
xavi@tricholoma:~/tmp$ java -jar ../bfg-1.13.0.jar --strip-blobs-bigger-than 22Mcurs-r-introduccio.git
12
Error: Unable to access jarfile ../bfg-1.13.0.jar
13
xavi@tricholoma:~/tmp$ cp ../bfg-1.13.0.jar .
14
cp: ha fallat stat() sobre '../bfg-1.13.0.jar': El fitxer o directori no existeix
15
xavi@tricholoma:~/tmp$ java -jar bfg-1.13.0.jar --strip-blobs-bigger-than 22M curs-r-introduccio.git
16
​
17
Using repo : /media/xavi/Maxtor/XAVI_01/tmp/curs-r-introduccio.git
18
​
19
Scanning packfile for large blobs: 854
20
Scanning packfile for large blobs completed in 374 ms.
21
Found 7 blob ids for large blobs - biggest=89772799 smallest=24005151
22
Total size (unpacked)=433099439
23
Found 131 objects to protect
24
Found 2 commit-pointing refs : HEAD, refs/heads/master
25
​
26
Protected commits
27
-----------------
28
​
29
These are your protected commits, and so their contents will NOT be altered:
30
​
31
 * commit 7b14cd56 (protected by 'HEAD')
32
​
33
Cleaning
34
--------
35
​
36
Found 166 commits
37
Cleaning commits:       100% (166/166)
38
Cleaning commits completed in 263 ms.
39
​
40
Updating 1 Ref
41
--------------
42
​
43
    Ref                 Before     After  
44
    ---------------------------------------
45
    refs/heads/master | 7b14cd56 | fc566573
46
​
47
Updating references:    100% (1/1)
48
...Ref update completed in 19 ms.
49
​
50
Commit Tree-Dirt History
51
------------------------
52
​
53
    Earliest                                              Latest
54
    |                                                          |
55
    ..................................................DDmmmmmmmm
56
​
57
    D = dirty commits (file tree fixed)
58
    m = modified commits (commit message or parents changed)
59
    . = clean commits (no changes to file tree)
60
​
61
                            Before     After  
62
    -------------------------------------------
63
    First modified commit | aa7b8697 | ff4173a2
64
    Last dirty commit     | aa8623c2 | 051e8b60
65
​
66
Deleted files
67
-------------
68
​
69
    Filename              Git id            
70
    ----------------------------------------
71
    arxiu               | e48cc445 (75,9 MB)
72
    arxiu.50.fst        | 9488b9cf (28,6 MB)
73
    arxiu.base.csv      | 917953d0 (85,6 MB)
74
    arxiu.base.csv.gz   | a9f3b581 (22,9 MB)
75
    arxiu.datatable.csv | e48cc445 (75,9 MB)
76
    arxiu.feather       | c53071c0 (60,4 MB)
77
    arxiu.tidyverse.csv | b9e892f8 (75,9 MB)
78
    arxiu.xlsx          | cfdd5af5 (63,7 MB)
79
​
80
​
81
In total, 30 object ids were changed. Full details are logged here:
82
​
83
    /media/xavi/Maxtor/XAVI_01/tmp/curs-r-introduccio.git.bfg-report/2019-03-19/01-52-58
84
​
85
BFG run is complete! When ready, run: git reflog expire --expire=now --all && git gc --prune=now --aggressive
86
​
87
​
88
--
89
You can rewrite history in Git - don't let Trump do it for real!
90
Trump's administration has lied consistently, to make people give up on ever
91
being told the truth. Don't give up: https://github.com/bkeepers/stop-trump
92
--
93
​
94
​
95
xavi@tricholoma:~/tmp$ java -jar bfg-1.13.0.jar --delete-files arxiu.gz curs-r-introduccio.git
96
​
97
Using repo : /media/xavi/Maxtor/XAVI_01/tmp/curs-r-introduccio.git
98
​
99
​
100
This repo has been processed by The BFG before! Will prune repo before proceeding - to avoid unnecessary cleaning work on unused objects...
101
Completed prune of old objects - will now proceed with the main job!
102
​
103
Found 131 objects to protect
104
Found 2 commit-pointing refs : HEAD, refs/heads/master
105
​
106
Protected commits
107
-----------------
108
​
109
These are your protected commits, and so their contents will NOT be altered:
110
​
111
 * commit fc566573 (protected by 'HEAD')
112
​
113
Cleaning
114
--------
115
​
116
Found 166 commits
117
Cleaning commits:       100% (166/166)
118
Cleaning commits completed in 603 ms.
119
​
120
Updating 1 Ref
121
--------------
122
​
123
    Ref                 Before     After  
124
    ---------------------------------------
125
    refs/heads/master | fc566573 | 3abf839a
126
​
127
Updating references:    100% (1/1)
128
...Ref update completed in 6 ms.
129
​
130
Commit Tree-Dirt History
131
------------------------
132
​
133
    Earliest                                              Latest
134
    |                                                          |
135
    ..................................................DDmmmmmmmm
136
​
137
    D = dirty commits (file tree fixed)
138
    m = modified commits (commit message or parents changed)
139
    . = clean commits (no changes to file tree)
140
​
141
                            Before     After  
142
    -------------------------------------------
143
    First modified commit | ff4173a2 | 4d1f5d89
144
    Last dirty commit     | 051e8b60 | 903d8c77
145
​
146
Deleted files
147
-------------
148
​
149
    Filename   Git id            
150
    -----------------------------
151
    arxiu.gz | bd2f68cd (20,1 MB)
152
​
153
​
154
In total, 60 object ids were changed. Full details are logged here:
155
​
156
    /media/xavi/Maxtor/XAVI_01/tmp/curs-r-introduccio.git.bfg-report/2019-03-19/01-55-53
157
​
158
BFG run is complete! When ready, run: git reflog expire --expire=now --all && git gc --prune=now --aggressive
159
​
160
​
161
--
162
You can rewrite history in Git - don't let Trump do it for real!
163
Trump's administration has lied consistently, to make people give up on ever
164
being told the truth. Don't give up: https://www.aclu.org/
165
--
166
​
167
​
168
xavi@tricholoma:~/tmp$ cd curs-r-introduccio.git
169
xavi@tricholoma:~/tmp/curs-r-introduccio.git$ git reflog expire --expire=now --all && git gc --prune=now --aggressive
170
S'estan comptant els objectes: 854, fet.
171
Delta compression using up to 4 threads.
172
S'estan comprimint objectes: 100% (840/840), fet.
173
S'estan escrivint els objectes: 100% (854/854), fet.
174
Total 854 (delta 525), reused 299 (delta 0)
175
xavi@tricholoma:~/tmp/curs-r-introduccio.git$ git push
176
Username for 'https://gitlab.com': xavidp
177
Password for 'https://xavidp@gitlab.com':
178
S'estan comptant els objectes: 854, fet.
179
Delta compression using up to 4 threads.
180
S'estan comprimint objectes: 100% (315/315), fet.
181
S'estan escrivint els objectes: 100% (854/854), 205.11 MiB | 1.44 MiB/s, fet.
182
Total 854 (delta 525), reused 854 (delta 525)
183
remote: Resolving deltas: 100% (525/525), done.
184
remote: GitLab: You are not allowed to force push code to a protected branch on this project.
185
To https://gitlab.com/radup/curs-r-introduccio.git
186
 ! [remote rejected] master -> master (pre-receive hook declined)
187
error: s'ha produït un error en pujar algunes referències a «https://gitlab.com/radup/curs-r-introduccio.git»
188
xavi@tricholoma:~/tmp/curs-r-introduccio.git$ git push
189
Username for 'https://gitlab.com': xavidp
190
Password for 'https://xavidp@gitlab.com':
191
S'estan comptant els objectes: 854, fet.
192
Delta compression using up to 4 threads.
193
S'estan comprimint objectes: 100% (315/315), fet.
194
S'estan escrivint els objectes: 100% (854/854), 205.11 MiB | 1.44 MiB/s, fet.
195
Total 854 (delta 525), reused 854 (delta 525)
196
remote: Resolving deltas: 100% (525/525), done.
197
To https://gitlab.com/radup/curs-r-introduccio.git
198
 + 7b14cd5...3abf839 master -> master (forced update)
199
xavi@tricholoma:~/tmp/curs-r-introduccio.git$ cd ..
200
xavi@tricholoma:~/tmp$ cd ..
201
xavi@tricholoma:~$ cd tmp2
202
xavi@tricholoma:~/tmp2$ ls
203
xavi@tricholoma:~/tmp2$ git clone https://gitlab.com/radup/curs-r-introduccio.git
204
S'està clonant a «curs-r-introduccio»...
205
remote: Enumerating objects: 311, done.
206
remote: Counting objects: 100% (311/311), done.
207
remote: Compressing objects: 100% (125/125), done.
208
remote: Total 854 (delta 179), reused 306 (delta 178)
209
S'estan rebent objectes: 100% (854/854), 205.47 MiB | 638.00 KiB/s, fet.
210
S'estan resolent les diferències: 100% (499/499), fet.
211
xavi@tricholoma:~/tmp2$


And after that, you need to Navigate to Project > Settings > Repository > Repository Cleanup:

Image

Upload the object-id-map.old-new.txt file and press Start cleanup. This will remove any internal git references to the old commits, and run git gc against the repository. You will receive an email once it has completed

1.15. Change origin from https to ssh

Verify origin of your repo through a terminal window in the root folder where your project code is located: 

xxxxxxxxxx
 
git remote -v


It should say something like: 

xxxxxxxxxx
 
origin  http://gitlab.com/{USERNAME}/{PROJECTNAME}.git (fetch)
origin  http://gitlab.com/{USERNAME}/{PROJECTNAME}.git (push)


Then you can run in your terminal window the following command 

xxxxxxxxxx
 
git remote set-url origin git@gitlab.com:{USERNAME}/{PROJECTNAME}.git


And last, you can check again that the change was successful, with: 

xxxxxxxxxx
 
git remote -v



Alias names for this page: github | gitlab

Comments
Image Seed: noun \ˈsēd\ : the beginning of something which continues to develop or grow

Knowledge seeds

Switch Language