I recently found out about gitleaks. It's an open source tool that detects leaked secrets. If you just want to try it locally, it's a simple command:

gitleaks detect --verbose

Sometimes you'll get some false positives. To exclude files or commits from check, you will add an allowlist . Such example is given here https://github.com/gitleaks/gitleaks/blob/master/.gitleaks.toml. 

title = "Custom Gitleaks configuration"

[extend]
useDefault = true

[[allowlists]]
paths = [
    "path_to_your_file"
]
commits = [ "commitsha"]

.gitleaks.toml

For example, for a react-native app, google service files should go into allowlist if you want to version those files. Technically, those files are public, they are needed by your app.

Gitleaks found leaks, now what?

Gitleaks will scan your entire repository, all your commits and find problematic files and reference to commits.

What you need to do is rotate your secrets and remove problematic file from all your commits.To find out commit sha's, just do a log search:

git fetch --all
git log --all -- leakedfile.js

If anything is logged, the only solution is to rotate the secret. You can remove a file from git commit history, but it won't be removed on GitHub.

How do you remove a file from git commit history

Be careful when you do this because this action rewrites history and all collaborators will need to reset to new history. There's a useful discussion in https://stackoverflow.com/questions/43762338/how-can-i-remove-file-from-git-history so you can see different approaches. I prefer to use a git filter-repo.

First, clone your git repository to a new folder:

git clone <repo.git> <dest>

Install git filter-repo

git filter-repo --path <path_to_leaked_file> --invert-paths

You'll lose your origin once it's done, re-add it.

git remote add origin <your repo.git>
git push --force --all                                    
git push --force --tags

Ok, now delete your folder and git clone it again.

Do a log search.

 git log --all -- ".env"

No results is what you want.

You can't remove it from GitHub

I want you to open the leaked file in a web browser, It will be something like <https://github.com/owner/repo/blob/commitsha/leakedfile>.

And you'll get that exact file, not a 404 not found.

You successfully removed the problematic file from all your commits, and yet you can still open the old commit in Github. How is that possible?

Force push creates loose commits. GitHub will not garbage collect loose commits. There is no api to do it yourself.

I recommend that you read https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository.

If you can't rotate the secret, then only way is to contact GitHub support and request removal. You will have to be specific, which file in which commit sha-s.

I think I read that there is also an option to nuke your repo. I haven't tried it myself and I'm really skeptical of that option.

Prevention

Pre commit hook is the safest thing and also CI check for pushed commits.

Example of such hook, put in root of your repo:

repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.30.0
    hooks:
      - id: gitleaks

.pre-commit-config.yaml

Activate with pre-commit:

brew install pre-commit
pre-commit install
pre-commit run --all-files

Next time you try to commit a sensitive file, pre-commit hook will run and notify if the commit contains sensitive data.

Youtube: https://youtu.be/Rrs-CAolDKY

If you want to keep sensitive configuration in your repo, you can encrypt it with sops.

#Security #GitHub