Scott Berrevoets - Developer Tools

How GitHub handles Git LFS

2026-07-01T00:00:00-07:00

Snapshot tests have become an increasingly popular tool for mobile teams to verify UI changes, but one question all teams run into is: where do we store hundreds if not thousands of PNG files? Git Large File Storage (LFS) is one of the more common ways to do this for various reasons¹.

At Speak we recently went down this same path and in doing so learned a few things about how Git LFS works and how GitHub handles it. Initially we just learned the basic premise: instead of checking in a file, check in a small human-readable pointer file that identifies the large file:

version https://git-lfs.github.com/spec/v1
oid sha256:907b5c652cce59e009e3c7fb2dc92d3bf598251315bed741aee037f2046bd32e
size 251222

The oid is an object ID for the file that's fetched from the configured LFS server. Use git lfs track ./Snapshots/*.png to specify snapshots should be stored in LFS and then commit .gitattributes to preserve that setting.

But soon we started getting warnings from GitHub that we were exceeding our Git LFS budget. In investigating, we learned five more things that formed our mental model around how to think about Git LFS when using GitHub:

GitHub charges for storage and bandwidth: similar to S3, both storage and bandwidth cost money. Our snapshots don't take up too much storage but CI clones the iOS and Android repos for every build, which is when all PNGs are downloaded, incurring bandwidth usage.
You can skip downloading certain LFS objects to avoid bandwidth usage: to avoid this, Git LFS offers a clone option to avoid downloading large files when they aren't needed through git clone --config lfs.fetchexclude="./Snapshots/*.png". This saves bandwidth, clone time, and local storage. git lfs pull --include="./Snapshots/*.png" lets you redownload them.
Removing a tracked file only removes it from git, not from storage: removing the pointer files that get checked in and committing that change does nothing to the actual object in storage. Removing the file from git leaves a dangling object that still takes up storage space.
Fully removing references requires rewriting history: removing a file from git isn't sufficient because older commits may still refer to an LFS file and fail future clones. Fully purging it requires rewriting the git history using git filter-repo, which is obviously not ideal but arguably still better than irreversibly not being able to check out certain commits in the repo ever again.
Rewriting history doesn't delete the file from storage: even after all references to a file are deleted, the LFS object itself still needs to be deleted as well for storage to be freed up. The only way to do this other than to contact support, is to delete the repo (and all its associated data) and recreate it.

All of these details make it clear that Git LFS isn't a regular git repository that handles large files better; there's a sophisticated cloud storage layer behind it that needs the same considerations that more mainstream cloud storage does. Although git does the version control of those files, it doesn't touch the raw objects itself.

Checking in large binary files that change regularly causes repo bloat, and cloud storage solutions like S3 or GCS make for a more complicated setup. ↩

Scaling my git/tmux workflows for AI agents

2026-02-17T00:00:00-08:00

After I started to ramp up on agentic coding, I found my workflow lagging behind how I ended up working. Previously I would clone my project's repo twice and switch back and forth between the clones. I used stack-based git workflow git-pile, which eliminated the overhead of switching branches and stashing changes.

When I was occasionally in the middle of something and needed to context switch and have a clean repo, I would have the other clone and that was about as much as I could handle at a time. I knew of git worktrees but never needed it so didn't bother upgrading my workflow.

In the age of AI I found myself needing a lot more. Understanding various parts of the codebase is one prompt away and making changes requires a much smaller context switch than before so I do it much more frequently. However, the two clones weren't really sufficient for this anymore, as it became more painful to sync changes back and forth before they landed on main.

So I had Claude design me a new workflow that scaled better and allowed more parallelization. I wanted:

one repo with n worktrees with no prior set up required
tmux integration so each worktree gets its own tmux window
no dangling worktrees and branches if I abandon work
compatibility with git-pile, but no direct integration
packaged in an easy to use CLI with powerful commands that capture a lot of complexity

I wanted to use worktrees under the hood, but also have that be an implementation detail as I'd prefer to not deal with them directly.

The end result is a Python script I called agents that has 6 commands:

new [name] - create a new worktree
list - lists all existing tasks/worktrees
merge - merge the current changes into the parent repo on main
rm - delete this worktree and close its tmux window
open - create a tmux window with the specified worktree (in case it was closed)
clean - delete all stale worktrees and branches, and closes their tmux windows

This provides a very simple workflow:

Bash# Initialize worktree
# No subcommand is shorthand for `new`
agents find-bug

# Run some agents
codex exec "fix the race condition in DataManager"

# ...
# switch to another agent/tmux window, do some work there, then return
# ...

# Codex output looks good - let's upstream (or have the agent do it)
git add . && git commit -m "Fix race condition"

# Merge commit back to main repo
agents merge

# Delete worktree and close tmux window
agents rm

The main branch on my repo clone now has this commit, and I can use git-pile to submit a pull request with this change. This workflow works in any git repo out of the box, no special setup required.

While Codex is running, I can switch back to the previous tmux window and start another agent, or I can even branch off that particular worktree into a new one. I'm not sure I want that from the workflow since I prefer working on the repo's main branch in the first place, so time will tell if I end up using that branching behavior.

Ideally these worktrees are shortlived and will just get their work merged in. However, there are situations where there's longer running work or work I end up not moving forward with, and for that I had Claude implement an agents clean command. It deletes all stale worktrees, with some protections for unmerged work to avoid losing uncommitted changes.

To understand what work is in progress, agents list gives a nicely-formatted output that's piped to fzf for easy selection:

The arrows indicate commit status, which is helpful to understand if any worktrees may need rebasing.

After building and working this tool a bit I realized the workflow works even when not using an agentic programming workflow, so agents is probably not the best name, but since I developed it with that use case in mind I'll probably continue to use that for now.

The full script is in my dotfiles' ./bin folder, which is automatically accessible anywhere from my Terminal. I've also written a Claude/Codex skill to run this automatically for me in case I forget to do so myself, and I'll probably keep tweaking the workflow to my liking as I work with it more.

Composing a resume in markdown

2025-03-15T00:00:00-07:00

I recently updated my resume, and after a few iterations of tweaking the same Pages file I started more than a decade ago, I decided that I really wanted source control to make it easier to keep track of changes and to have different versions of the same resume. Additionally, the very consistent structure and simplicity of a resume makes it a perfect candidate to be written in markdown, which is a text-based format and easy to have in source control.

Some quick searching confirmed my suspicion that many people were already doing this and that there are various tools available to make this easier. The primary tool is pandoc, which is a universal document converter. It can convert between many different formats, including markdown and PDF.

Note

The other tool that makes for a very structured resume is JSON Resume. It's a JSON schema with standard fields for common resume sections like 'experience' and 'education' that can then easily be themed. I found this a little too limiting and not as readable in plain text as markdown.

In my case, I'm converting from markdown to HTML, injecting some CSS to make it look the way I want it to, and then converting that to PDF. LaTeX was another option but I wasn't familiar with either LaTeX or ConTeXt and didn't want to bother learning it just for this exercise.

Converting to HTML was generally easy.

Bashpandoc --standalone --include-in-header resume.css --output resume.html resume.md

This gives a pretty good starting point even with no CSS at all. pandoc uses a custom markdown dialect with some extensions and other features, which makes it easy to add ids and classes to <h1>-<h6> elements. This is useful as it makes applying the right style to the right element easy. The resume itself is simple and so is the resulting HTML. pandoc does inject its own CSS as well, which is easy to override in the custom CSS file.

After tweaking the CSS to my liking, the next step was to convert the HTML to PDF format. This was a bit more complicated as there were multiple ways to do that:

Safari (or another browser): There is an export as PDF option or the option to print and then save as PDF. This worked but required manual steps I wanted to automate.
wkhtmltopdf: a command line tool that converts HTML to PDF using a WebKit engine
pandoc: can convert to PDF directly with some options that, supposedly, affect the final PDF output. I tried using it with wkhtmltopdf as its engine, and it can also use LaTeX or ConTeXt as an intermediate format that bypasses HTML generation altogether.

I struggled a bit with which to choose, as the format for all of them wasn't quite right except for the print to PDF option in Safari. I ultimately settled on using wkhtmltopdf directly (without using pandoc for this step), after experimenting with these options and seeing which output I liked best. Converting the HTML to PDF was as simple as:

Bashwkhtmltopdf -s letter resume.html resume.pdf

The -s letter option sets the paper size to US Letter since the default is A4. Nowadays it doesn't really matter as resumes are only ever used in digital format so the size of the PDF doesn't really matter anyway.

I then wrapped both commands in a simple shell script to make it easier to run:

Bashpandoc \
    --standalone \
    --include-in-header resume.css \
    --output resume.html \
    resume.md

wkhtmltopdf -s letter resume.html resume.pdf

open resume.pdf && open resume.html

After some modifications to the markdown, running this script will quickly generate the HTML and PDF files and open them both so I can validate the result.

This setup ends up working pretty well for me for 3 reasons:

With everything in a git repo I can make changes as I like, undo them easily, and use all of git's features on my resume. I push the markdown, HTML, and PDF versions to GitHub so I always have the latest version handy in a few different formats.
The format and the content are for the most part separate, making it easier to focus on one or the other.
Since everything is now text-based and not in proprietary binary formats, it's also easy to overengineer this in the future by including certain sections when applying to different jobs, including certain information only when certain flags are set, differentiating between a CV and a resume, etc.

Looking up words in a dictionary with Neovim

2024-12-08T00:00:00-08:00

When writing code in neovim, I frequently use K to look up documentation, function signatures, variable definitions, etc. In markdown files that doesn't make too much sense, but I wanted to use it to look up words in the dictionary instead. This didn't appear to be possible out of the box but the power of (neo)vim is its extensibility so I decided to build it myself.

The steps to build this roughly were:

Write a dictionary lookup tool
Parse this tool's output and show it in neovim
Configure neovim's K functionality

`dict`

dict seemed like an appropriate, easy name to look up a word in the dictionary. As I'm a macOS user I first wanted to use the built-in dictionary through pyobjc-framework-CoreServices but DCSCopyTextDefinition doesn't return dictionary entries in a well-structured or consistent way, so I ultimately decided to write a quick lookup using dictionaryapi.dev. I also added some other formatting and ANSI coloring to improve the CLI output:

Python#!/usr/bin/env python3

import requests
import argparse

_TERMINAL_RESET = "\033[0m"
_TERMINAL_BOLD = "\033[1m"
_TERMINAL_DIM = "\033[2m"
_TERMINAL_RED = "\033[91m"
_TERMINAL_GREEN = "\033[92m"
_TERMINAL_BLUE = "\033[94m"


def _fetch_word_definition(word):
    url = f"https://api.dictionaryapi.dev/api/v2/entries/en/{word}"
    response = requests.get(url)

    # Handle API errors
    if response.status_code != 200:
        return None

    return response.json()


def _format_definition_output(word_data):
    if not word_data:
        return f"{_TERMINAL_RED}error: word not found{_TERMINAL_RESET}"

    word = word_data[0]["word"]
    meanings = word_data[0].get("meanings", [])

    output = f"{_TERMINAL_GREEN}{_TERMINAL_BOLD}{word}{_TERMINAL_RESET}\n"

    for part_of_speech in meanings:
        pos = part_of_speech["partOfSpeech"]
        output += f"\n{_TERMINAL_BLUE}{pos}{_TERMINAL_RESET}"

        synonyms = ", ".join(part_of_speech["synonyms"])
        output += (
            f"  {_TERMINAL_DIM}({synonyms}){_TERMINAL_RESET}"
            if synonyms
            else ""
        )
        output += "\n"

        for i, definition in enumerate(part_of_speech["definitions"], 1):
            definition_text = definition.get("definition", "N/A")
            example = definition.get("example", "")
            output += f"  {i}. {definition_text}"
            if example:
                output += f" (e.g., {example})"
            output += "\n"

    return output


def _get_word_definition(word):
    word_data = _fetch_word_definition(word)
    return _format_definition_output(word_data)


def main():
    parser = argparse.ArgumentParser(
        description="Fetch word definitions from API dictionary.dev."
    )
    parser.add_argument("word", help="Word to look up")
    args = parser.parse_args()

    word = args.word
    print(_get_word_definition(word))


if __name__ == "__main__":
    main()

This file needs to be in the PATH which I did by putting it in ~/.bin/dict.py and setting export PATH="$HOME/.bin:$PATH".

dict ambiguous now gives a nicely formatted definition:

Neovim integration

I wanted to invoke dict using K passing in the word under the cursor as the trigger and showing the output in a popup window. Other than the plain vim configuration this required some text manipulation so I put all that in ~/.vim/plugin/dictionary.lua:

Lua-- Create a new buffer with the given message and apply ANSI highlighting
local function create_buf(message)
  local buf = vim.api.nvim_create_buf(false, true)

  local lines = vim.split(message, "\n", { plain = true })
  for i, line in ipairs(lines) do
    local stripped = line:gsub("\27%[[0-9;]+m", "")
    vim.api.nvim_buf_set_lines(buf, i - 1, i, false, { stripped })
    apply_ansi_highlighting(buf, i - 1, line)
  end

  return buf
end

-- Show a popup window with the given buffer
local function show_popup(buf)
  vim.g.popup_window = vim.api.nvim_open_win(buf, false, {
    relative = "cursor",
    width = 100,
    height = 15,
    row = 1,
    col = 0,
    anchor = "NW",
    style = "minimal",
    border = "single",
  })
end

-- Close the popup window
function ClosePopup()
  local window = vim.g.popup_window
  if vim.g.popup_window and vim.api.nvim_win_is_valid(window) then
    vim.api.nvim_win_close(window, true)
    vim.g.popup_window = nil
  end
end

-- Look up the definition of the word under the cursor
function DictionaryLookup()
  local word = vim.fn.expand("<cword>")
  local file = assert(io.popen("dict " .. word, "r"))
  local definition = assert(file:read("*a"))
  file:close()
  show_popup(create_buf(definition))
end

This creates 2 functions, DictionaryLookup() and ClosePopup() that show and dismiss the popup, stripping the ANSI codes from the output as they were rendered raw in the popup.

However, note the call to apply_ansi_highlighting(buf, i - 1, line) near the end of create_buf(), which replaces the ANSI codes with neovim specific highlight groups. This turned out to be quite involved but I wanted the formatted output and used this as an opportunity to learn lua a bit more.

Luavim.api.nvim_set_hl(0, "Normal", { fg = "fg", bold = false, underline = false, italic = false })
vim.api.nvim_set_hl(0, "Bold", { bold = true, underline = true })
vim.api.nvim_set_hl(0, "Dim", { fg = "gray", italic = true })
vim.api.nvim_set_hl(0, "Red", { fg = "red" })
vim.api.nvim_set_hl(0, "Green", { fg = "green" })
vim.api.nvim_set_hl(0, "Blue", { fg = "blue" })

local ansi_patterns = {
  { pattern = "\27%[0m", highlight = "Normal" },
  { pattern = "\27%[1m", highlight = "Bold" },
  { pattern = "\27%[2m", highlight = "Dim" },
  { pattern = "\27%[91m", highlight = "Red" },
  { pattern = "\27%[92m", highlight = "Green" },
  { pattern = "\27%[94m", highlight = "Blue" },
}

-- Find the highlight group for a given ANSI escape code
local function find_ansi_pattern(text)
  for _, entry in pairs(ansi_patterns) do
    if text:find(entry.pattern) then
      return entry.highlight
    end
  end
  return nil
end

-- Apply ANSI highlighting to a line of text in the given buffer
local function apply_ansi_highlighting(buf, line_num, text)
  text = text:gsub("\27%[0m$", "")
  local pattern = "\27%[[0-9]+m"

  local start_pos, end_pos, match = text:find("(" .. pattern .. ")")
  local cursor = 1
  local pos_in_stripped = 0

  local next = text
  while start_pos do
    if next:find("^" .. pattern) then
      start_pos, end_pos, match = next:find("(" .. pattern .. ")")
      cursor = end_pos + 1
      vim.api.nvim_buf_add_highlight(
        buf,
        -1,
        find_ansi_pattern(match) or "Normal",
        line_num,
        pos_in_stripped,
        -1
      )
    else
      start_pos = next:find(pattern)
      if start_pos then
        pos_in_stripped = pos_in_stripped + start_pos - 1
        cursor = start_pos
      end
    end

    next = next:sub(cursor, -1)
  end
end

Configuring `K`

Opening any file and calling :lua DictionaryLookup() already works (and :lua ClosePopup() to dismiss it) but that's a lot of typing and it better fits K in markdown files since that's otherwise unused anyway.

I mapped the first to a vim command :Dict and wanted to hide the window with any cursor movement, similar to how this works for other documentation lookups. keywordprg is the command that gets invoked through K (and can also be set through set keywordprg).

I put this in ~/.vim/ftplugin/markdown.lua so that it automatically is applied to markdown files:

Luavim.api.nvim_create_user_command("Dict", DictionaryLookup, { nargs = 1 })
vim.opt["keywordprg"] = ":Dict"

vim.api.nvim_create_autocmd("CursorMoved", {
  pattern = "*",
  callback = ClosePopup,
})

With the end result being:

Dictionary lookup also still works in non-markdown files by directly invoking :Dict, which is a bit more typing but also much less common. That's all that was needed, now K performs a dictionary lookup and shows it nicely formatted in-line and is automatically hidden when needed. This may be nice as a generic plugin as well so I might do that at some point in the future as well.