GitHub Developer Workflow

This page is the step-by-step version of the developer cycle summarized in Managing Projects with GitHub. It assumes the one-time setup is complete: protected main and next branches, connected Default projects, and a developer project of your own with its isolation Variables set.

Every operation you perform below appears as a row on your project's Imports tab — that is where you confirm each step landed, and where the log lives when one fails.

The cycle at a glance

1. Create a feature branch from next, point your project at it, and Initial Pull to baseline.
2. Build and test in your project.
3. Push, and open a pull request into next.
4. After the merge, Pull next into the Default Dev project and test the integrated result.
5. Promote with a pull request from next into main, then Pull main into the Default Prod project.

Rule of thumb: use an Initial operation when connecting a project to a branch (first sync, or any branch switch); use the regular operations for every sync after that. What Push and Pull do has the full semantics.

Step 1 — Start from a fresh branch

1. In GitHub, switch the branch picker to next, then create feature/<short-description>. A new branch starts from whatever branch you are standing on — starting from next means you begin from everyone's integrated work rather than from production.
2. In DataForge, open your developer project's Git tab, set the branch to your new feature branch, confirm the path is dataforge, and save. The dropdown lists only branches that already exist — if yours is missing, it was not created in GitHub.
3. Click Initial Pull.

Verify: the Imports tab shows the import completed. Your project now matches next exactly, so the only differences you push later are your own changes.

Warning

Baseline before you make changes, never after. An Initial Pull replaces the project with the branch, and a regular Pull that imports does the same — running either after you have made unpushed changes overwrites them.

Note

Switching branches carries your old branch's sync history with it, which is why the baseline is an Initial Pull rather than a regular one. If your project still holds objects from a previous piece of work, the Initial Pull warns before deleting sources — expected here, since the point is to reset your sandbox to match next. Confirm the list is only your own leftovers, then proceed.

Step 2 — Build and test

Make your changes in DataForge — sources, rules, relations, outputs — and run them in your project. Your project's Variables keep its Outputs, Schedule, and compute separate from everyone else's, so you can build and break freely.

If the change involves a custom notebook, switch your Databricks Git folder to the same feature branch and edit the notebook there. The notebook and the configuration that calls it then travel in the same pull request.

Step 3 — Push and open the pull request

1. On the Git tab, click Push. Each push lands as a single commit on your branch, with a message recording the project and your DataForge user. A Push with nothing new reports the branch up to date and commits nothing.
2. Verify: the commit is visible on your branch in GitHub.
3. Open a pull request. GitHub defaults the base branch to main — change it to next, or the pull request proposes shipping straight to production.
4. Request a review. If the reviewer asks for changes, make them in DataForge and Push again — the new commit joins the same pull request.

Reviewing a generated-YAML pull request

Exports are generated, so diffs run long — reviewers are not expected to read every line. Check that only the expected objects changed, that nothing was deleted unexpectedly, and that no environment-specific name is hardcoded where a Project Variable belongs.

If someone edited the branch directly

Push does not overwrite commits that landed on your branch since your last sync — a hand-edited YAML file, or a reviewer's suggested change committed in GitHub. It merges them into your push and imports the merged result back into your project, so the two sides stay identical. If the same object changed in both places, the Push fails as a merge conflict instead of guessing — see Troubleshooting.

Step 4 — Integrate into next

1. Merge the approved pull request.
2. Open the Default Dev project's Git tab and click Pull. Your change is now combined with everyone else's merged work. (A Pull when nothing new has merged simply reports the project up to date.)
3. Test the integrated result — this is the step that catches two changes that are each correct alone and conflict together.
4. Delete the feature branch in GitHub; it has served its purpose.

Step 5 — Promote to production

Before opening the release pull request, run the pre-flight checks. A merge carries only Project configuration — the objects below do not travel with it:

• Every new workspace object (Connection, Schedule, Compute Configuration) exists in the Prod workspace, with exactly matching names.
• Every new Variable has a value in the Prod Default project.
• The Prod Databricks Git folder is ready to pull any new notebooks.
• Both workspaces are on the same major DataForge version.

Then:

1. Open a pull request from next into main, and merge it once approved.
2. On the Prod Default project's Git tab, click Pull. Confirm the import completes on the Imports tab.
3. If the change includes custom notebooks, update the Prod Databricks Git folder to main — configuration first, then notebooks, so the pipeline never calls a notebook whose configuration has not arrived.
4. Verify: the promoted sources are active and a pipeline run succeeds.

Releases and Rollback covers this step in depth — the release checklist, who approves what, rolling back a bad release, and automating the production Pull.

Hotfixes

A hotfix is an urgent production fix that cannot wait for the normal cycle through next. Create the hotfix branch from main, not from next — branching from main means the fix carries exactly what production is running plus your change, and nothing else that is still being tested.

1. Create hotfix/<name> from main in GitHub.
2. Point a project at it, Initial Pull to baseline, make and test the minimal change, and Push.
3. Open two pull requests: one from the hotfix branch into main, and one from the hotfix branch back into next, so production and integration stay in sync.
4. After both merge, Pull main into the Prod Default project and next into the Dev Default project.

Troubleshooting

Symptom	Likely cause	Fix
My branch is missing from the Git tab dropdown	The branch does not exist in GitHub yet	Create it in GitHub first, then reopen the dropdown.
DataForge cannot retrieve branches at all	The DataForge GitHub App is not installed, or lacks access to the repository	Check the app's installation and repository access in GitHub.
Only Initial Push and Initial Pull are offered	The project has never synced with this connection — the Commit SHA field is blank	Run the Initial operation that matches your intent; the regular buttons appear once the project has synced.
Push and Pull are grayed out even though the project is configured	Button state follows your active project (the drop-down in the header) — it is locked, or you lack Editor access on it	Switch your active project to one you can edit, then retry. Locking blocks manual edits, not imports — a locked project can still be pulled into.
Push or Pull failed with a merge conflict	The same object changed in the project and on the branch since the last sync	Nothing is lost — the error names a temporary branch (<branch>-<workspace>-<import id>) holding your changes. Either resolve in Git (open a pull request from the temporary branch into your branch, resolve the conflicts, merge, then Pull), or pick a winner: Initial Push keeps the project's version, Initial Pull keeps the branch's. Delete the temporary branch afterwards.
Branches like feature/x-4-1321 appeared in the repository	Temporary branches created during Push and Pull	Deleted automatically when the operation succeeds; preserved when it fails so nothing is lost. Safe to delete once the operation is resolved.
A Push committed to GitHub but the import after it failed	Push imports the merged result back into the project, and that import failed — often a YAML hand edit that does not validate	Open the log on the Imports tab, fix the cause, and restart the import from the same row. The branch finishes updating only after the import succeeds.
Pull failed with an empty Variable	The Variable exists in the incoming configuration but has no value in this project	Set the value on the Variables tab, then restart the import from the Imports tab.
Pull failed on a Connection, Schedule, or Compute Configuration name	The workspace object is missing in this workspace, or named differently	Create it with the exact name, then retry.
Pull paused with a warning that sources will be deleted	The branch genuinely lacks sources this project has	Expected when re-baselining your own sandbox with Initial Pull. Anywhere else, choose Fail Import and investigate — see the deletion warning.
My changes disappeared after a Pull	The Pull imported over unpushed changes — an importing Pull replaces the project with the branch	Redo the changes, then Push. Always baseline first, then work, then Push.
I edited YAML on the branch and the project never changed	Editing files does not change the project by itself	Click Pull to import the edits. A regular Push would also merge them in, but Pull is the direct route.
I edited YAML on the branch and the edits vanished	An Initial Push overwrote the branch with the project	Use a regular Push, which merges branch-side edits instead of discarding them. Recover the lost edits from the branch's commit history in GitHub.
The pull request shows thousands of changed lines	The project was out of sync before work started	Branch from next and Initial Pull before making changes; review the diff for unintended changes before merging.
Import rejected because of the export format version	The workspaces are on different major DataForge versions	Align the workspace versions before promoting.
A pipeline calls a notebook that does not exist in Prod	Notebooks are not migrated by export/import	Pull the branch in the Prod Databricks Git folder — see Repository Structure.

For import mechanics and log locations, see Projects Overview → Importing a Project.