Setting up the GitHub Workflow

This page walks through the one-time setup for the recommended branching workflow — from an empty repository to protected branches, connected Default projects, and an isolated project per developer. After this setup, day-to-day work follows the GitHub Developer Workflow.

The steps assume the two-workspace topology — a Dev and a Prod workspace, each with a Default Project — and use the folder layout from Repository Structure. Working out of a single workspace? The setup is nearly identical; Environments and Workspaces lists what changes.

Before you begin

Confirm you have:

• Permission to install GitHub Apps, create repositories, and configure branch protection in the GitHub organization.
• Editor access or higher on the Default project in each workspace.
• Both workspaces on the same major DataForge version — export/import does not cross major versions.
• Access to each data platform workspace or account (Databricks or Snowflake), if your pipelines use custom notebooks.

Step 1 — Install the DataForge GitHub App

DataForge authenticates to GitHub with the DataForge GitHub App — there is no personal access token to create or rotate.

1. Open the DataForge GitHub App page.
2. Install it on the GitHub organization that owns (or will own) the repository, granting access to that repository. GitHub's own steps are in Installing a GitHub App from a third party.

Note

The installer needs permission to install GitHub Apps for the organization. If GitHub shows Request instead of Install, an organization owner must approve the request before you can continue.

Step 2 — Create the repository

Create a repository in GitHub with main as its default branch. An empty repository is fine — DataForge seeds it in the next step.

Step 3 — Connect production and seed main

Connect the Prod workspace's Default project first, so the repository starts from what production is actually running.

1. In the Prod workspace, open Menu → Projects, open the Default project, and select the Git tab.
2. Enter the GitHub organization/owner and repository name, set the branch to main, and set the path to dataforge.
3. Save, then click Initial Push — a never-synced project offers only the Initial operations, and Initial Push writes the project into the empty folder.

Verify: the push appears as a row on the project's Imports tab, the repository now contains a dataforge folder with one YAML file per object (committed with a message recording the project and your DataForge user), and the Git tab's Commit SHA field is populated. Once it is, the regular Push and Pull buttons appear alongside the Initial ones.

Note

The branch dropdown lists branches that already exist in the repository — to connect to a branch, create it in GitHub first.

Step 4 — Create next and connect development

1. In GitHub, create a branch named next from main.
2. In the Dev workspace, open the Default project's Git tab, enter the same repository, set the branch to next and the path to dataforge, and save.
3. Click Initial Pull.

Verify: the Imports tab shows the import completed, and the Dev Default project's sources and outputs now match production.

Warning

An Initial Pull is a full replacement — it deletes objects the branch does not contain. If the Dev Default project holds configuration that has never been merged, export it as a backup before this first Pull. An import that would delete sources pauses for confirmation; choose Fail Import if you did not expect it.

If the Pull fails, the two common causes are missing workspace objects and unpopulated Variables — see Troubleshooting.

Step 5 — Protect the branches and lock the projects

Close both doors: branch protection stops direct pushes to the shared branches, and locking stops manual edits to the projects they feed.

In GitHub, under Settings → Branches, add a protection rule for main and one for next:

• Require a pull request before merging.
• Require at least one approval (teams commonly require more for main).
• Dismiss stale approvals when new commits are pushed.
• Restrict who can push to the branch.

In DataForge, open each Default project and enable Lock Project, so those projects change only by Pull — never by hand. Locking blocks manual configuration edits, not imports: Pull is exactly how a locked project is meant to change.

Step 6 — Create the developer projects

Each developer works in a project of their own, so unfinished work never sits in the project that gets promoted.

1. On the Projects page, click New +, name the project (for example alice-dev), and save. Creating a project requires the workspace Power User role or above — see Manage Users.
2. Give the project its own Project Variables so it does not collide with the Default project. Set the same variable names on the Default project and on every developer project, with different values in each:

Variable type	Default project value	Developer project value
Output Schema Name	reporting	dev_alice_reporting
Schedule	Daily	On Demand
Custom Compute Configuration	Prod-sized cluster	Dev cluster

Why each one matters: every project in a workspace writes to the same data platform, so without its own Output schema (or Output Connection) a developer project writes to the shared tables; without its own Schedule it ingests on the Default project's timetable; without its own Compute Configuration it competes for the same clusters.

Warning

Set these Variables when the project is created, before its first run. A Variable that is missing on import is auto-created empty and the import fails until it is populated — populate the value on the Variables tab, then restart the import from the Imports tab.

Step 7 — Version-control your custom notebooks

Skip this step if your pipelines use no custom notebooks.

Custom notebooks are not migrated by Project export/import. Each workspace keeps its own copy, and it is on you to keep that copy aligned with the branch the workspace's DataForge project tracks.

Each Databricks workspace keeps its own Git folder, checked out on the matching branch:

1. In each Databricks workspace, create a Git folder for the repository.
2. Enable Sparse checkout mode and set the cone pattern to dataforge-sdk, so the folder contains only the notebooks — see Repository Structure.
3. Point the Dev workspace's folder at next (developers switch it to their feature branch while working) and the Prod workspace's folder at main.

Note

Sparse checkout cannot be disabled after the Git folder is created — if the pattern is wrong, delete the Git folder and create a new one.

Snowflake notebooks live in Snowsight and are attached to a Source or Output by name, not by path — see Setting up DataForge with Snowflake Custom Notebook. There is no folder to check out, so promotion turns on those names instead:

1. Create the notebook in each Snowflake account you promote into, under exactly the name the Source or Output configuration references. An import that names a notebook the account does not have will fail when the process runs.
2. Keep the copies aligned when you change one — nothing propagates a Snowsight edit from one account to another.
3. Keep the notebook's source in the repository's dataforge-sdk folder anyway, so a change to a notebook and the configuration that calls it still travel through one reviewed pull request.

Setup checklist

• DataForge GitHub App installed with access to the repository
• Repository created; main seeded by the Prod Default project's Initial Push
• next created from main; Dev Default project completed an Initial Pull from it
• Branch protection on main and next; both Default projects locked
• One project per developer, with isolation Variables set before first run
• Custom notebooks version-controlled and present in every workspace (if your pipelines use them)

With setup complete, developers follow the GitHub Developer Workflow.