External API¶

DataForge exposes machine-to-machine (M2M) API routes under /m2m for automated integrations. Use these routes to request a short-lived bearer token, list sources, trigger source pulls, start project Git pull or push imports, and check import status.

The legacy /external/* routes are no longer used.

Before you start¶

Ask a DataForge administrator to create a machine user in Users and generate credentials for that user.

Use the machine user username as the token name.
Assign the minimum project role required for the automation:
Read-Only: list sources and query inputs.
Operator or above: trigger source pulls.
Editor or above: run project Git pull or push imports.
Store the generated client_id and client_secret securely. The secret is displayed only when it is generated.

Base URL and authentication¶

Use the API hostname for your DataForge environment. All routes except /m2m/token require a bearer token from /m2m/token.

BASE_URL = "https://stage.api.dataforgedev.com"
CLIENT_ID = dbutils.secrets.get(scope="api_connections", key="client_id")
CLIENT_SECRET = dbutils.secrets.get(scope="api_connections", key="client_secret")
PROJECT_NAME = "Demo Project"

Common headers:

Authorization: Bearer <access_token>
Content-Type: application/json

Common error responses:

Status	Body	Meaning
400	`{"error":"client_id and client_secret are required"}`	Token request is missing required credentials.
401	`{"error":"Invalid client credentials"}`	Credentials are unknown, inactive, or invalid.
429	`{"error":"Too many requests"}`	API rate limit exceeded.
429	`{"error":"Too many token requests"}`	Token request rate limit exceeded.
500	`ERROR: <message>`	Validation or server error, such as missing project name or insufficient permission.

Endpoints¶

POST /m2m/token¶

Creates a short-lived access token for M2M API calls. This endpoint does not use bearer authentication.

Request body

Field	Type	Required	Description
`client_id`	string	Yes	Machine credential client ID.
`client_secret`	string	Yes	Machine credential secret. Displayed only when generated.

Request

import requests

token_response = requests.post(
    f"{BASE_URL}/m2m/token",
    json={
        "client_id": CLIENT_ID,
        "client_secret": CLIENT_SECRET,
    },
    timeout=30,
)
token_response.raise_for_status()
token = token_response.json()["access_token"]

Response

{
  "access_token": "<jwt>",
  "token_type": "Bearer",
  "expires_in": 3600
}

POST /m2m/source-pull/{sourceId}¶

Triggers a source to run "pull data now" and begin ingestion. The machine user must have Operator access or above to the source's project.

Path parameters

Field	Type	Required	Description
`sourceId`	integer	Yes	Source ID to pull.

Request

source_id = 19643
headers = {"authorization": f"Bearer {token}"}

response = requests.post(
    f"{BASE_URL}/m2m/source-pull/{source_id}",
    headers=headers,
    timeout=30,
)
response.raise_for_status()

Response

{}

POST /m2m/inputs-filtered¶

Returns inputs filtered by source, limit, and effective status. Include project_name in the request body so DataForge can validate the machine user's project access.

The machine user must have Read-Only access or above to the project.

Request body

Field	Type	Required	Description
`project_name`	string	Yes	Project name used for authorization and filtering.
`source_id`	integer	Yes	Source ID whose inputs should be returned.
`limit`	integer	No	Maximum rows to return. Defaults to `50`.
`effective_filter`	boolean	No	When `true`, returns only inputs with effective records. Defaults to `false`.
`filter_status`	string	No	Input status code. Use `I` to include in-progress/loaded rows.
`search_text`	string	No	Matches source file name or input ID prefix.
`date_from`	date	No	Lower bound for `received_datetime`, formatted `YYYY-MM-DD`.
`date_to`	date	No	Upper bound for `received_datetime`, formatted `YYYY-MM-DD`.

Request

body = {
    "project_name": PROJECT_NAME,
    "source_id": 19709,
    "limit": 100,
    "effective_filter": False,
}

response = requests.post(
    f"{BASE_URL}/m2m/inputs-filtered",
    headers=headers,
    json=body,
    timeout=30,
)
response.raise_for_status()
print(response.json())

Response

{
  "inputs": [
    {
      "input_id": 456,
      "ingestion_status_code": "P",
      "received_datetime": "2026-07-09T14:15:22Z",
      "source_file_name": "customers.csv",
      "record_count": 1000,
      "effective_record_count": 998,
      "status_code": "P",
      "last_completed_process_type": "Enrichment",
      "current_process_type": null
    }
  ],
  "pull_now_flag": false,
  "total_count": 37
}

GET /m2m/sources¶

Returns the source list for a project. The machine user must have Read-Only access or above to the project.

Query parameters

Field	Type	Required	Description
`projectName`	string	Yes	Project name used for authorization and filtering.

Request

response = requests.get(
    f"{BASE_URL}/m2m/sources",
    headers=headers,
    params={"projectName": PROJECT_NAME},
    timeout=30,
)
response.raise_for_status()
print(response.json())

Response

[
  {
    "source_id": 19709,
    "source_name": "Customers",
    "connection_id": 12,
    "connection_name": "Warehouse",
    "connection_type": "databricks",
    "group_id": 3,
    "group_name": "CRM",
    "refresh_type": "full",
    "active_flag": true,
    "status_code": "P",
    "scheduled_start_datetime": null
  }
]

GET /m2m/sources-filtered-by-name/{filter}¶

Returns sources whose source name contains the supplied filter text.

The machine user must have Read-Only access or above to the project.

Path parameters

Field	Type	Required	Description
`filter`	string	Yes	Case-sensitive text matched within `source_name`.

Query parameters

Field	Type	Required	Description
`projectName`	string	Yes	Project name used for authorization and filtering.

Request

name_filter = "demo"

response = requests.get(
    f"{BASE_URL}/m2m/sources-filtered-by-name/{name_filter}",
    headers=headers,
    params={"projectName": PROJECT_NAME},
    timeout=30,
)
response.raise_for_status()
print(response.json())

Response

[
  {
    "source_id": 19709,
    "source_name": "Demo Customers",
    "connection_id": 12,
    "connection_name": "Warehouse",
    "connection_type": "databricks",
    "group_id": 3,
    "group_name": "CRM",
    "refresh_type": "full",
    "active_flag": true
  }
]

PUT /m2m/git/{operation}¶

Starts a project Git pull or push import and returns the import_id. The machine user must have Editor access or above to the project.

Supported operations:

pull
push
init-pull
init-push

Path parameters

Field	Type	Required	Description
`operation`	string	Yes	One of `pull`, `push`, `init-pull`, or `init-push`.

Query parameters

Field	Type	Required	Description
`project_name`	string	Yes	Project name used for authorization and import setup.

projectName is also accepted for compatibility, but project_name is preferred.

Request

operation = "pull"

response = requests.put(
    f"{BASE_URL}/m2m/git/{operation}",
    headers=headers,
    params={"project_name": PROJECT_NAME},
    timeout=30,
)
response.raise_for_status()
result = response.json()
import_id = result["import_id"]
print(result)

Response

{
  "import_id": 12345,
  "test": false
}

If DataForge starts the import but core processing fails immediately, the response includes both error and import_id so you can still check the import log:

{
  "error": "<error message>",
  "import_id": 12345
}

GET /m2m/imports/{importId}/status¶

Checks an import started by the M2M API. If the import failed, the response includes the error body from the import log.

The machine user must have Editor access or above to the import's project.

Path parameters

Field	Type	Required	Description
`importId`	integer	Yes	Import ID returned by `PUT /m2m/git/{operation}`.

Request

response = requests.get(
    f"{BASE_URL}/m2m/imports/{import_id}/status",
    headers=headers,
    timeout=30,
)
response.raise_for_status()
print(response.json())

Response

{
  "import_id": 12345,
  "project_id": 1,
  "status_code": "I",
  "error": null
}

Failed import response:

{
  "import_id": 12345,
  "project_id": 1,
  "status_code": "F",
  "error": "<failure details>"
}

Complete Git pull example¶

import time
import requests

BASE_URL = "https://stage.api.dataforgedev.com"
PROJECT_NAME = "Demo Project"
CLIENT_ID = dbutils.secrets.get(scope="api_connections", key="client_id")
CLIENT_SECRET = dbutils.secrets.get(scope="api_connections", key="client_secret")

token_response = requests.post(
    f"{BASE_URL}/m2m/token",
    json={"client_id": CLIENT_ID, "client_secret": CLIENT_SECRET},
    timeout=30,
)
token_response.raise_for_status()
token = token_response.json()["access_token"]
headers = {"authorization": f"Bearer {token}"}

start_response = requests.put(
    f"{BASE_URL}/m2m/git/pull",
    headers=headers,
    params={"project_name": PROJECT_NAME},
    timeout=30,
)
start_response.raise_for_status()
import_id = start_response.json()["import_id"]

while True:
    status_response = requests.get(
        f"{BASE_URL}/m2m/imports/{import_id}/status",
        headers=headers,
        timeout=30,
    )
    status_response.raise_for_status()
    status = status_response.json()
    print(status)

    if status.get("status_code") in ("P", "F"):
        break

    time.sleep(10)