Skip to content

Outcrop

Outcrop, the DataForge agent that runs inside your network, is a lightweight application installed on a machine to enable data ingestion from systems that DataForge isn't directly networked into. Outcrop configurations can be managed from the Agents page in the main menu (shown as Agent in the DataForge UI). For installation steps, see Installing Outcrop.

Settings Tab

Found by clicking on an Outcrop instance, the Settings tab controls heartbeat intervals, auto-updating, security, concurrency, and plugin configurations. Note that changing certain settings will prompt a new config file to be downloaded — Outcrop will stop working until the updated file is placed on the target machine. See the installation guide for details.

Base Parameters

Asterisks (*) mean the Parameter is mandatory and must be specified by users.

  • Name*: Unique Outcrop name displayed on the Agents screen and on connected Connections.
  • Description*: Description of the Outcrop instance.
  • Code*: Backend identifier used during MSI install. Recommended: lowercase, alphanumeric, underscores instead of spaces.
  • Default Agent*: If True, auto-populated when configuring a new Connection.
  • Active*: If Active, Outcrop will trigger its configured ingestions.
  • Region*: AWS or Azure region for the cloud environment.
  • Machine Guid*: GUID of the host server. See Installing Outcrop for how to retrieve it.
  • Authentication Protocol*: Controls how Outcrop authenticates with the Workspace and determines the config bin file format.
  • IAM Access Key* (AWS only): IAM access key for a user with datalake access. Encrypted on save; not stored in the metastore.
  • IAM Secret Key* (AWS only): IAM secret key for the same user. Encrypted on save; not stored in the metastore.
  • Storage Account Name* (Azure only): Name of the Azure storage account hosting the datalake container.
  • Storage Account Key* (Azure only): Access key for the Azure storage account.
  • Datalake Container Name* (Azure only): Name of the Azure datalake container.
  • AkkaStreamTimeout: JDBC connection timeout in seconds.
  • AutoUpdate: If True, Outcrop auto-updates when a new DataForge version is deployed.
  • MaxResources: Maximum concurrent table ingestion processes. Default is 4.
  • HeartbeatInterval: Outcrop heartbeat in seconds; controls how often it checks for ingestions to run. Not recommended below 30 seconds.
  • Launch Retries: Whether Outcrop retries ingestions that were in-progress and marked failed when Outcrop restarted.

Restarting Outcrop

If Outcrop is heartbeating, it can be restarted from the UI: navigate to the Agents screen, click the triple dots to the right of the Outcrop instance, and click Restart Agent (shown as Agent in the DataForge UI). Restarting will mark all in-progress ingestions as Failed.

Outcrop should restart within 30–60 seconds. Check the Outcrop logs to confirm.

If Outcrop isn't heartbeating, restart the Outcrop service directly on the host machine. The local Outcrop instance runs as a container in the cloud provider — stop the container task (AWS) or restart the container instance (Azure).