Viewing Source Data¶

Every source has a hub table behind the scenes that represents all of the input data brought into the source.

Querying Source Data in Hub Tables¶

All source data is visible via hub tables in your data platform, Databricks or Snowflake. Hub tables are stored in a standalone "dataforge" catalog or database. The Source Data View tab queries the hub table through the DataForge API and displays the results inside DataForge.

Data View, expression validation, and Talos data queries require the customer-configured serverless SQL warehouse identified by databricks-warehouse-id. DataForge does not create or automatically select a replacement warehouse. See Global System Configurations for setup and permission requirements.

If the setting is missing or the warehouse is invalid, inaccessible, or not serverless, DataForge displays the configuration error in the affected workflow.

Data View continues to use the configured Snowflake JDBC connection.

To query any particular source, open a SQL query or notebook (requires compute to be created and running) and run a query using the following syntax for a hub table: dataforge.hub_ or *. from Source settings view name. Visit the following documentation to learn more about using the SQL editor or Notebooks.

Databricks: SQL Editor or Notebooks

Snowflake: Worksheets or Notebooks

The Source ID can be found in the site URL when the Source is opened or a list is provided on the main Sources page before opening a Source. To query source view names, the project schema should be used which can be found in the project settings in the Projects page.

Examples:

select * from dataforge.hub_1

select * from default.customer

Viewing sub-source data¶

To view data in a sub-source, query the parent source to view the rule driving the sub-source where all sub-source attributes and rules are managed.

For full documentation, visit Sub-Sources.