Skip to content

You are viewing documentation for Immuta version 2023.3.

For the latest version, view our documentation for Immuta SaaS or the latest self-hosted version.

cancel

Connect Data Sources Using dbt Cloud Integration (Preview Feature)

Audience: Data Owners

Content Summary: This tutorial outlines the process of connecting a dbt Cloud integration. This integration allows Immuta to be connected to your dbt Cloud jobs so that updates run through dbt populate within Immuta as new data sources, column descriptions, data source descriptions, and tags.

This dbt Cloud integration is a Preview Feature that must be enabled by an Application Admin on the App Settings page.

1 - Connect a New dbt Integration

  1. Click the plus button in the top left of the Immuta console.
  2. Select New dbt Cloud Integration.

2 - Enter dbt Connection Information

  1. Input either

    • Account Admin service account token: This token has full read + write access to the dbt Cloud account and can be created in Account Settings of your dbt Cloud account. See the dbt Cloud documentation for more information.
    • Account ID: This can be found in the URL of your dbt Cloud project (example: https://cloud.getdbt.com/#/accounts/<AccountId>/projects/<ProjectId>/dashboard/).

    Or

    • Metadata Only service account token: This token can authorize requests to the metadata API and can be created in Account Settings of your dbt Cloud account. See the dbt Cloud documentation for more information.
    • User API token: This token inherits the permissions of the user and can be found in your dbt Cloud Profile. See the dbt Cloud documentation for more information.
    • Account ID: This can be found in the URL of your dbt Cloud project (example: https://cloud.getdbt.com/#/accounts/<AccountId>/projects/<ProjectId>/dashboard/).

  2. Click Get Projects.

3 - Enter dbt Job Information

  1. Use the dropdown menu to select the dbt Project and dbt Job.

  2. Add the webhook URL to any dbt job in your selected project with the same environment as the selected job to update Immuta data sources after each run.

4 - Enter Credentials

To connect the dbt Cloud project in Immuta, enter the credentials used to authenticate to the underlying database in dbt Cloud.

Databricks or Databricks SQL

  1. Enter the Databricks API Token used to authenticate to the database in your dbt project.
  2. Opt to enter any Additional Connection String Options or Upload Certificates.
  3. Click Test Connection.

BigQuery

  1. Enter the Username and Password used to authenticate to the database in your dbt project.
  2. Opt to enter any Additional Connection String Options or Upload Certificates.
  3. Click Test Connection.

Snowflake

Select the Authentication Method from the dropdown menu.

  1. Enter the Snowflake Username and Password used to authenticate to the database in your dbt project.
  2. Opt to enter any Additional Connection String Options or Upload Certificates.
  3. Click Test Connection.
  1. Enter the Snowflake Username used to authenticate to the database in your dbt project.
  2. Opt to enter the private key file password in the Additional Connection String Options. Use the following format: PRIV_KEY_FILE_PWD=<your_pw>
  3. Click Select a File.
  4. Use the pop-up to select and upload a Snowflake key pair file.
  5. Click Test Connection.

Redshift

Select the Authentication Method from the dropdown menu.

  1. Enter the Redshift Username and Password used to authenticate to the database in your dbt project.
  2. Opt to enter any Additional Connection String Options or Upload Certificates.
  3. Click Test Connection.
  1. Enter the Additional Connection String Options
  2. Opt to Upload Certificates.
  3. Click Test Connection.
  1. Enter the Additional Connection String Options
  2. Opt to Upload Certificates.
  3. Click Test Connection.

5 - Enter Basic Information

Provide information about your source to make it discoverable to users.

  1. Enter the SQL Schema Name Format to be the SQL name that the data sources pulled from the dbt integration will exist under in the Immuta Query Engine. It must include a schema macro but you may personalize the format using lowercase letters, numbers, and underscores. It may have up to 255 characters.

  2. Enter the Schema Project Name Format to be the name of the schema project in the Immuta UI. This field is disabled if the schema project already exists within Immuta.

  3. Select the Data Source Name Format, which will be the format of the name of the data sources pulled from the dbt integration in the Immuta UI.

    <Tablename>

    The data source name will be the name of the remote table, and the case of the data source name will match the case of the macro.

    <Schema><Tablename>

    The data source name will be the name of the remote schema followed by the name of the remote table, and the case of the data source name will match the cases of the macros.

    Custom

    Enter a custom template for the Data Source Name. You may personalize this field as you wish, but it must include a tablename macro. The case of the macro will apply to the data source name (i.e., <Tablename> will result in "Data Source Name," <tablename> will result in "data source name," and <TABLENAME> will result in "DATA SOURCE NAME").

  4. Enter the SQL Table Name Format, which will be the format of the name of the tables in Immuta. It must include a table name macro, but you may personalize the format using lowercase letters, numbers, and underscores. It may have up to 255 characters.