Creating a new Pipeline Connection

Create API Connection using Consent URL

Prerequisites (Tenant-level)

• For the Google API,

  • You must be a Google Workspace Super Admin for your domain.

  • The following APIs might be disabled as part of your Tenant's security compliance. Make sure they are enabled before integration:

    • Admin SDK API

    • Enterprise License Manager API

• For Microsoft Graph and Power BI API, you should have

  • A Microsoft Entra (Azure AD) tenant.

  • A Global Administrator or Privileged Role Administrator to grant tenant-wide admin consent for application permissions.

  • A pre-configured admin user to configure the Pipelines for fetching app usage reports.

  • In Power BI-specific flows:

    • A Power BI Tenant Admin and the ability to change Tenant settings in the Power BI Admin portal.

    • Following special configurations. For more information, refer https://yarken.atlassian.net/wiki/x/B4A2Kg.

      • Create a Microsoft Entra Security Group (e.g., Power BI).

      • Add an application (e.g., Yarken Power BI) as a member to the Power BI group.

      • Enable the service principals access (read-only) in PBI Tenant Admin portal.

Procedure

Use the procedure below to create the API connections automatically for Microsoft Graph API, Power BI API, Google API, and Google Reseller API.

To create,

1. Navigate to Settings > Pipelines > Connections.

2. Click + NEW CONNECTION.

3. Choose any API connection Type, for example, Microsoft Graph API.

4. Choose one of the options for configuring API:

   1. Option 1, CONNECT API — allows you to immediately connect to API using in-app OAuth flow. The CONNECT API button name varies based on connection type you selected, for example, CONNECT MICROSOFT GRAPH API, CONNECT POWER BI API, or CONNECT GOOGLE API.

      1. Click CONNECT button.

      2. Sign in with the your cloud provider specific credential account and grant the requested permissions.

      3. Once accepted, you will be redirected to Yarken, and the required permissions should have been granted.

   2. Option 2, COPY CONSENT URL — allows you to copy and share the consent URL to authorize via email or another device. This option is ideal for team admins or offline setup.

      1. Click COPY CONSENT URL.

      2. Paste the URL into a browser on the other device (or send it to the admin who will perform consent).

      3. Complete sign-in and grant admin consent.

      4. Once accepted, you will be redirected to Yarken, and the required permissions should have been granted.

5. Verify that the option to create a pipeline and fetch data instantly is checked.

The application creates a connection and its associated pipeline.

Create API Connection manually

Use the procedure below to create the API connections manually for Microsoft Graph and Power BI API.

Prerequisites

When creating an API connection manually, you must have:

• Tenant Id, Client Id, and Client Secret

Procedure

To create,

1. Navigate to Settings > Pipelines > Connections.

2. Click + NEW CONNECTION.

3. Choose Microsoft Graph or Power BI API connection Type.

4. In the Authentication Type, select Custom Credentials from the drop-down list.

5. On Create a new connection dialog box, enter the details below:

   1. Name: The name of the connection.

   2. Tenant Id: The unique identifier of your Azure Active Directory tenant.

   3. Tenant Name: The name of your Azure Active Directory tenant.

   4. Client Id: The unique identifier for your Azure AD application (App Registration).

   5. Client Secret: The secret key used to authenticate your application with the Microsoft Graph API.

6. Click TEST CONNECTION to validate credentials.

7. If the test passes, click SAVE.

Create Cloud Storage Connection (manual only)

Use this procedure to create cloud storage connections for Azure Blob Storage, AWS S3, and Google Cloud Storage. After a connection is created, you can use it when configuring a Cloud Storage Pipeline.

Prerequisites

Before you begin, ensure you have the required credentials for your cloud storage provider.

Azure Blob Storage

Choose one of the following authentication methods:

Account Key authentication

• Storage Account Name

• Storage Account Access Key

• Container Name

Service Principal authentication

• Storage Account Name

• Tenant ID

• Client ID

• Client Secret

• Container Name

AWS S3

• Access Key

• Secret Key

• Bucket Name

• (Optional) Region

For more information on how to retrieve AWS access keys, refer Retrieve AWS Access and Secret keys.

Google Cloud Storage

• Access Key

• Secret Key

• Bucket Name

Procedure

To create a cloud storage connection:

1. Navigate to Settings > Pipelines > Connections.

2. Click + NEW CONNECTION.

3. Select the required Connection Type (for example, Azure Blob Storage, AWS S3, or Google Cloud Storage).

4. Enter the connection details.

The required fields vary depending on the selected connection type and authentication method.

Azure Blob Storage

Account Key authentication

Select Account Key as the Authentication type, then provide the following information.

Service Principal authentication

Select Service Principal as the Authentication type, then provide the following information.

AWS S3

Google Cloud Storage

5. Click TEST CONNECTION to verify that the credentials are valid.

6. If the connection test is successful, click SAVE.

The cloud storage connection is created and is available for use when configuring Cloud Storage Pipelines.

Next steps

• Edit a Connection

• Delete a Connection

Related content

• How to configure Power BI specific requirements

• Retrieve AWS Access and Secret keys.

• View Azure Storage Account Access Keys

• Create an Azure Service Principal