Automate File Uploads from Cloud Storage with Advanced Integration

Overview

Automated file uploads enable Yarken to retrieve files from supported cloud storage providers and process them without manual intervention.

This integration combines an advanced connection, an automation recipe, a Data Mapping Template, and a Yarken pipeline to automate the data ingestion process.

Use this integration when you want to:

• Automatically import files from cloud storage.

• Eliminate manual file uploads.

• Schedule recurring data imports.

• Trigger Yarken pipelines when new files become available.

Supported cloud storage providers depend on your configured integration.

How automated file uploads work

The automation recipe monitors the configured cloud storage location, retrieves matching files, and submits them to Yarken for processing.

Cloud Storage
       │
       ▼
Advanced Connection
       │
       ▼
Automation Recipe
       │
       ▼
Yarken Upload API
       │
       ▼
Yarken Pipeline
       │
       ▼
Reports, Dashboards, Analytics, and Planning

Once the upload is complete, Yarken processes the file using the selected Data Mapping Template and associated pipeline.

Before you begin

Before configuring this integration, ensure that:

• The required cloud storage connection has been authorized.

• The automation recipe has been deployed.

• The required Data Mapping Template has been created.

• The destination pipeline has been configured.

• A Yarken API key has been generated, if required by the integration.

For more information, see:

• Authorize Advanced Connections

• Manage Automation Recipes

• Data Mapping Templates

• Pipelines

• API Keys

Configure the integration

Complete the following steps to configure automated file uploads.

Responsibilities

Step 1: Authorize the cloud storage connection

Authorize the connection for your cloud storage provider.

Depending on your environment, this may include:

• Amazon S3

• Azure Blob Storage

• Google Cloud Storage

• Other supported storage providers

For instructions, see Authorize Advanced Connections.

Step 2: Configure the Yarken connection

If the recipe uploads files through the Yarken API, configure the Yarken connection using:

• Your Yarken environment URL

• A valid Yarken API key

For more information, see API Keys.

Step 3: Configure the automation recipe

Open the required automation recipe and configure the appropriate settings for your environment.

Depending on the integration, you may configure:

• Source folder or bucket

• File naming pattern

• Execution schedule

• Destination upload action

• Required input parameters

Review the recipe configuration before activating it.

Step 4: Select the Data Mapping Template

Select the Data Mapping Template that matches the files being imported.

The mapping template determines how source fields are mapped to Yarken during processing.

Using the correct mapping template helps ensure consistent and accurate data ingestion.

For more information, see Data Mapping Templates.

Step 5: Activate the automation

Start the automation recipe after confirming that:

• All required connections have been authorized.

• The recipe configuration has been reviewed.

• The correct Data Mapping Template has been selected.

Once activated, the recipe monitors the configured storage location according to its trigger.

Validate the integration

After the first execution, verify that:

• The automation recipe completed successfully.

• The file was retrieved from cloud storage.

• The upload completed successfully.

• The destination pipeline processed the file.

• The imported data is available in Yarken.

If your imported data is used for reporting or analytics, refresh the affected cube if required before validating the results.

Best practices

Follow these recommendations when configuring automated file uploads:

• Store only the files required for the integration in the monitored folder.

• Use a consistent file naming convention.

• Verify the Data Mapping Template before activating the recipe.

• Test the integration with a sample file before enabling scheduled execution.

• Monitor recipe execution regularly to identify upload failures.

Troubleshooting

Related content

• Authorize Advanced Connections

• Manage Automation Recipes

• Data Mapping Templates

• Pipelines