Automate

Use Automate to turn supported file uploads into recurring cloud-storage ingestion workflows. Yarken uses the selected connection, folder, Data Mapping Template, and schedule to create and run the related pipeline.

Go to Admin > Pipelines > Automate to review file types that support automation and start a new setup.

What you can do

Use Automate to:

• Automate recurring uploads for supported file-based data types.

• Retrieve files from a selected folder in Azure Blob Storage, AWS S3, or Google Cloud Storage.

• Reuse or create a Data Mapping Template for recurring uploads.

• Schedule ingestion to run hourly, daily, or monthly.

• Start automation from the central Automate page or from a previously uploaded file.

• Review whether automation is active for each supported file type.

• Replace existing expense data for a target month before importing a new Expenses file, where required.

• Manage the generated pipeline from the Pipelines page.

Before you begin

• Configure a BASIC cloud storage connection for the source location.

• Confirm that the source folder contains only the files you want Yarken to process.

• Prepare or identify the correct Data Mapping Template.

• Keep the source file structure and column headers consistent across recurring uploads.

• Confirm the schedule you want to use.

How Automate works

A typical Automate workflow follows this sequence:

1. Select a supported file type.

2. Select the cloud storage provider and connection.

3. Select the source folder or directory.

4. Review or create the Data Mapping Template.

5. Set the upload schedule.

6. Review the generated pipeline name and save the setup.

7. Yarken uses the linked pipeline to process matching files on the configured schedule.

Supported file types

The Automate page lists the file types that support recurring file ingestion in your environment.

Understand the Automate status

The Automate icon shows whether recurring ingestion is configured for a file type:

• Blue Automate icon — Automation is active.

• Gray Automate icon — Automation is inactive or has not been configured.

You cannot deactivate automation directly from the Automate page. Manage the linked pipeline from Admin > Pipelines > Pipelines.

Choose where to start

You can start Automate from two locations.

Set up automation from the Automate page

Step 1: Open Automate

1. Go to Admin > Pipelines > Automate.

2. Find the file type you want to automate.

3. Select the gray Automate icon.

   

Step 2: Select the cloud storage provider

Select the storage provider that contains the recurring source files.

• Azure Blob Storage

• AWS S3

• Google Cloud Storage

If the required connection does not exist, create it before you continue.

See Connections.

Step 3: Select the connection

Select the BASIC cloud storage connection that provides access to the source location.

Step 4: Select the folder or directory

Enter or select the source folder that contains the files Yarken should process.

Yarken processes only files from the configured folder.

Step 5: Review the Data Mapping Template

Review the template that Yarken will apply to recurring files.

• If a suitable template exists, keep or select it.

• If the file structure requires a different mapping, create or replace the template where supported.

• If you started from UPLOADED FILES, Yarken uses that file's mapping as the starting mapping.

See Data Mapping Templates.

Step 6: Choose whether to delete existing Expenses data

For Expenses (spend) file uploads, you can use Delete before load when you need to replace the existing records for the target month.

When selected, Yarken removes the existing expense records for that month before importing the new file.

Step 7: Set the upload schedule

Choose how often Yarken should run the recurring ingestion:

• Hourly

• Daily

• Monthly

Step 8: Review and save the automation

Yarken generates a pipeline name based on the selected file type. Update the name if required.

1. Review the connection, folder, mapping, and schedule.

2. Select SAVE.

3. Confirm that the Automate icon turns blue for the file type.

Set up automation from UPLOADED FILES

Use this workflow when you already loaded a file manually and want future files to use the same mapping structure.

Step 1: Open the uploaded file

1. Open the applicable area in Yarken Designer.

2. Select the UPLOADED FILES tab.

3. Find the file you want to automate.

4. Select the Automate icon in the Actions column.

   

Step 2: Review the inherited mapping

Yarken uses the field mapping from the selected uploaded file as the starting mapping for future automated uploads.

Review the mapping before you save the automation. Replace or create a template when the recurring file structure requires a different mapping.

Step 3: Complete the automation setup

Select the connection, folder, schedule, and remaining pipeline details. Then save the automation.

Manage an automated upload

After you save an Automate setup, manage the linked workflow from the Pipelines page.

Edit the automation

Edit the linked pipeline when you need to change:

• The source connection.

• The source folder or directory.

• The Data Mapping Template.

• The schedule.

• The pipeline name or description.

Deactivate the automation

1. Go to Admin > Pipelines > Pipelines.

2. Find the linked BASIC pipeline.

3. Deactivate the pipeline.

The Automate icon becomes gray when the linked automation is no longer active.

Delete the automation

Delete the linked pipeline when you no longer need the recurring workflow.

For full pipeline management instructions, see Manage BASIC pipelines.

Recommended folder and file practices

After automation runs

After the linked pipeline completes successfully:

1. Review the pipeline run status.

2. Confirm that Yarken processed the expected files.

3. Open the Yarken feature associated with the destination data type.

4. Select the applicable entity, year, and month.

5. Verify that the expected records are available.

6. Refresh the affected cube when you need immediate reporting results.

7. Review the data in the relevant report, dashboard, or Analytics data area.

For refresh guidance, see Cube refresh and best practices for Pipelines.

Troubleshoot Automate

For additional help, see Monitor pipeline runs.

Related content

• Pipelines and integrations

• Connections

• Data Mapping Templates

• Pipelines

• Manage BASIC pipelines

• Monitor pipeline runs

• Advanced integrations