Skip to main content

Varicent ELT Assistant

Amazon S3 import

Connect your Varicent ELT and Amazon S3 accounts to import your data. Amazon S3 accepts the following file types:

  • CSV

  • Excel

  • Parquet

  • .txt

Importing data

  1. In Varicent ELT, select the Data module from the sidebar navigation pane.

  2. Click Import data.

  3. In the Third party sources section, click Amazon S3.

  4. Click New connection.

  5. On the Connect to your Data page, under Connector name, enter the name of the connection.

  6. Under Account ID, the ID of the AWS account that you want to use.

  7. Under Bucket Name, enter the name of the AWS bucket in your Amazon S3 instance that you connect to.

  8. Under Region, enter the region of your Amazon S3 bucket, and then click Next.

  9. On the next Connect to your Data page, follow the on-screen steps to set up your account.

    1. Follow the on-screen steps to create a new policy in your AWS account.

      Note

      The policy body is pre-populated with the information that you had entered in the previous step. Use the copy button to copy the code.

    2. Follow the on-screen steps to create a new role in your AWS account.

      Note

      The account ID value is based on the environment that you are in.

    3. Create a unique name in the Role Name field.

    4. Click Next.

  10. On the Select data page, under Directory, select the directory for the file that you want to import. For example, if you want to select Parquet files, ensure that you select the folder containing your Parquet files.

  11. Under File type, select the file type that you want to import, such as CSV, Excel, .txt or Parquet.

    Caution

    If you select a directory with Parquet files, ensure that you select the Parquet file type. If you select another type of file, the import will fail.

    Note

    If you select the Parquet file option, you won't have to specify the Delimeter and other configuration options, just follow the import wizard.

  12. Under Delimiter, select the delimiter that you want to use from the following delimiters in the drop-down field:

    • Comma (,)

    • Pipe (|)

    • Semi-colon (;)

    • Space ( )

    • Tab ( )

    • Tilde (~)

  13. Under Quote, select the type of quotes that you want to use, such as double quotes (").

  14. Under Exclude excess data, enter the number of rows that you want to ignore in the beginning or end of your file.

  15. Under Header, check to designate the first unignored row as the column header. Leave unchecked to have column headers automatically generated, and then click Next.

  16. On the Import data page, review your data and select the data to obfuscate (if any), and select Preserve out of bound dates to maintain any dates outside the date range, and then click Next.

  17. On the Filter data page, optionally click Filter data to refine your data, or click Import data to continue.

    1. Filter the rows in your data by selecting the column to filter, the filter category and the value or text.

    2. Click Import data to continue.

    Note

    For more information about how to configure data filters, see: Filter data.

  18. On the Review page, review the data sets that you are importing.

  19. Click Import data.

    The data is available on the My Data page.