Top Menu

Jump to content
Home
    Modules
      • Projects
      • Activity
    • Getting started
    • Introduction video
      Welcome to Accure Project Management System
      Get a quick overview of project management and team collaboration with OpenProject. You can restart this video from the help menu.

    • Help and support
    • Upgrade to Enterprise edition
    • User guides
    • Videos
    • Shortcuts
    • Community forum
    • Enterprise support

    • Additional resources
    • Data privacy and security policy
    • Digital accessibility (DE)
    • OpenProject website
    • Security alerts / Newsletter
    • OpenProject blog
    • Release notes
    • Report a bug
    • Development roadmap
    • Add and edit translations
    • API documentation
  • Sign in
      Forgot your password?
      Create a new account

Side Menu

  • Overview
  • Documents
  • Impulse DW
    Impulse DW
  • Inset BI
    Inset BI
  • Momentum User Guide
    Momentum User Guide
  • Wiki
    Wiki

Content

You are here:
  1. Impulse DW
  2. 2. Warehouse Management
  3. 2.4 Ingesting Data Into Tables or Datasources
  4. 2.4.3 Ingesting From External File/Storage System

2.4.3 Ingesting From External File/Storage System

  • More
    • Print
    • Table of contents

Impulse supports ingesting data from the following external file systems:

  1. Amazon S3: Ingest a file of files stored in S3 bucket. This is the default storage system if Impulse is running on Amazon EC2 or you purchased the impulse license from the AWS Marketplace.
  2. HDFS: Ingest file or files stored in Hadoop Distributed File System (HDFS)
  3. Momentum: Ingest data from Momentum storage. Momentum provides a highly scalable ETL, including data ingestion from a wide variety of sources, transformation, cleaning, blending, and merging with multiple sources. It also allows ingesting data in automated fashion and creating indexes in Impulse.
  4. Google Cloud Storage: This is the default storage system if Impulse is running on Google Cloud.

To ingest data from the external system, follow these steps:

  1. From the main navigation menu, click "Load Data" (See Figure 2.4.3a below as an example)
  2. Fill out the form:
    1. Warehouse: Select the warehouse from the drop down options
    2. Datasource: Enter the table or datasource name.
    3. Input Source: Select the external system to ingest data from.
    4. Input Path: Provide the fully qualified path to the data directory or a single file. For example:
      1. Momentum: fully qualified component name, e.g. accure.tr.sampledata
      2. S3: absolute path of the file or directory, e.g. s3://mybucket/mydir
      3. GCS: absolute path of the file or directory, e.g. gs://mybucket/mydir
      4. HDFS: absolute path of the file or directory, e.g. hdfs://ip-address:port/directory/path
    5. File Format: Select the input file format
    6. Input Header: Enter a comma separated list of header columns if the input format is CSV, TSV or PSV and the input files do not contain the header in the first line.
    7. Click Next and follow the Step 2 as described in the previous section Uploading File Using Impulse UI

Figure 2.4.3a: Screen showing the form fields for ingesting data from external systems

Loading...