This function starts ETL/ELT pipeline a.
Returns the ETL/ELT job number on success, "-1" else.
The name of a map containing ETL/ELT variables can be passed in parameter b.
Parameters
Parameter | Description |
|---|
a | Name of the ETL/ELT pipeline to start. |
b | (optional) Name of the map containing ETL/ELT variables to be passed (name of the variable in the map key, value of the variable in the map value). |
A module for high-performance transfer and transformation of large amounts of data. An ETL/ELT pipeline consists of tasks, each of which defines a data source and a data destination. Typical use cases: database to database, CSV to database, or database to CSV.
ETL (Extract, Transform, Load) is the traditional data integration workflow. Data is collected from sources and sent to a staging area to be cleaned, formatted, and enriched, and then loaded into a destination. It ensures data quality and security before storage, making it ideal for smaller, structured datasets.
ELT (Extract, Load, Transform) is a data integration process that pulls raw data from a source and loads it directly into a target system (such as a cloud data warehouse) before processing it. This leverages the power of the target to perform transformations, enabling faster ingestion of big data.