Transformation as a Part of ETL/ ELT
During analytics projects, data may be transformed at multiple stages of
the data pipeline. Data transformation process is one of the important
parts of data management, which deals with changing the format,
structure, or values of data. Transformation is an important step in both the
traditional ETL process as well as the ELT process that is popular in a
Data Lake paradigm. It includes processes such as data integration,
migration, warehousing, wrangling, etc. This might include adding data,
replicating data, deleting unnecessary data, standardizing data, and other
Data Transformation is mainly needed to organize the data so that they
can be easily used from a data analytics perspective. It also facilitates the
compatibility between multiple systems, applications, and data sources.
Techniques such as extraction and parsing of data, translation of data,
data mapping, aggregation of data, filtering, enrichment of data, data
imputations, indexing, ordering, renaming, formatting, etc. are used during
the data transformation process.