ETL processes: how to get value from data! Learn more about it

ETL processes: how to get value from data

ETL processes: how to get value from data

A study by Bain & Company estimates that managers who rely on data analytics make decisions five times faster than those without. This is possible thanks to the ETL processes, which allow to obtain the maximum utility of the data.

Analytics and business intelligence are areas that are gaining strength in the business area. In fact, a Gartner report predicts that the market value of these areas will reach $ 20 billion in 2019, multiplying the demand for technologies related to data processing.

What are ETL processes?

ETL processes ( extract , transform and load ) are those by which information is extracted from one or more data sources. It is transformed to suit the needs of business and then be loaded on a shared site for consultation by all stakeholders.

One of the most important characteristics of the ETL system is that it allows data from heterogeneous sources to be integrated into a homogeneous environment. However, the workflow setup has a certain level of complexity, and a poorly designed ETL process can cause costly operational problems.

Why are ETL processes important?

ETL processes are the main source of information for the business layer of the company. Sometimes the only one. These processes allow managers to have a clear, detailed and in-depth picture of business management . And thanks to this information, in a later phase of analysis, patterns and trends can be detected on which to make high-impact strategic decisions.

However, the data alone do not add any value, since they are collected in a raw state, which does not allow reading or analysis by people. For data to be valuable, it must be processed and transformed from its raw state to a readable and practical state. This is where the ETL process comes into play.

For example, a customer’s purchase history is made up of, among other things, the following information:

  • Customer contact details and other relevant information.
  • List of items purchased.
  • Details about these items: descriptions, prices, quantities, etc.
  • Final amount of each order.
  • Transactional status of orders (in progress, completed, canceled, etc.).

In order for the purchase history to provide all this information in a clear and legible way, the ETL process has to intervene to obtain and process the raw data until it is adapted to the appropriate formats. This is because the data is not stored in the organization’s systems as it is shown in the history.

If a manager wants to consult the history of a single client, the system should limit his search to that only data (among the huge amount handled by the company), perform calculations, select the fields to be displayed in the history template and give them the right format for their presentation.

Phases of an ETL process

1. Extraction (extract)

The first phase of an ETL process is data extraction. In this, the “raw material” is obtained with which the following two phases will be worked, until a practical utility is achieved.

The data comes from different origins and sources : system records, CRM, points of sale and devices used, among others. It is very likely that these sources are heterogeneous, therefore the data must be normalized. Data containing the same type of information, even if it comes from different sources, must also be collated in common structures.

The objective of the extraction phase is to synthesize all the information in one or more common structures in which the normalized data are stored, leaving them completely homogenized, organized and ready for the transformation phase. It is possible that in this first stage some transformations specific to the data source will be executed. The objective is that these are optimized for the transformation stage.

An example of extraction would be a file of sales data aggregated by months and departments, a row for each department and a column for each month, that is, each new month a column would be added. This format is suitable for handling manual files but very inefficient for a database.

On the other hand, in this technical phase most of the I / O (input / output) operations are carried out, which are usually quite expensive.

2. Transformation (transform)

It is in the transformation phase that we add value to the data obtained in the extraction phase. It is where the foundation of the ETL process lies.

Being isolated from the first, in which technical issues are managed, the business layer can be involved in the development of this transformation, with the aim of transforming the data and adding value.

3. Load

The third and final phase is charging. It is when the system receives the duly processed data and stores it in the destination that has been defined. At this stage, the transformation phase is also isolated from possible changes in the destination.

For example, it is possible that initially the result of the ETL is stored in a file for use by the business layer, but later it is decided to store those results in a database.

In certain environments, issues such as defining, validating, or implementing a data model and ingest processes can take months. For this reason, the business layer can obtain great value by having the possibility of generating immediate results in files and, later, storing them in the database by changing only the loading phase.

Flexible and agile development methodology in ETL processes

The way companies run ETL systems is evolving. Today, everything revolves around an agile and efficient perspective. That is why, according to a Gartner study , 90% of large companies will have hired a Chief Data Officer in 2019. This with the aim, among others, of having staff specialized in ETL optimizations.

All three phases can be developed in an integrated way, especially when the ETL processes are not complex. However, developing them flexibly and independently  provides the following advantages:

  • Each phase can be assigned to different teams to shorten work deadlines.
  • The development of each stage can be done with different technologies.
  • Changes that affect one phase (for example, the change in data source) can be isolated from other phases.
  • Tests can be done at each stage separately, since only the stage in which the test is being entered will be affected.
  • Work done for one ETL process can be reused for other ETL processes, especially the extraction and loading phases.
  • The business layer can be involved in the transformation phase, in order to define the logic of the ETL system, without addressing technical issues that are isolated in the extraction phase and would make it difficult to understand the business layer.
  • The results can be generated in the short and long term without affecting the logic of the ETL process, making changes only in the loading phase.

More Relevant Topics:-

It is important to apply a flexible development approach, since its greatest importance is to be able to involve the business layer in the definition of the ETL logic in order to generate short-term results , without the need to get involved in technical issues.

Read Previous

Shopping experience: why are you unable to retain your customers?

Read Next

Chatbot online: automate your processes with artificial intelligence

Leave a Reply

Your email address will not be published.

Situs Togel 4D Bo Togel Bandar Togel Deposit Pulsa hondatoto Agen Togel Deposit Ovo Togel Bet 100 Perak Bandar Togel Terbesar Toto Togel Bandar Togel Daftar Bandar Togel Terbesar agen togel deposit ovo situs togel 4d Slot Bonus 100 Situs Judi Slot Terbaik Slot Gacor Slot Pragmatic Link Slot Gacor Dingdong Togel Slot Deposit Pulsa Situs Slot Judi Online Agen Slot Online BO Togel 5 Bandar Togel Terpercaya Agen Togel Bet 100 Perak Toto Togel Agen Togel Resmi Bandar Togel Deposit Pulsa Dingdong Togel Situs Togel Terbesar Bandar Darat Togel Togel Toto Online Togel Singapore BO Togel Bet 100 Perak Agen Togel Deposit Pulsa Situs Bandar Togel Hadiah 4D 10 Juta 10 Situs Togel Terpercaya 5 Bandar Togel Terpercaya Situs Slot Deposit Pulsa Agen Slot Online Togel Hari Akun Slot Bet Game Situs Judi Slot Situs Judi Slot Resmi Game Slot Slot Game Bonus New Member 100 Agen Togel terpercaya 10 Situs Togel Terpercaya Agen Togel Online Agen Togel Singapore Togel Toto Macau Situs Togel Terpercaya 10 Situs Togel Terpercaya Situs Bandar Judi Togel Situs Togel BRI Online 24 Jam situs togel dan slot SITUS TOGEL DEPOSIT VIA DANA situs togel terbaik 5 bandar togel terpercaya bandar togel hadiah 4d 10 juta terpercaya BANDAR TOGEL HADIAH 4D 10 JUTA Situs Slot Bet 100 Perak situs bandar togel terpercaya Bandar Togel Deposit Pulsa Situs Slot Situs Slot deposit Pulsa Agen Slot Dan Togel Toto Slot Bandar Togel Hadiah 4D 10 Juta Situs Togel Online Situs Slot Togel Terlengkap Agen Slot BO Togel Agen Togel Online Situs Bandar Togel Dan Slot Situs Togel Togel Deposit Pulsa Slot Gacor Bandar Togel Lengkap Situs Slot 5 Bandar Togel Terpercaya Bo Togel dan Slot Toto Togel Judi togel Togel Hadiah 2D 100rb Bandar Togel Dan Slot Togel Bet 100 Lotre Togel bandar darat togel 5 bandar togel terpercaya Situs Togel Terpercaya bandar judi terpercaya Bandar Togel Terbesar Bandar Togel Online bandar togel bandar togel terpercaya hadiah terbesar judi togel bandar togel terbesar situs judi online bandar togel terbesar Togel Online Bandar Togel Bolak Balik Dibayar Bandar Togel Lengkap situs togel Bandar Togel Togel Hadiah 4d Terbesar bandar togel hadiah terbesar Bandar Togel Hadiah 4d Terbesar judi togel Togel Online Terbaik Situs Togel Terbesar Bandar Lotre Togel Togel Kamboja Dingdong Togel Bandar Togel Terbesar dan Terpercaya Bandar Togel Terpercaya Daftar Bandar Togel Togel Deposit Via Dana 5 Bandar togel terpercaya agen togel terpercaya slot online Situs Bonus New Member Slot Gacor 10 Situs Bandar Togel Slot Online Agen Slot Online Slot Pragmatic Slot Pragmatic Agen Togel Lengkap Bandar Togel Hadiah 4d 10 Juta Slot Resmi Bo Togel Situs Togel Terpercaya 5 Bandar Togel Terpercaya Agen Bo Togel Bandar Togel Slot Pragmatic Judi Slot Pragmatic agen togel online Agen Togel Terpercaya toto togel 5 Bandar Togel Terpercaya Bandar Togel Terpercaya Bandar Togel Online Situs Togel Resmi Agen Slot Judi Slot Online 10 Situs Togel Terpercaya Slot Pulsa Tanpa Potongan Situs Slot Gacor Bo Togel Togel deposit pulsa Agen Judi Slot Online Terpercaya Situs Slot Gacor Mesin Slot Bandar Togel dan Slot Agen Judi Slot Pragmatic Slot Online Via Dana Link Slot Gacor Situs Game Slot Online SITUS SLOT ONLINE GAME SLOT Agen Togel Terpercaya Bandar Togel Slot Bandar Togel dan Slot Bandar Togel Terpercaya Hadiah 4d 10 Juta Bo Togel Bandar Togel Deposit Pulsa Bandar Togel Terpercaya Bandar Togel Hadiah 2d 100rb Slot Pragmatic 5 Bandar Togel Terpercaya Bandar Darat Togel Bandar Togel Dan Slot Bandar Togel Hadiah 2d 100rb Bandar Togel Hadiah Terbesar Bandar Togel Lengkap Bandar Togel Online Bandar Togel Terbesar Daftar Bandar Togel Tertua Agen Slot Bonus 100 Agen Togel Mesin Slot BANDAR TOGEL HADIAH 2D 100RB Bandar Togel 77 TOTO SLOT 5 Bandar Togel Terpercaya 5 Bandar Togel Terpercaya 2022 Situs Judi Slot Terbaik Bandar Darat Togel Bandar Lotre Togel Bandar Togel Bet 100 Rupiah Bandar Togel Bet 100 Bandar Togel Hadiah 4d 10 Juta Terpercaya BANDAR TOGEL HADIAH 4D 10 JUTA Bandar Togel Terpercaya BANDAR TOGEL TERPERCAYA HADIAH TERBESAR Bandar Togel Terbesar BANDAR TOGEL LENGKAP Bandar Togel Hadiah Full Togel Online Bandar Togel Resmi Bandar Togel Terbaik Bandar Togel Terbesar Situs Slot Online Bandar Judi Slot Online BANDAR JUDI SLOT ONLINE BANDAR TOGEL TERPERCAYA KUMPULAN SITUS JUDI SLOT 4D Situs Bandar Togel Hadiah 4d 10 Juta LINK BANDAR SLOT TERPERCAYA Daftar Bandar Togel Terbesar Dan Terpercaya Daftar Bandar Togel Tertua Dingdong Togel Daftar Situs Judi Slot Online BANDAR TOGEL TERPERCAYA AGEN JUDI SLOT ONLINE TERPERCAYA DAFTAR JUDI SLOT ONLINE Slot Pragmatic BO JUDI SLOT ONLINE DAFTAR SITUS SLOT GACOR Nama Nama Situs Judi Slot Online Terpercaya Togel China Bandar Judi Slot Online SITUS JUDI SLOT DEPOSIT PULSA Situs Slot Deposit Pulsa Togel Kamboja Agen Togel Resmi AGEN SLOT ONLINE Slot Bonus New Member 100 LINK BO TERPERCAYA SITUS SLOT GACOR Slot Gacor Agen Judi Slot Online Resmi DAFTAR NAMA NAMA SITUS JUDI SLOT ONLINE DAFTAR SITUS JUDI SLOT ONLINE AGEN TOGEL RESMI DEPOSIT PULSA Toto Slot Agen Judi Slot Gacor SLOT ONLINE KUMPULAN NAMA SITUS SLOT ONLINE TERPERCAYA Bandar Togel dan Slot GAME SLOT ONLINE Togel Bandar Darat BANDAR TOGEL TERBESAR TOGEL HADIAH PRIZE 123 BANDAR TOGEL TERPERCAYA Agen Slot Online Situs Slot Deposit Pulsa Situs Judi Online Situs Slot Online Situs Judi Togel Situs Judi Slot Online Terpercaya 5 Bandar Togel Terpercaya Toto Slot Agen Judi Slot Online Situs Slot Terbaru Togel Deposit Pulsa Slot Online Situs Judi Slot Online Terpercaya Slot Pragmatic Bandar Togel Bet 100 Perak Situs Slot Slot Deposit Pulsa Slot Mpo Situs Judi Slot Online Slot Online Terbaik dan Terpercaya Game Slot Agen Slot Pulsa Slot Gacor Agen slot dan Togel Agen Judi Slot Online Terpercaya Togel dan Slot Agen Togel Slot Gacor Slot Gacor Agen Judi Slot Online24Jam Slot Gacor Judi Online Judi Slot Online Bo Togel Slot Online Sultanplay Situs Judi Slot Online Situs Mpo Slot Gacor Slot PG Soft Slot Online Sultanplay Situs Togel Online Agen Judi Slot Online Situs Togel dan Slot Online 10 Situs Togel Terpercaya 10 SITUS TOGEL TERPERCAYA Slot Online Terpercaya Joker Slot Bandar Togel Dan Slot Bandar Togel Hadiah Full Bandar Togel Hadiah Terbesar Bandar Togel BANDAR TOGEL TERPERCAYA Akun Togel Resmi Togel Macau Togel Kamboja Slot Deposit 5000 Slot Terpercaya Togel Singapura Toto Macau Nama Nama Situs Slot Online Bandar Togel Kamboja Bandar Togel Terpercaya Togel Hadiah 4d 10 Juta Terpercaya Agen Slot Agen Judi togel Resmi Situs Togel Terpercaya Situs Togel Terbesar Bandar Toto Terpercaya Agen Slot Gacor Togel Hongkong Judi Slot Online Hondatoto Ojktoto agen slot agen togel resmi togel online agen togel terpercaya agen togel slot online bandar togel situs slot bandar togel online judi slot agen bandar togel agen togel situs judi slot slot online 5 bandar togel terpercaya bo togel terbesar agen slot judi togel online agen togel terpercaya situs togel terpercaya slot online bo togel slot online bo togel bo togel agen togel online agen togel online bo togel deposit via pulsa bet 100 tanpa potongan agen togel terpercaya toto slot bo togel bo togel idn slot bo togel judi togel online slot gacor murah slot situs judi slot situs togel resmi daftar bandar togel tertua bo togel wap togel aman bandar togel resmi bo togel situs slot situs slot togel online bandar togel terpercaya 10 situs togel terpercaya judi slot online situs slot gacor situs togel deposit via dana situs slot online terbaik 5 bandar togel terpercaya situs slot daftar togel bandar togel lengkap agen slot dan togel terpercaya bandar slot terpercaya dingdong togel agen judi togel situs togel situs judi slot situs togel bandar togel situs togel terpercaya bo togel hadiah terbesar togel 100 perak bo togel terpercaya seratus bandar togel togel 100 situs judi slot online togel hongkong togel sdy situs togel slot 4d bandar togel terbaik togel singapore situs togel terbaik agen togel resmi situs judi slot online bandar togel hadiah 4d terbesar situs slot deposit pulsa agen judi slot online agen judi slot gacor online situs judi slot online togel macau togel macau nama situs togel terpercaya situs bo togel togel kamboja slot mpo situs judi slot terbaik situs judi slot online situs togel situs agen togel judi slot bandar togel online situs judi slot toto sdy togel deposit pulsa 10rb tanpa potongan togel taiwan situs togel resmi toto situs togel terpercaya daftar bandar togel situs togel resmi