Is Hand-Coded ETL the Way to Go? Absolutely Yes, or Absolutely No, Depending…

This article originally ran in May 31, 2003 edition of Intelligent Enterprise magazine, written by Gary Nissen, edited by Ralph Kimball.

Recently, there was a question posted on a data warehousing e-mail list that is frequently asked, yet always surprises me. The question asked whether it is acceptable or recommended to hand-code the Extract-Transform-Load (ETL) logic for a data warehouse or if tool-based ETL is always preferred.

My standard answer is that it is perfectly appropriate to custom develop the ETL code. In the right situation, custom developed ETL can be cheaper and more maintainable. But in other situations, a tool-based approach is better. Just like any other technology decision, it depends. It all comes down to understanding the scope and requirements of the project.

I guess what surprises me is that this decision is no different than any other technology decision in any other development project. You simply consider all significant requirements, balance your options and make a business decision. In other words, just follow your standard project plan and it will lead you to the right decision. MORE>