Working on the data warehouse project, that guy has given us a tutorial that we define a lot using SQL queries There is a lot of data flow changes, ETL boxes such as quotation marks will consume a lot of memory so that we can leave the processing in the DB box. Is this really advice? Where is the balance between being dependent on the GUI tool when executing a bunch of SQL scripts on your Unity package?
And honestly, I want to avoid writing SQL queries as much as I can. (But it is next to the point. I really want to see this fairly.)
The answer is: It depends, but you want to pick one or the other for any job and avoid the two blends wherever possible.
Generally, it is best to do everything possible within the device as much as possible. Stored procedure code when you have a large amount of logic split between layers, then system trace and debug Becomes harder for them.
-
Where data can change without data flow can be weird and complex, so that you can use tools and questions have little or no logic. This means that there is a business logic in a single layer and it should clearly know where to find it. However, ETL tools are capable of handling highly complex changes in a relatively poor manner. For such an approach, the sweet place is on the system where you have a large number of data sources but relatively simple change
-
If you have relatively complex changes, then you may be better able to put all business logic and conversions in one layer of stored procedures. SQL code is better in implementing complex changes in a variable way - I have a very good authorization that almost all data warehouse projects in banking and insurance sectors use this type of architecture for exactly the same reason.
In this case ETL equipment can be used to implement relatively dumb data copies. The source data can be copied to the staging areas, essentially verbatim and then stored procedure can be picked up by a body of code which ETL does. ETL equipment can be used for data copies, bulk load operations, logging, scheduling and other framework functions.
In either case, you are better off choosing an approach. Otherwise, you can end up with business logic extracted in extraction layers, database views, data flow and stored procedure code. . The argument spread in many layers is very difficult.
When all logic (for example) is included in stored procedures or ETL focuses on conversion actions, then you can test the unit conversion given in isolation. Design also helps with clarity maintenance and auditing.
Comments
Post a Comment