Sunday, February 19, 2012

Load 500,000 rows in 100 minutes?

My friend is a data analyst. He frequently loads the data from Access into MS SQL Server using OLE DB. Yesterday he asked me if there is a fast way to load data. I told him to save Access data into csv, then  using bcp to insert into MS SQL Server. He said he will try bcp, since it makes him crazy to load 500,000 rows in 100 minutes using OLE DB.

Hold on a second, load 500,000 rows in 100 minutes? I am pretty sure OLE DB is not the primary reason. I suspect there are indexes and constraints on the table, which causes the problem. I suggested him to remove indexes on the table before loading the data, and re-create indexes after the load.

OK, here is my conclusion: when a user asks a question, please have a better understanding why he asks this question. That is the way of "good communication".

No comments:

Post a Comment