Bulk impoprting CSV files

abruck's profile image abruck posted 4 years ago in General Permalink

Is there a way to load several .csv files in batch on a Windows 10?

I have 58 csv files I want to load into a table.

8+GB of data.

What I concat them all together, it fails. It processes for about 20-30 seconds and then gives me the "An error occurred the application" window. (I sent the bug report and I have attached it)

1 attachment(s):
ansgar's profile image ansgar posted 4 years ago Permalink

You should prefer the "Server parses files contents" method, which is normally way faster and less memory consuming.

You may also compose the LOAD DATA commands yourself, but first retry the import dialog.

abruck's profile image abruck posted 4 years ago Permalink

I tried "Server parses ...' but still didn't like it

This worked (and really fast):

LOAD DATA LOCAL INFILE '/path/to/your/csv/file/model.csv' INTO TABLE test.dummy FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';

abruck's profile image abruck posted 4 years ago Permalink

Just a note to anyone who will be using the LOAD method:

In windows (not sure about linux) you will need to double the "\" slashes in the file PATH: \

ansgar's profile image ansgar posted 4 years ago Permalink

The "Server parses..." method composes and fires such a LOAD DATA command. You probably had different control characters in the import dialog.

Please login to leave a reply, or register at first.