Cannot import more than 500 records

The number of records import at one time is defined in modules/Import/ImportStep2.php

$_SESSIONrecordcount? = 500;

File: modules/Import/ImportStep3.php deletes the data file used for importing after reading the contents first time. So the second invocation of import will fail to import any more data.

The data file used for import should be deleted only in the last step.

vtiger-504-import-records.patch

I have found the fix for this issue and have attached vtiger-504-import-records.patch this ticket. Kindly validate it further.

I applied the patch to two different 504 installations today. The fix seems to work fine :-)

This issue has been fixed and integrated. Please refer to the changeset #12068

This is still a problem in version 5.1.0

You may change the /modules/Import/ImportStep2.php line 45 from

$_SESSION['recordcount'] = 500;

to

$_SESSION['recordcount'] = 7000;

Make sure that php.ini (at least on Linux) can execute for a long and time and is allowed to use quite some memory:

My /etc/php.ini

max_execution_time = 6000     ; Maximum execution time of each script, in seconds
max_input_time = 6000   ; Maximum amount of time each script may spend parsing request data
memory_limit = 2048M      ; Maximum amount of memory a script may consume

we are not able to reproduce the issue with vtigercrm 5.1.0 GA, it would be good if you can share more details perhaps with the file. so we can nail down the issue.

Last month when we were importing lot of data, we faced this problem many times in many modules [accounts, contacts, service contracts, etc]. Though there is no consistency as the same module appears to work fine at times. The way it behaves is import 500 records fine and then keep refreshing the page again and again till the all the records are complete [for example if there are 3000 records it will refresh 6 times, with correct success message, but importing only first 500].