Some our our Store Locator Plus customers are loading huge lists of locations.   Power 4.9 was reworked to make this process far more efficient and prevent browser connections from being weak point in the process.  This has helped thousands of customers import larger lists than ever before.   Configuring your web server and PHP may be the “secret sauce” you need if your site is still having import issues.

Luckily our MySLP users do not have to worry about this as our MySLP servers are already tuned to handle large location lists.    For those of you on WordPress and managing your own plugins , it may be time to talk to your hosting company.

Import processing in 4.9

With Power 4.9 , which is bundled in our year-end update for MySLP users, launched with an entirely new process for importing locations.    Previously the import was a one-step process which tied up your browser.   If your network connection went down, you stopped the browser page loading for any reason, or your allowed server-to-browser response time (usually 120 seconds) was exceeded the import would stop.   That was not a good thing.    In Power 4.9 we broke the process into 3 steps to give the server far more time to deal with the slower parts of the import process such as getting the data from the CSV file into the database and asking Google for latitude & longitude data for every site.    It is not only faster but more resilient.   The files typically upload to the server quickly and then our data-loading process takes over.   It continually checks to make sure the import is still running and if not auto-restarts.

This process works well for most servers and most location lists.   Customers are easily importing thousands of locations without any issues.  Even on under-powered servers.     Some customers, however, are still having issues when importing tens-of-thousands of locations.     After a lot of trial-and-error we finally reproduced the problem on our internal development servers and learned a lot in the process.

Importing locations background

A customer was having issues importing a list of 25,000 locations on their server.    Many of the improvements in Power 4.9 came about as we worked with this customer and several others processing tens-of-thousands of locations on a daily basis.    It did not take long to refine the import process and get more than 10,000 locations to load into the server in less than the 30-second PHP maximum execution time allowed by some hosting services.   A problem was still lurking in the background.  One that cannot be fixed by clever coding alone.

It turns out that if we used the default tuning parameters for our nginx server on our local VVV development box we could simulate what the customer was seeing on their import of 25,000 locations.    We did have to set our PHP time limit to 30 seconds as forcing the fatal PHP error for Maximum Execution Time Exceeded was a critical trigger for this process.   It also turns out we needed to be running the slower PHP 5.6 version with FPM enabled as well.

The problem?

When the PHP maximum execution time is exceeded the program, in this case WordPress processing a file import, crashes with a fatal error.   Since fatal errors have no way to be caught, you just have to let it happen.  Luckily Store Locator Plus is smart enough to check in on import progress every few minutes and pick up the pieces after a crash like this.

Sometimes this would not happen.   In fact when the import did not continue the entire web server would stop responding.  Sometimes permanently. Sometimes for 10 minutes.    It was odd.

What has happening?   The web server and PHP were fighting.    Turns out that the fatal crash from maximum execution time was throwing the web server into  a tailspin because the web server would keep trying to run more background processes to serve more web requests while PHP was trying to get out of the mess that was created.

Web server and PHP limits

We quickly learned that if we restarted the PHP 5.6 FPM process on our server everything would come back to life.  Almost instantaneously.   The web pages would start coming back up and , as if  by magic, the location import process would continue right where it left off.     Running this simple server command fixed the problem every time:

# sudo service php5.6-fpm restart

But why?  Why do we need to restart PHP every time an import stops?

It is not the import itself but a lower-level PHP issue.

Nearly all production servers set a fixed amount of time that a program is allowed to use up in CPU resources.  Smaller shared hosts may give your program 30 seconds of allowed CPU time to finish all the work it needs to do.    Our development server is only for us so it allows 600 seconds which is eons in computer time (enough to process a 500,000 location import with time to spare).     This is the “safety valve” for your PHP applications to ensure that a single program cannot take over an entire server.  It could be a coding error running in an infinite loop or a rogue application doing nefarious things.  Either way you don’t want one program taking over the universe , so the limit prevents that.     It is also a HARD limit meaning once it is reached PHP , quite abruptly, pulls the plug.  NO warning.  No “hey, you may want to stop now”.   BOOM.  Done.   The program crashes hard at whatever point it happens to be at during the import.

Properly configuring your web server and PHP can help

Normally when this happens the process ends and all other programs go on their merry way.     However this requires some choreography between the web server and PHP to make this work well.     One of the issues that often comes up when you reach a maximum execution time error is that one of your web server’s PHP processes has been busy for quite some time.   That means one process has been taken up an not allowed to send web pages to people.    Your web server, as you can guess, does not rely on a single process but several processes to send pages.  It uses one of the backups.     But what happens when 10 people show up at once?  For example when Jetpack runs its every-5-minute background checks?   Or someone runs a social media publishing campaign?    All the extra processes available to serve a web page can be quickly used up.

All web servers have a process manager that handles these things.    They also have memory limitation settings for each process.   Your server also has a limited amount of memory available to do all this work.       If all 3 elements are not configured intelligently all hell can break loose.

Checking the nginx process manager is critical.     The process manager (pm in apache or worker in nginx) determines how many connections are “ready at all times”.    This is your initial connections allowed.     There is also a setting that determines how many “children” or extra processes can be started up as needed.    There are even special modes to tell the process manager about how to behave in general such as “worker mode” or “on demand”.      Your system administrator should know enough about your server to ensure that there are enough processes ready at all times without using up all the server memory, which will make everything ground to a halt in a hurry.    They also should know how much overhead your server needs for “breathing room” for “janitorial stuff” which they factor into how many extra children can be created to handle a web request.

Knowing your memory limits and usage per web request is also important.   This is where your PHP configuration and nginx configuration need to be intelligent about what is going on.  Many system administrators run into a limitation and “crank up” the memory limit to “10”.    On many servers this is an issue.   If your server has 16GB of RAM, which is a BIG hosting plan with most web hosts, you cannot tell PHP to allow 512MB per process and then tell the nginx process manager to start 16 processes but allow it to grow to 48 as needed.    If your processes each used up all of the allowable 512MB of RAM then your server runs out of memory and everything stops working.

Your system administrator should know how much memory your typical web requests uses.  They should know how much peak memory is used by the most intensive operations you may run, such as a nightly import or backup process.    They should set your PHP memory limits to a reasonable value based on this information.   They can work backwards from there to determine the maximum number of processes your web server should allow EVER.

They will also need to have some general information about your typical web traffic.  How many simultaneous visitors do you have on average and how long does it take to serve up a web page to them?   If you get 10 visitors per minute but each page only takes 1 second to render then having a handful of processes ready at all times is all you need.   If you are getting 100 visitors per minute and you take 5 seconds to show them the page you’ll need a lot more processes ready at the start.      Do you have high fluctuation where you get 10 visitors per hour most days but your coffee shop locator page is slammed with 300 visits per minute at 7AM?   You need to configure your starting processes at a lower value but allow a large number of children to be spawned.

Want to get “all techie” about it?  Keep an eye on my personal blog over at Lance.BIO where I share more details tech-geek stuff.

Why is all of this important?  How does it relate to imports?

If you are doing an import of tens-of-thousands of locations and your server hits that maximum execution time limit it should go unnoticed.   If everything is tuned properly your server will stop the import process when the resource limit is reached and Store Locator Plus will start a new process within a minute or two and continue where it left off.   If things are NOT configured properly then your web server and PHP start fighting and everything hangs.    Your admin pages stop responding and your import never finishes.      If you ask your web host to restart the web server and the import continues it is a good indicator your web server is misconfigured.

Workarounds

How do you get around the issue if your do not have a managed hosting plan or a cooperative ISP?

For Store Locator Plus imports you can cut your file size down into smaller chunks.      Only import one at a time and when it finished go on to the next piece.

You can also ask your hosting provider to increase the maximum execution time for your hosting plan.   You should be allowed to have at least 60 seconds to process files and not have a significant impact on the server.   60 seconds on a decent web host should allow 20,000 locations to be imported.

Still having issues? Consider changing hosts to someone with a bit more experience or considered our MySLP SaaS service.