With the release of the WordPress plugins version 4.9 of Store Locator Plus and the Power add on we focused on improving the location import process for large lists. MySLP Power users will see this release by the end of the year after it has passed full testing on our SaaS platform.
Why Did We Change The Process?
Prior to version 4.9 there were a number of limitations that could prevent larger lists of locations from importing properly. For the fastest servers with lots of memory and super-fast networking as well as a premium tier on the Google API service, Store Locator Plus could import 40,000 locations without any problem.
As WordPress started adding more network overhead, all those update notifications and the news feed in the WordPress dashboard comes from somewhere, the network started to become an issue. On top of that Google started tracking every single map request so they could start charging people using their maps on a per-lookup basis. Both of these changes increased the network traffic, slowing down how quickly Store Locator Plus could get a latitude/longitude for those new locations being imported.
Often your server would give up, saying “you had your 30 seconds of my time” and stopping the import mid-stream. The PHP time limitation on processing is often the first thing that “gives out” when importing large lists of locations. For most shared hosts the limit is 30 seconds. Even on a dedicated host it is good practice to not let a PHP process consume more than 120 seconds of CPU time. This prevents “runaway” processes that take over the entire server which can grind EVERYTHING to a halt forcing a server restart.
The bad news is that if you are importing 10,000 locations and the process gives up after 9,800 locations you have to start over. At least that is how it USED TO BE with SLP 4.8 and earlier.
How Imports Improved In Power 4.9
With version 4.8 the location import process was “tied to the browser” until it was complete. When you clicked the “upload CSV” button your browser would sit there attached to the website while the file was uploaded, read line-by-line, each location sent to Google to get a latitude and longitude and then get added to the Store Locator Plus locations list. If the network was slow this could take a long time, often 10 minutes or more. That is FOREVER to keep a browser window open these days.
With Power 4.9 we split this process into 3 stages.
Stage 1 : Get The File On The server
This is the only step of the process that stays “attached to your browser”. The browser will bar processing the page while the file is copied from your laptop to the web server. This is typically a quick process even for a file with 250,000 locations. The time it takes depends on how fast your local network and ISP connection is.
Previous versions stored the uploaded location files in a “hidden” SLP-specific directory under WordPress uploads. Version 4.9 stores them in the media library. This gives you an easy way to see that the file was uploaded properly, see what the current import state is, and later download or delete the files when the import is complete.
Stage 2 : Import The Locations Into SLP
After the file is uploaded a detached process starts to work on the actual location import. This ensures your browser doesn’t time out as this part happens in the background. Your browser is “free to go” while your server does the work of loading in the locations.
It runs a WP Cron job in the background to import the locations WITHOUT geocoding. This is much faster and ensures the import doesn’t disconnect partway through.
As part of improving visibility into this process, Power 4.9.1 will be out soon and will be included in our end-of-year MySLP update. It includes a cool interactive progress bar as well showing you how many records have been read.
If your server times out before our background location loader finishes reading the file , SLP remembers the last record read and re-spawns a new background job to pick up where it left off. It will do this until it reaches the end of the file no matter how often PHP times out.
Stage 3: Geocode The Locations
AFTER the file has been loaded it will then geocode all uncoded locations. That means you may catch SLP with a list of locations none of which are geocoded and not showing up on the map. This is a temporary state while SLP talks to Google. Since that is always the slowest part of an import we decided to do this in a separate process so at least SOME locations start showing up right away and eventually of them are loaded.
This is similar to the import. It does it in the background and if PHP times out it will re-spawn itself until it has tried to geocode very location at least once. This is a big step toward not having an import time out. It also keeps you from manually having to re-geocode if you hit your daily 2500 location limit and didn’t opt for a paid API key.
