EO Settings Import/Export tab "Fatal Error" when iCal Extension is active

[John Sundberg]

Stephen,

On November 5 I updated Event Organiser to 2.9.2 from 2.8.5 on a client website. At the time I was using the iCal Extension 1.3 to import a Google Calendar feed, though as of today I’ve updated iCal to 1.4.1.

The client contacted me today to say that their website calendar was no longer syncing with their Google Calendar. When I went to the Import/Export tab of Event Organiser Settings, the only things on the page were the “ICAL Feeds” heading and this error message:

Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 64 bytes) in [ … ]/wp-includes/wp-db.php on line 1977

The primary error log for the site is showing that these errors began on the same day that I updated to EO 2.9.2, and it’s logging multiple errors per day.

I’ve gone through the code for 2.9.2 and compared it to 2.8.5 but I haven’t been able to find what change might be causing this.

Also, when I deactivate the iCal Extension the error disappears and the Import/Export tab resorts to normal, though without the feed import interface. That might cause me to think that iCal is causing this, except that it was working previous to the EO 2.9.2 update.

WordPress: 4.0.1
Event Organiser: 2.9.2
EO iCal Extension: 1.4.1

Thanks,
John

[Stephen Harris]

This might be a case of the straw that broke the camel’s back in terms of memory allocation.

If it were just causing the feed sync that was breaking I’d be inclined to think it were the feeds being imported in were so large so as to exhaust a lot of memory – and that perhaps something should be done (if it could) to ‘spread’ the load across several request. But the fact that this is happening on the settings page were relatively little is being done, makes me think that perhaps its simply a case of there being a high number of plug-ins, or memory-hungry plug-ins to the point where under the current settings, the server cannot cope.

This can happen occasionally – so you may want to see if following the recommendations here fix the problem: https://github.com/stephenharris/Event-Organiser/issues/206/.

A briefy summary of that link – it essentially details how you can increase memory allocation to PHP, and also recommends a plug-in for trying to determine if there are any particular themes/plugins that are consuming a lot of memory.

[John Sundberg]

Got it working – here’s what I had to do:

This site, like all my sites, has a 256M memory limit defined in wp-config, my dedicated server has memory_limit for scripts set at 256M, and the server itself has 64GB of RAM, 24 CPUs, hard drives are SSDs, etc. … so I don’t think it’s a server performance issue in this case.

I installed the Query Monitor plugin on this site, and it’s showing that each admin page is using an average of 55MB of memory, except the EO Settings page > Import/Export tab, which wouldn’t even allow Query Monitor to finish loading because of the fatal error on that page/tab.

I increased the server’s memory_limit to 512M, and had to use define( ‘WP_MAX_MEMORY_LIMIT’, ‘512M’ ); in wp-config.php to increase the admin memory limit.

At this point the Import/Export tab loaded, but Query Monitor showed that it was using 271MB of memory. There was also an EO error in the feed UI, [Line 211] Invalid date “”. Date expected in YYYYMMDD format.

I downloaded the .ics file for the feed and looked at line 211. This was part of a recurring event, and the date for that line was empty. EXDATE;TZID=America/Denver:

(Perhaps we need a code tweak so that an error isn’t thrown when an exclusion date is empty? Or would that cause other problems?)

I logged in to the client’s Google Calendar, deleted that recurring event, and re-added and edited it by removing the excluded dates.

Next I trashed and deleted all the events in Event Organiser.

Finally, I went back to Import/Export and tried to delete the existing feed. I had to try multiple times before it stayed deleted, and I have no idea why that was happening. I then added a new feed with a different name and the same source.

The old feed is finally staying deleted(?), the new feed doesn’t have any errors, and the memory usage of the Import/Export page is right around 54MB.

Not sure what the take-home is from this experience, or what, if anything, needs to be changed?

[Stephen Harris]

The error itself “[Line 211] Invalid date “”. Date expected in YYYYMMDD format” shouldn’t cause the problem you describe. It’s not an uncaught exception, but is just a logged error. In fact the plug-in would have stopped parsing at that point, and ignored the offending event.

So it’s not clear at all why it would cause any issues.

Perhaps it might be wiser to abort the entire feed import when an error in the feed is found?

[John Sundberg]

Here’s a screenshot of the page after I increased the memory_limit to 512M: https://db.tt/Lswz654V

Somehow, I think, this was causing the super-high memory usage which caused the fatal error I mentioned previously. After I fixed the event in Google Calendar things returned to normal.

If aborting the feed import, could we trigger an email to the admin?

[Stephen Harris]

Well I’m glad it’s sorted, but I’m still stuck as to the actual cause. If it does happen again, if you’re able to download a copy of the iCal feed that would give me something to test against. In the mean time I’ll add a unit test for a missing “EXDATE” to see if I can replicate the problem.

[John Sundberg]

I’ve still got the basic.ics file that caused the problem. Would it help if I sent you a copy of that?

[Stephen Harris]

That’d be great. Think you might have my e-mail, if not you can just use this form: http://wp-event-organiser.com/contact/

[Stephen Harris]

Thanks, received – will report back if I find anything.

[Stephen Harris]

Hi John,

As expected, the error on line 211 had no affect on the plug-in (other than the expected error message that this line did not contain a date in the expected format). I was able to replicate the memory limit issue, indeed tracking the memory usage I that:

• By the time the feed was parsed, 5% of allocated memory had been used
• Each event took ~0.2% – 0.3%
• At just over 300 events when the feed had completed the sync ~87% memory had been used
• The server than ran out of memory, while executing script in Query Monitor (a debugging plug-in I use).

However, after deactivating Query Monitor, I found that each

• By the time the feed was parsed, 4.1% of allocated memory had been used
• Each event took negligible amount.
• At just over 300 events when the feed had completed the sync ~6% memory had been used

This time the server did not run out of memory usage.

Predictably the performance of syncing 100s of events is greatly affected by any other plug-ins/themes running. In particular, I have no doubt that it is the updating a lot of events and the subsequent triggering of save_post which might (in general) be the culprit.

Raising the memory limit will treat the immediate problem, and although I think this situation is rare there is still the question of what (if anything) should I do to help avoid the situation. I’ve come up with two potential solutions:

• Use a hash to identify when an event needs updating, and so skipping the ones that do not need it. (Though in some circumstances, this could increase memory usage).
• Updating in batches. This might not be possible on some servers. It would essentially involve temporarily storing the parsed iCal feed (in json format say) locally, and then spawning a series of requests each updating a batch (of 100, say) events.

I’ll look into the feasibility of these two suggestions, but for the time being the recommendation would be to increase the memory limit.

[Author]

Posts