Recently, I had a strange issue where some pages on my multisite were being cached, and others were not. There seemed to be no clear pattern as to what got cached and what failed the cache test.
At first, it seemed a bit daunting to jump into the w3tc page caching code, but it actually turned out to be a somewhat fun and surprisingly painless process. I talk about my debugging efforts here, as well as how the w3tc page caching process works under the ‘Page Enhanced’ cache setting.
Note – This is an advanced tutorial where I make changes to core plugin files. Do not attempt any of this unless you have ftp access to your server, and are very comfortable with restoring plugin files back to their original state.
w3tc Page Cache – How It Works
The first line of redirection is through the server .htaccess file. In particular, if a cached file already exists, rewrite rules in our domain root directory will point a page request directly to the appropriate file.
In version 0.9.2.8, cached files are kept in
wp-content/cache/page-enhanced/mysite.com/
The rewrite rule looks something like this –
RewriteRule .* "/wp-content/cache/page_enhanced/%{HTTP_HOST}%{REQUEST_URI}/_index%{ENV:W3TC_UA}%{ENV:W3TC_REF}.html%{ENV:W3TC_ENC}" [L]
advanced-cache.php
If the cache file does not exist, then WordPress starts up and the page caching process starts with the wp-settings.php file. In particular, the file should contain the following lines of code-
// For an advanced caching plugin to use. Uses a static drop-in because you would only want one. if ( WP_CACHE ) WP_DEBUG ? include( WP_CONTENT_DIR . '/advanced-cache.php' ) : @include( WP_CONTENT_DIR . '/advanced-cache.php' );
If our site is not caching any pages at all, then we may want to check the wp-settings.php file to see if the cache code above is present. This code is necessary for page caching to work in w3tc.
Sometimes, the advanced-cache.php file may be missing from our wp-content directory. When this happens though, there is usually a notice in the w3tc ‘Performance’ menus.
At the bottom of the advanced-cache.php file, an instance of the W3_PgCache object is created, and the process function is run to handle caching for a page request.
W3_PgCache Object
The W3_PgCache object is stored in the w3-total-cache/lib/W3/PgCache.php file. It contains the guts of the page caching process, so this is where we will be concentrating most of our effort.
I start my debugging process by opening the PgCache.php file, and then putting in a debug echo statement at the top of the process function. I encapsulate my echo messages within HTML comment markers so that it does not disrupt layout and content of my site pages.
echo "<!-- Start of process function -->";
If I am running on a multisite configuration, then I use blog_id to limit debugging to only a test sub-site.
if (w3_get_blog_id() == 2) {
echo "<!-- Start of process function -->";
}
In general, it is better to do our testing and debugging on a test site, but since I could not replicate the problem I was having on one of my live sites, I had no choice.
Next, I continue to move down my debug test statement until I reach a section where it fails.
Note – If we are in the ob_callback function, then we cannot use echo, and need to append our message to the current buffer instead.
if (w3_get_blog_id() == 2) {
$buffer .= "<!-- Ob callback -->";
}
My page source looks something like this –
<!-- Start of process function --><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" lang="en-US" xml:lang="en-US"> ... <!-- Performance optimized by W3 Total Cache. Learn more: http://www.w3-edge.com/wordpress-plugins/ ... Served from: mysite.com @ 2013-03-28 06:06:34 by W3 Total Cache --> <!-- Ob callback --> <!-- W3 Total Cache: Page cache debug info: Engine: disk: enhanced ... Content-type: text/html -->
Depending on the location of my debug statements, the messages could appear at the top or bottom of my html source file.
Why Some Files Were Cached but Not Others
Finally, I tracked the problem down to the _is_cacheable_content_type function. It was failing because there is a case-sensitive check for the ‘Content-Type’ string in the headers_list. The problem is that sometimes, the headers list contains ‘Content-type’, with a lowercase t.
Once I located the issue, the fix was really simple – I changed strpos in the _is_cacheable_content_type function to stripos.
if (strpos($header, 'Content-Type') !==false) {
now becomes
if (stripos($header, 'Content-Type') !==false) {
Now, all the files are properly cached.
Other Useful Debugging Tools
When debugging on a multisite setup, it may also be helpful to turn on debugging information for just a single test blog. I did this by adding the code below to my theme init action hook.
// Add to theme init action hook
if (w3_get_blog_id() == 2) {
$w3_pgcache = w3_instance('W3_PgCache');
$w3_pgcache->_debug = TRUE;
}
Happy Debugging!
Hmm. So I added the line
$buffer .= “<!– DEBUG: $can_cache –>”;
to line 332 (right after $can_cache is defined), but nothing appears in the html. Did I misunderstand how to get output from within ob_callback?
Do you know where I can stick a debug line to get the output of $this->cache_reject_reason from _can_cache2()?
I did submit a query to the author but have not heard back as of yet. I also posted on the WP support forum. I’m trying every avenue I can think of, as I try to sort through the code, etc.
If you’d be willing to take a peek, I absolve you of any risk. I promise no tears, fists, angry words, or recrimination (I know you’d make a copy of any file before you edited it, and I can live with a couple of seconds of downtime while troubleshooting!). Email me and I’d be glad to set you up with an ssh account! I do understand if you don’t want to take on the risk though.
Thanks again,
Daniel
Try adding a debug message at the beginning of the ob_callback function, something like –
$buffer .= “<!– Start ob_callback ” . $this->_is_cacheable_content_type() . ” –>”;
It could be that we are failing on the condition in line 328 and so we never get to line 332.
In general, if nothing is showing, then likely the fail point is somewhere above it.
Well, I added the debug code as line 328, and nothing got output to the html. But ob_callback has to have been called, right?
ob_callback is called on line 214, but if I try adding
echo “_is_cacheable_content_type() . ” ->”;
above it, php pukes. If I use
echo “_debug . ” ->”;
there instead, it outputs as expected. Which makes sense, since _is_cacheable_content_type() is a private function, now that I look closer. And adding
$buffer .= “<!– DEBUG: ” . $this->_is_cacheable_content_type() . ” –>”;
to the end of the function (before the $cache_headers variable is set) adds nothing to the html either.
Still hammering at it…
Daniel
To call the function, you need to do –
$this->_is_cacheable_content_type()
because the function is part of the object class.
That is very strange. I would try just adding a simple debug line and see if it appears.
$buffer .= “<!– Start ob_callback –>”; return $buffer;
If not, then there may be something deeper going on than just the page cache. The ob_callback functions all get called in the TotalCache.php file, in line 1351.
Ok, that’s weird – TotalCache.php only has 823 lines in mine. And I’m running the latest version.
I’m going to try uninstalling w3tc and then reinstalling it and see what happens. I have a backup of my settings.
Hello. Would you be interested for a price to help me fix my w3tc issue? I am running nginx instead of apache, but the problem seems to lie somewhere in the wordpress/w3tc code.
No matter what I try, page cache is not working. Pages are not getting pre-cached. They aren’t being cached when hit. Except – and this is really weird, feeds – the only _index.html files that are being generated are for feeds.
I tried every combination of options in w3tc, and reset it to the defaults. I stepped through some debugging thanks to your post, and this is where it gets *really* weird – If I view source in the browser, there is no indication at all that w3tc exists (although it is doing some good right now with object/database caching). But if I enable some debugging echos in w3-total-cache/lib/W3/PgCache.php, those lines do show up in the source code!
w3tc used to work just fine, and I don’t know, after all this frustration, how or why it happened. But it is seriously impacting site performance for sure.
Thank you for considering assisting.
Daniel
Yeah, currently, the debug messages from w3tc only get tagged on at the end, so if some condition fails in the interim, then they won’t show up.
This is why I do the echo messages within the page caching process, it is a simple (albeit primitive) way to pinpoint which exact condition the cache is failing on. Another possibility is to set up a PHP debugger.
http://stackoverflow.com/questions/888/how-do-you-debug-php-scripts
Are you using the most recent w3tc version? Do you have a test site that is also showing this behavior?
Interesting site btw.
I am running the most recent version. I don’t have a test site up at the moment.
Are there any specific places in w3-total-cache/lib/W3/PgCache.php that you would suggest I add the debugging? And where in ob_callback did you append to $buffer?
Thank you. We’ve put many years of work into the site. I’m anxious to have it running properly again. I even reset the w3tc config to the default settings, but no improvement. Any help you can provide is so very welcome.
Daniel
I usually add my debug messages before conditional statements, i.e., before there is some sort of test.
Note – I make sure to keep an old copy of any of the files I modify so I can easily restore if necessary. If working on a live site, I only make very small changes so that I don’t get myself into a bad or irrecoverable state.
For example, page caching starts with the process() function. On line 195 there is a conditional checking for $this->_caching. Therefore, I place an echo before this and print out the $this->_caching value.
If it is false, then I have my starting position. I look at where $this->_caching gets set (i.e. by the $this->_can_cache() function) and work from there. I do the same thing for the ob_callback function.
Good luck bug hunting. Let us know if you find anything.
Yeah, I learned my lesson a long time ago about editing files without having a copy of the original at hand! Thanks.
I got a value of 1 for $this->_caching. I looked through the code as best I could, but I’m still no closer to figuring out why it’s only caching feeds. I even have it set to preload the cache.
Thanks for trying to help. If you feel inclined to help me dig deeper, I’d be very grateful, but I appreciate the time you’ve given me.
Daniel
1 means that condition passed, so I would move on to the next condition, i.e. in the ob_callback function.
For example, the issue with my caching resided in the $this->_is_cacheable_content_type() function (which was fixed in a later plugin update). The issue can also be from $this->_can_cache2. I would try printing out those two values (1 = ok, 0 = fail, so the problem is probably somewhere in there).
I could try and hunt down the bug for you, but as a rule, I don’t want to mess around with a live site. There is just too much risk for something going wrong, and then there will be tears, fists, and angry words. 😉
Have you tried contacting the w3tc author? I think he does consulting work.