WordPress Upload HTTP Error Fix

Several versions of WordPress ago, I started sporadically encountering a problem while trying to use the Flash uploader in WordPress. Every time I tried to upload something using the Flash uploader, I would just get this HTTP Error message:

A Screenshot of the HTTP Error message when attempting to upload a files in WordPress

I searched for a solution to the problem and found that it was a bug in one of the older versions of WordPress. Now, several versions later, it’s still happening. I’ve seen it on both WordPress 2.7 and WordPress 2.8. More Googling revealed several tips about about adding directives to the .htaccess file. I tried each of them, but with no success.

The most puzzling part of this problem for me was that I have several different installations of WordPress on different servers and different versions, but this error was only happening on some of them. After I actually sat down and thought about the problem, I realized that the error was only occurring on the WordPress installations that I had set to be private through the authorization control using my .htaccess files. Those sites have an .htaccess file in the root directory which starts with something like this:

AuthName "private site"
AuthType Basic
AuthUserFile /home/private/.htpasswd
Require valid-user

This causes a dialog box to pop up when you try to go to the site. You have to specify a correct username and password, as specified in the .htpasswd file, in order to gain access to the site.

After realizing there might be a connection with this, I tested using the flash uploader on one of my sites with the htaccess authentication turned off. Sure enough, it worked like a charm. So after realizing that the HTTP Error was definitely related to this, a solution was easier to find.

The solution that worked for me was to create an .htaccess file in the wp-admin directory. The htaccess file should have the following rules in it:

AuthType Basic
AuthName share
Satisfy Any
Order deny,allow
Allow from all

<IfModule mod_security.c>
<Files async-upload.php>
SecFilterEngine Off
SecFilterScanPOST Off
</Files>
</IfModule>

Since adding this file, I’ve been using the image uploader on my protected sites without a problem.

Perishable Press 3G Blacklist and WP Super Cache

I’ve been following the Building the 3G Blacklist series on Perishable Press for the last week or two and have been implementing each of the rules as they were released. For the most part, there have been no problems. I’ve seen a huge increase in 403 errors (Forbidden Access) in my logs, which has been good. Judging from my access.log, all of the requests have been bogus.

After the final list came out, I implemented any changes to the rules, tested it in my default browser (Safari) and called it good. Several days later however, I tried to pull up this site on my home PC using Firefox and was greeted with a big fat 403. Uh oh. I switched over to IE and got the same results. After some cursory checking, I switched over to using my laptop and Safari and noticed that there was no problem there. Weird. Even weirder because I’m using Version DSL with router, so as far as my server is concerned, both computers have the same IP.

Most weird: when I actually checked my access.log, I could see my own requests that had been served 403 errors. But instead of the normal 403, the requests actually showed a single request with a 200 status for each time I tried to load a page.

IP - - [30/May/2008:10:20:02 -0700] "GET /about/ HTTP/1.1" 200 363 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7" 
IP - - [30/May/2008:10:20:07 -0700] "GET / HTTP/1.1" 200 364 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7" 

That’s a request for my about page, and then my main page. Both recieved a “Forbiden” in my browser, but both show a status of OK in my log. Additionally, if the request had actually been successful, a bunch of other files would have been requested as well.

I quickly decided that it was more important to get the site up and running again rather than spend a bunch of time trying to figure out what the problem was and how to fix it. After some selective commenting in my .htaccess file, I discovered that the culprit rule was the following one from the 3G Blacklist:

RedirectMatch 403 \/\/

I commented out the rule for the time-being so that I could test further at a later point in time.

This particular rule redirects all requests that contain a double slash after the http:// section. I thought that this was very odd that this rule should break my site because I can’t see any reason why a legitimate request would need to utilize a double slash. I also was concerned, because judging from my access.log, this is the rule that does the bulk of the work concerning 403 errors.

I did some more scanning of my .htaccess folder and arrived at the conclusion that the culprit must be within the rules for the WP Super Cache plugin I had recently installed. This plugin creates a static html page to serve instead of the normal WordPress PHP pages. Here’s an explanation from their site:

When a visitor who is not logged in, or who has not left a comment, visits they will be served a static HTML page out of the supercache subdirectory within the WordPress cache directory. If you navigate to that directory you can view an exact replica of your permalink structure as well as the HTML files within the directories. To determine if a page has been served out of the Super Cache, view the source and the last line on the page should read or .

Hmm, I’d say we’re getting closer now. The section that WP Super Cache adds to my .htaccess file looks like this:

# WP SUPER CACHE
<IfModule mod_rewrite.c>
AddDefaultCharset UTF-8
RewriteCond %{REQUEST_METHOD} !=POST
RewriteCond %{QUERY_STRING} !.*s=.*
RewriteCond %{QUERY_STRING} !.*attachment_id=.*
RewriteCond %{HTTP_COOKIE} !^.*(comment_author_|wordpress|wp-postpass_).*$
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{HTTP_HOST}/$1index.html.gz -f
RewriteRule ^(.*) /wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html.gz [L]

RewriteCond %{REQUEST_METHOD} !=POST
RewriteCond %{QUERY_STRING} !.*s=.*
RewriteCond %{QUERY_STRING} !.*attachment_id=.*
RewriteCond %{HTTP_COOKIE} !^.*(comment_author_|wordpress|wp-postpass_).*$
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{HTTP_HOST}/$1index.html -f
RewriteRule ^(.*) /wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html [L]
</IfModule>
# END WPSuperCache

Now, perhaps a mod_rewrite ninja can see immediately what the problem is, but I was having trouble actually figuring out what was going on. Since I’m using a shared host, I don’t have access to an httpd.conf and therefor cannot use the RewriteLog directive to actually see what’s going on in the rewrites.

After some research I discovered that by adding an R flag to each of WP Super Caches RewriteRule directives, it would force a temporary redirect and therefore allow me to see in the browser what was actually being requested. I changed each RewriteRule to the following:

RewriteRule ^(.*) /wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html.gz [R,L]
RewriteRule ^(.*) /wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html [R,L]

Now after running the same tests again, I could see in my browser how things were getting screwed up. I typed the address https://blog.nerdstargamer.com into Firefox and sure enough, got the 403 error. This time though, when I looked at the URL it showed the following redirect:

https://blog.nerdstargamer.com/wp-content/cache/supercache/blog.nerdstargamer.com//index.html

There’s the culprit right there. The double-slash right before index.html. So, basically, every time WP Super Cache serves a cached page, it’s serving a URL with a double slash before the file name. I did a quick check by deleting the cache folders of WP Super Cache and confirmed that pages not cached loaded fine while cached pages always got redirected to a 403 error. Bingo.

So, now to fix the problem. Why on earth the .htaccess code for WP Super Cache does this in the first place, I’m not sure. It seems wrong to me, but I’ll defer to the experts on this one. Basically what’s happing is that the variable $1 is being replaced with the path name that was requested which includes a trailing slash. The next part of the rewrite starts with a slash, thus the double slash problem.

I was able to fix the conflict in the WP Super Cache code by removing one of the slashes like so:

# WP SUPER CACHE
<IfModule mod_rewrite.c>
AddDefaultCharset UTF-8
# not post
RewriteCond %{REQUEST_METHOD} !=POST
# not a search
RewriteCond %{QUERY_STRING} !.*s=.*
# not an attachment page
RewriteCond %{QUERY_STRING} !.*attachment_id=.*
RewriteCond %{HTTP_COOKIE} !^.*(comment_author_|wordpress|wp-postpass_).*$
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{HTTP_HOST}/$1index.html.gz -f
RewriteRule ^(.*) /wp-content/cache/supercache/%{HTTP_HOST}/$1index.html.gz [L]

RewriteCond %{REQUEST_METHOD} !=POST
RewriteCond %{QUERY_STRING} !.*s=.*
RewriteCond %{QUERY_STRING} !.*attachment_id=.*
RewriteCond %{HTTP_COOKIE} !^.*(comment_author_|wordpress|wp-postpass_).*$
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{HTTP_HOST}/$1index.html -f
RewriteRule ^(.*) /wp-content/cache/supercache/%{HTTP_HOST}/$1index.html [L]
</IfModule>
# END WPSuperCache

This certainly looks funny but at least it works. I’m sure there is a more elegant way to do this, like say, rewriting the original request to remove the trailing slash and then applying the cache rules. Perhaps this is really a problem with the way WordPress is doing its permalinks (I’m on 2.5.1 by the way). Who knows? Ninjas chime in.

Getting Geeky With YSlow

I spent a good amount of time over the last couple of days attempting to make my site a little bit faster. I’ve been pretty negligent about it up until now, because I know that much of the slowness of my site can be directly attributed to my web hosting company. 1 Even so, I decided to spend some time doing what I could to speed things up.

The first thing that I did was run a test in YSlow to see how my site was doing. Yikes! I got an F right off the bat. After some further review and research, I realized that this wasn’t necessarily something that should have me freaking out. If you’re not entirely familiar with YSlow and what it does, Jeff Atwood’s article, “YSlow: Yahoo’s Problems Are Not Your Problems” on Coding Horror is a must-read. Basically, YSlow offers a lot of good advice that should be taken, but with a grain of salt.

With that said, here are the steps that I’ve taken to speed up my site.

Make Fewer HTTP Requests

The first time I ran YSlow, I discovered that all of my pages were making a ridiculous number of HTTP Requests for JavaScript and CSS files. I was requesting four CSS files: screen, print, IE hacks, and one for Lightbox 2. Unfortunately, the IE hacks stylesheet is still necessary. Obviously the screen an print ones are as well. After taking a look at the Lightbox 2 CSS file, I decided that it was small enough to simply tack on to the bottom on my existing screen stylesheet. That’s one down.

There were also quite a few JavaScript files being requested, including all of the files for Google Analytics, Mint, WP Stats and Lightbox. What can I say? I like my tracking software.

The first thing that I decided to do was to reduce the number of tracking utilities I was using to two. I love Mint and Google Analytics seems to be necessary, so I had to get rid of WP Stats. That wasn’t such a big deal for me. That’s another one down.

The next step was to take a long hard look at Lightbox 2. I originally installed this for my Gallery page, and then decided to include it on all my pages on the off chance that I might want to use it in a few posts. While it works and looks great, I’ve been decidedly unhappy about how much baggage Lightbox comes with. There are five JavaScript files that need to be included, just to have that neat little image trick. Even worse, the included Prototype JavaScript library weighs in at a staggering 124KB. What a waste.

I made a mental note to do some research to find a more lightweight solution for my image gallery. Smashing Magazine has a good list of them, which I will inspect at a later point in time. For the time-being, I compressed the Javascript files and was able to bring the total size of the Javascript files down to about 125KB from 196KB. I also decided to only include the scripts on my actual Gallery page. It seems like too much of a waste to require all those files when I rarely use them.

Put CSS At the Top and JS at the Bottom

When I first set up Lightbox, I wanted to avoid using a WordPress plugin for it, so I cooked up my own method of including it. Most of the work was simply trying to find a way around hard-coding my template directory in it and also using a function to keep my header.php file clean and easy to read.

The first problem with my original method was that all of those JavaScript files were at the top of the page, meaning that almost 200KB of JavaScript had to be loaded before any of the content on my page started to load. That’s no good! The simplest thing to do was to move my function down to the bottom of the page, right before the scripts for Google Analytics and Mint. The only other problem was that the function included the CSS file as well. Since I had already decided to merge the Lightbox CSS with my main CSS, all I actually had to do was remove the call to load the CSS.

Use Google’s APIs

Unless you’ve been living under a rock (or just don’t care), you’ve probably heard that Google just released their AJAX Libraries API. This was pretty much perfect timing for me since I was already looking at how Lightbox used the Prototype Framework and Scriptaculous Effects Library. It makes a whole lot more sense to use a version hosted by Google than it does to require clients to download the same exact version of a standard library from my slow web host. Ajaxian has a good rundown of the features of this new API and why you would want to use it.

After doing a relatively quick setup, I was able to call the Prototype framework from the Google API. It came in from Google at only 29KB; that’s the same file that I was just complaining was 124KB. That’s a no-brainer. Scriptaculous was a bit more of a problem though, since it takes a modular approach. Lightbox 2 actually only uses two of the eight possible modules. As far as I can tell, there is no way to use the standard type of of script tag to only include the libraries you want like this:

<script type="text/javascript" src="https://blog.nerdstargamer.com/wp-content/themes/positiveGrey-v2.0/js/scriptaculous.js?load=effects,builder"></script>

One of the comments on Ajaxian by jdalton, addresses this:

Another issue google will need to work out is that MooTools, Scriptaculous, and Dojo are modular (meaning you don’t have to load the kitchen sink and can just load the parts you want). This can effect the file size footprint as well. This may be beyond the scope of a CDN though.

Because I couldn’t find a way to only include the modules I needed, I decided to continue serving them locally instead. So, my function to include Lightbox now looks like this:

function AKM_include_lightbox() {
    $templateDir = get_bloginfo('template_directory');

    $output = <<<EOT
<script src='http://www.google.com/jsapi'></script>
<script type="text/javascript">
    var tplDir = "${templateDir}/images/lightbox/";
    google.load('prototype', '1.6.0.2');
</script>
<script type="text/javascript" src="${templateDir}/js/scriptaculous.js?load=effects,builder"></script>
<script type="text/javascript" src="${templateDir}/js/lightbox.js"></script>
EOT;

echo $output;
}

Reorganize Template Directory

Although this doesn’t actually have anything to do with the speed of my website, it seemed appropriate to take this opportunity to reorganize my template directory a little bit. I was striving to create a more traditional web setup within my WordPress template that included all CSS in a CSS folder, JavaScript in a JS folder, and images in an image folder.

This first issue to address was the WordPress default style.css file. This file is necessary for WordPress template to function properly, as explained in the WordPress Theme Development Codex page. What I decided to do was to remove all of the actual styles from this file and simply leave the WordPress information:

/*
Theme Name:Positive Grey
Theme URI:http://nerdstargamer.com
Description:A simple theme using a fluid 2 column layout with green and grey
Version:2.0
Author:Alissa Miller
Author URI:http://nerdstargamer.com
*/

/* See css/screen-x.x.css for styles */

I then moved all of the styles to a new file called screen-x.x.css in the CSS folder. This allows me to have all stylesheets (with the exception of style.css) in the CSS folder. It also allows me to use versioning in the filename, which as we will see, will be important after I’ve implemented better caching and expires headers.

I previously put all of the Lightbox files in their own folder, to keep things neat. I’ve now decided to roll those files into the normal directory structure instead of keeping them separate. The CSS file got merged with screen.css and all of the Lightbox JavaScript files got moved into the js folder. Lightbox also includes several images, which I decided to put in images/lightbox/ so as not to confuse them with my own template images.

Gzip Components, Improve Caching

One of the rules for YSlow includes Gziping components. Some of my scripts are for Mint and JavaScript, which I can’t really control. The others however, along with my CSS are fair game. I had a little bit of trouble figuring out how to do this since I did not want to use any of the more common php methods to compress my pages on the fly and was looking at just using either mod_gzip or mod_deflate. The YSlow page gives the following information:

Gzipping generally reduces the response size by about 70%. Approximately 90% of today’s Internet traffic travels through browsers that claim to support gzip. If you use Apache, the module configuring gzip depends on your version: Apache 1.3 uses mod_gzip while Apache 2.x uses mod_deflate.

After some research, I figured out that my website is hosted on Apache 2 (not earlier). I included this block in my root .htaccess file:

# GZIP CSS AND JS
<IfModule mod_deflate.c>
 <FilesMatch "\.(js|css)$">
  SetOutputFilter DEFLATE
 </FilesMatch>
</IfModule>

I also decided to make the move to using WP Super Cache instead of WP Cache. WP Super Cache is much like WP Cache but does offer some performance benefits. Once I got WP Super Cache configured and running, it seemed to have an immediate effect on the speed of my blog. Of course, that could have just been wishful thinking on my part.

Add an Expires Header

One of the last things I did was add an expires header in my root .htaccess file. This tells the client browsers not to look for a new version at all if the one in their cache hasn’t expired yet.

Now, I obviously don’t want to do this to the dynamic WordPress files (comments and posts would never update!), but that’s okay because WP Super Cache is taking care of those files already. What I do want to do is add the expires header to all of my images, JavaScript and CSS files. None of these will really change except for the CSS files. Fortunately, when I reorganized my template files, I gained the ability to append version numbers to my CSS files. So now I can go ahead and add that expires header to my CSS files, and then simply change the file name when I need to make changes in my CSS. The new file will download like normal, and it’s good practice to get some sort of versioning underway.

Here is the code that I put in my .htaccess file:

### ADD FAR OUT EXPIRES HEADDER TO STATIC CONTENT ###
<ifmodule mod_expires.c>
  <filesmatch "\.(jpg|gif|png|css|js)$">
       ExpiresActive on
       ExpiresDefault "access plus 1 month"
   </filesmatch>
</ifmodule>

After some thought I decided that one month was an appropriate length for my purposes. This depends entirely on what type of content it is, and how often you are going to change it.

After doing all of the previously mentioned fixes, I had improved the page load time quite a bit for most of my site. The only remaining bottleneck seemed to be my Gallery page. That wasn’t particularly surprising considering that the page includes 25 thumbnail images. The total size of all of the images, at full size, weighs in at a hefty 2MB. This page was also still using the Lightbox scripts.

One of the things I noticed while using YSlow was that some of my thumbnail images seemed to be unnecessarily large. Some of them were as big as 40KB for a 150×150 pixel image! Upon further inspection I decided that all of the thumbnails were too large. I had used WordPress’ feature to automatically create thumbnails of images to set this up. I’m not sure exactly how WordPress does this, but after taking a look at the file sizes I’m sure that it sucks. I recreated all of the thumbnails in Photoshop and the biggest one is now only 17.6KB.

I had also originally set up the gallery page in WordPress’ admin screen (using the file browser and things like that). Once I was no longer using the dynamically generated thumbnails, it didn’t make sense to lay out the page in WordPress’ page section. Instead I created a page template called gallery.php, which includes all of the images and code for the page.

I also copied all of the full size and thumbnail images into my template image folder. This way the links to the images are no longer being stored in my database.

Conclusion

After all of these changes my website does seem to be a little bit faster. These types of exercises are good practice for any web designer/developer. Having a slow web host is no excuse for not doing what you can on your end to make the site faster.

As always, any tips or improvements from more experienced developers in this area are greatly appreciated.

  1. You get what you pay for, right?


appointive
appointive
appointive
appointive