Table of contents
- 1. Introduction
- 2. Changes overview
- 2.1. API optimizations
- 2.2. Infrastructure optimizations
- 2.3. Frontend optimizations
- 2.3.1. Javascript
- 2.3.2. PHP
- 3. Performance numbers
Introduction
Deki Wiki 8.05.1 focused on increasing performance to support up to a million uniques a month. The performance work for Deki Wiki 8.05.1 can be categorized into three areas in areas of importance: code changes in the API, infrastructure applications around Deki Wiki, and front-end changes.
Based on our tests on a standard EC2 large instance without the assistance of a reverse proxy, Deki Wiki hit 24 requests/second for anonymous users, and 15 requests/second for authenticated users. Based on a 8 hour workday, 5 days a week, 4 weeks a year, this works out to greater than 10 million pageviews/month that can be supported by Deki Wiki with minimal configuration for anonymous users, and 8.5 million pageviews/month for authenticated users on a single EC2 large instance!
Changes overview
API optimizations
| Bug# | Summary | Status | Opened By | Assigned To | Severity |
|---|---|---|---|---|---|
| Replace stored procedures with inline SQL queries | closed | RoyK | MaxM | minor | |
| DB Query and entity population | closed | MaxM | MaxM | major | |
| Deki XML Parser optimizations | closed | MaxM | MaxM | minor | |
| Misc optimizations to 8.05.1 | closed | MaxM | MaxM | minor | |
| New cache and optimization settings introduced in 8.05.1 | closed | MaxM | MaxM | minor |
Changes in the API fell within four categories: cleaning up database queries, adding caching behavior into the API, optimizing the Deki Wiki XML parser, and miscellaneous optimizations.
Because of our usage of mySQL stored procedures, we could not take advantage of query caching due to our usage of input parameters. A big component of the database query cleanup was to rewrite a lot of our stored procedures to inline SQL statements. With subsequent releases of Deki Wiki, we will continue to remove stored procedures in favor of inline SQL queries. We took notice of common places where mySQL query caching was not being used effectively and added the config key "stats/page-hit-counter" (default: true) to control updates to page_counter. Setting to false increases the query cache hit rate on pages table.
Profiling the API showed the bottlenecks in our applications, and this yielded new caching behavior inside the API, most notably cache/anonymous-output (default: false) and cache/bans (default: false). The former caches the output of expensive API calls such as GET:page/{id} and GET:page/{id}/contents, while the latter will cache the permission mask output of the effective ban query.
The XML parser received a lot of love as well. One of the more significant changes was to add a new content-type ("application/x.deki0805+xml") which processes free links and does content normalization on save, and also uses the XML parser instead of the SGML parser; this turn out to be five times more efficient.
We also submitteda patch for the mySQL .NET connector for TokenizeSql.
Infrastructure optimizations
| Bug# | Summary | Status | Opened By | Assigned To | Severity |
|---|---|---|---|---|---|
| Varnish caching (and invalidation) | closed | RoyK | PeteE | feature |
There are a plethora of off-the-shelf software (all free!) which can improve the performance of Deki Wiki. Two tools of great value are a PHP accelerator and a reverse proxy. PHP, by default, parses and compiles for every webrequest - eAccelerator caches the bytecode so that this overhead is avoided on subsequent requests. Installations for setting up eAccelerator for Deki Wiki are trivial and can increase performance at least 100%.
//todo: Varnish as a reverse proxy once it's implemented)
Frontend optimizations
| Bug# | Summary | Status | Opened By | Assigned To | Severity |
|---|---|---|---|---|---|
| Remove time offsets from UI | closed | RoyK | RoyK | minor | |
| Consolidate JS methods to use jQuery | closed | RoyK | RoyK | minor | |
| Move assumed styles out of general.css | closed | RoyK | RoyK | minor | |
| Consolidate onbody load methods | closed | RoyK | RoyK | minor | |
| UI caching does not expire on release updates | closed | Guerric | Guerric | minor | |
| Audit Javascript includes | closed | RoyK | RoyK | major | |
| Cache the language loading in PHP | closed | RoyK | mozhechkov | minor |
Javascript
One of the largest limiting factors for web applications is the parsing of Javascript by the client's browser. The guys over at PBwiki examined the load times of common Javascript libraries with a great follow-up from John Resig (author of jQuery); the numbers show that even for cached responses, Javascript libraries can take roughly 100-200ms to parse. As hard as we work on the backend to shave 100ms here and 100ms there, if it takes the client ~200ms to render just the JS, then it's all for naught!
Deki Wiki is heavy on the front-end. We support both jQuery as well as YUI, and we also throw down a full WYSIWYG editor down the pipe. We've minimized the impact of the WYSIWYG editor files (generally the largest overhead) by only sending those files to people with editing permissions. Furthermore, we've aggregated all our Javascript files through a PHP script which handles caching via Etags and sends all JS gzipped, which reduces the amount of data being passed down the network.
However, this wasn't enough: in the 8.05 release, we were still pushing down 174K of Javascript for administrators (includes all JS files) and 105K for anonymous users.
For 8.05.1, we re-examined the logic for including libraries for anonymous users and found a few redundant JS libs! We also took some time to refactor some of our old Javascript functionality to use jQuery (it's amazing how many of those 20-25 line functions can be reduced to just two lines in jQuery!) to reduce code complexity. After we refactored the PHP Javascript aggregation script to be smarter about which files to include, we cut down the our initial 174K/105K numbers down to 168K for administrators, and 68K for anonymous users! Although the administrators only received a trivial 3% reduction, anonymous users have a 35% faster experience! Another logic change we took was to consolidate our onload methods - in the past, we had our custom window onload event handler (from days long before JS libs), jQuery event handlers, and YUI event handlers. We knew that competing onload handlers can be troublesome, so we figured it'd be a good idea to have one library execute all our event handlers.
Even better, FCKeditor does lazy loading of its editor files - that reduces our 168K filesize for administrators down to 109K!
PHP
Optimization of PHP involved two work items: removing redundant API calls on the same page, as well as caching the output of the localization parsing. Under certain conditions, the front-end would make multiple requests for users at different APIs: /@api/deki/users/=Admin & /@api/deki/users/1/. To work around this, we've memoized both the usernames and ids of all user requests made by PHP and use those for subsequent requests. On average, this cut down about 5% of the time spent between PHP and the API.
All in all, PHP accounts for roughly 35-40% of total execution time. Of that PHP executing time, about 15% of that was spent parsing localization files. Every web request, prior to 8.05.1, would parse up to three localization files (resources.txt, your localization's resource file, and resources.custom.txt) and then convert to a native PHP data structure for localizing the UI. To reduce this overhead, we've started caching the output of the parsed localization files to disk - the cache invalidates any time the localization files are updated (so updating resources.custom.txt will cause the cache to invalidate). This change in behavior reduced the localization parsing time 92.5% (80ms to 6ms on a test machine).
Going forward, we know there are two more bottlenecks inside PHP: the library parsing time (loading 1MB worth of libraries for every request in the absence of a PHP accelerator - this number used to be as high as 2MB of library files in the Gooseberry days) as well as the general application's execution order. The library parsing time can only be solved by cutting more code from PHP, through our constant refactoring, as well as pushing more functionality to the API.
With the changes from MediaWiki to Deki Wiki for the Hayes release back in July, we approached updating the PHP component surgically - hence it does not take full advantage of the strengths of the API and has a lot of circuitous codepaths that can be optimized (from MediaWiki's PHP parser days). Although the evolution of the PHP from a MediaWiki system to a Deki Wiki system has been a series of small operations - one of these days, we'll undertake a major operation to completely take advantage of Deki Wiki's API strengths to streamline the PHP side of the application.
Going further in the future, the skinning component of Deki Wiki will also do more to lazy-load components like the nav pane and files for templates that do not require them. Given that lots of time is spent between PHP and the API for a cURL handshake, we can also optimize the PHP frontend by aggregating all API calls into one massive call and populate all relevant data at once.
Performance numbers
A series of apachebench tests were run against a Deki Wiki set-up on an EC2 large instance; the full reports are available for consumption. I've copied the relevant information below.
| Version | Requests/second | eAccelerator | API caching |
| 8.05 | 1.94 | no | no |
| 8.05.1 | 9.65 | no | no |
| 8.05.1 | 12.31 | yes | no |
| 8.05.1 | 15.68 | no | yes |
| 8.05.1 | 25.31 | yes | yes |
- Tag page
- What links here
