MindTouch Community Portal
Logged in: Logged in as: Log in
Register
Search:
Page last modified 12:22, 23 May 2008 by petee
User:PeteE > Caching Notes
Page Notifications Off

Caching Notes

                  

    • Identify all places to cache
      • GET:pages/{pageid} returns date.modified, PHP needs to use that and set Last-Modified header
        • petee - verified this can be easily accomplished and pushes requests/second to ~ 1800req/s using varnish (1000req/s using squid)
        • petee - Relying on Last-Modified isn't feasible for the following reasons:
          • We don't always updated page.touched: 0004072: page_touched not changed when page metadata is updated
          • Ajax requests (like editing comments) doesn't do a full page reload so the cache doesn't see a new Last-Modified header and mark the cached entry as stale
          • Making changes via the API won't be reflected until the cache expires or an authenticated user views the modified page
      • GET:files/{fileid} sends ETag
        • To avoid a hit to apache (and a 304 if the ETag is found) we should consider using Last-Modified as well
          • we are actually sending Last-Modified so no change is required here
      • feeds
        • GET:site/feed, GET:pages/{id}/feed, GET:users/{id}/feed  - should all be cached in the API
        • Special:RecentChanges - should also send the proper cache headers
          • we probably don't need to explicity purge this but send a much smaller maxage
    • Identify where we need to purge the cache
      • Pages
        • page modified
        • attachment added
        • comment added
      • Attachment added/deleted
        • For our first pass we can just pass through calls to /@api/files/{id} and let the API return the 304 based on ETag
      • both squid and varnish support purging via a PURGE request with the URL as an argument
        • varnish supports regex based purging but I'm not sure how well this will work for Deki Wiki since some pages have Uri's like:  index.php?title=some/page.  We'll most likely have to send individual purge reqests for each child page.  If purching all child pages proves difficult we may have to just set a shorter TTL and allow the breadcrumbs/navigation to be out of sync until the cached version expires
    • We need to disable caching for authenticated users
      • Using varnish, we can easily look for that authtoken= in the cookie and not cache if present
      • TODO: figure out if we can do this in squid using the cookie or if we'll need to move the authtoken in to a header instead       

    Max's (awesome) perf numbers

    Requests/sec output cache get:pages/{id} output cache get:pages/{id}/contents eAccelerator cache roles,users,services               Notes
    8 no no yes yes              

    api: ~23%

    mysql: ~40%
    9.3 yes no yes yes              

    api: ~20%

    mysql: ~25%
    10.3 no yes yes yes              

    api: ~15%

    mysql: ~20%
    13 yes yes yes yes              

    api: ~11%

    mysql: ~7%
    11  yes yes yes no                

    api: ~15%

    mysql: ~14%
    5.5 yes yes no yes            

    api: ~5%

    mysql: ~4%
    4.1 no no no no           

    api: ~12%

    mysql: ~23%

                 

    PeteE's perf numbers

    Requests/sec output cache get:pages/{id} output cache get:pages/{id}/contents eAccelerator cache roles,users,services

           

    		 Notes
           no no yes yes                     
           yes no no yes              

    api:

    mysql:
           no yes yes yes              

    api:

    mysql:
    3.87 yes yes yes yes             

    api: 16%

    httpd: 25%

    mysql: 0%
    1.86 yes yes no yes              

    api: 7%

    httpd: 25%

    mysql: 0%
    1.79 yes yes no no            

    api: 8%

    httpd: 25%

    mysql: 1%
    1.39 no no no no           

    api: ~16%

    httpd: 25%

    mysql: ~2%

         

    PeteE's perf numbers (worker MPM, mod_fcgid)       

    Requests/sec output cache get:pages/{id} output cache get:pages/{id}/contents eAccelerator cache roles,users,services     varnish,article

    cache patch    

              		 Notes
           no no yes yes                     
           yes no no yes              

    api:

    mysql:
           no yes yes yes              

    api:

    mysql:
    2833.84 yes yes yes yes yes N/A
    5.01 yes yes yes yes              

    api: 18%

    httpd: 26%

    mysql: 1%
    2.65 yes yes no yes              

    api:

    httpd:

    mysql:
          yes yes no no            

    api:

    httpd:

    mysql:
          no no no no           

    api:

    httpd:

    mysql:

                 

         

    Tag page
    What links here

    Files 0

    Attach file or image
     
    You must login to post a comment.
    MindTouch Social Knowledge Base

    Copyright © 2011 MindTouch, Inc. Powered by