Download Statistics

Introduction

Currently, the product team at MindTouch is unable to make any data-driven decisions about distribution. We have no insight into how well our downloads are doing, or what platforms we are most successful.

Purpose

At the end of this project, MindTouch should:

  • Visualize the daily distribution of MindTouch through all official MindTouch download sources
  • Understand the effect of changes throughout the download experience (website, landing pages)
  • Create a basis for data-driven metrics for future engineering quarterly goals
  • Coordinate marketing and engineering resources to focus on successful distributions
  • Make data-driven informed decisions about platform changes
  • Lay the foundation (future phase) for being able to hook-in our Loopfuse data to understand from beginning to end how many users are successfully making it through the download experience
  • Lay the foundation (future phase) for being able to hook-in a future opt-in statistics service inside MindTouch to understand usage throughout MindTouch deployments

Intended Audience

This feature is intended for MindTouch engineering to understand how and where MindTouch is being utilized.

Status

RC

Schedule

  1. Command line app for log parsing
    1. Parse logs of S3, Apache (July 9th)
    2. Add entries to database (July 9th)
    3. Maintain batch #s (July 13th)
    4. Have ability to rollback batches (July 13th)
    5. Throw messages if log file's already been run (July 13th)
    6. Sourceforge.net (Delay Indefinitely)
      1. ASK STEVE
    7. Code review with Arne
    8. log review thoughts and ideas (how to get logs)
    9. crunch full logs for roy (delayed)
      1. get stats.mindtouch.com mysql working
    10. Command line wrapper (July 14)
  2. DekiScript extension
    1. Spec and use cases (Due by July 14th - Guerric, Steve, Roy to contribute to spec) for reports/graphs that we want
    2. Development of DekiScript extensions (July 28th)
      1. changing server information to config key
    3. RC of dekiplugin (july 24)
    4. blog (21)
    5. documentation
    6. deployment with pete
    7. connection txt to lower
    8. Code review with arnie
  3. TODO:
    1. convert *versions population data into xml
      1. script to convert actual files into xml
      2. rewrite versions table to take in an xml file
    2. Add table for data display. 
    3.  Troubleshoot kml insertion. 
    4. data cacheing (arne/max)
    5. Include sourceforge data.
    6. Look into loopfuse integration. 

Technical Specification

The purpose of this project is to create a system whereby users who have access to a database containing formatted download information can then display this information in a meaningful way inside MindTouch. The project has two main parts to it. The first will involve processing of raw information into formatted information that is then loaded onto the database. The second part of this project will be the retrieving and displaying of this information.

The first part of this application will allow a user who has Apache Logs and a MySQL database to automatically parse the Apache Logs and load them onto the MySQL database.

The second part of this application will allow a user who has access to a MySQL database to retrieve and display the data stored in the database as total downloads and unqiue downloads; both of these will support date subsets. An additional layer of computation based on this information will also allow users to analyze this data further by creating subsets from other fields along with regression calculation and geomapping of ip.

With this feature, users will be allowed to better analyze download statistics in a collaborative environment. By being able to analyze and display this data in real time, it will be a great asset in assessing the audience of certain downloads and the affect that both external and internal variables have on the demand of the product.

Each component is being spec-ed out individually:

 

http://feedjit.com/stats/bloger.hr/map/

Tag page (Edit tags)

    Files 2

    FileSizeDateAttached by 
     map.mm
    No description
    972 bytes22:11, 23 Jun 2009RickyNActions
     Screenshot.png
    No description
    141.91 kB22:54, 24 Jun 2009RickyNActions
    Viewing 1 of 1 comments: view all
    quick notes - add an autoincrementing primary key on your database - helps with mysql perf. also, add an index onto your date column - you'll be doing a lot of group bys by dates (also possibly distribution, may want to index that).
    Posted 07:36, 25 Jun 2009
    Viewing 1 of 1 comments: view all
    You must login to post a comment.