Postmortem

Missed bugs

These are the bugs that were found critical and fixed after release:

#5025 and #5023 were not critical fixes, but could potentially be problematic. The others were critical fixes, and could have been easily avoided, with proper test coverage.

Improve testing procedure

Issues like

Callstack:
    at en/docs/MindTouch/Release_Notes/Kilen_Woods_(8.08)/Postmortem

MindTouch.Deki.Script.Runtime.DekiScriptUndefinedNameException: reference to undefined name 'mantis' Exception of type 'MindTouch.Deki.Script.Runtime.DekiScriptUndefinedNameException' was thrown.
  at MindTouch.Deki.Script.Compiler.DekiScriptExpressionEvaluation.Visit (MindTouch.Deki.Script.Expr.DekiScriptVar expr, DekiScriptExpressionEvaluationState state) [0x00000] in <filename unknown>:0 
  at MindTouch.Deki.Script.Expr.DekiScriptVar.VisitWith[DekiScriptExpressionEvaluationState,Range] (IDekiScriptExpressionVisitor`2 visitor, DekiScriptExpressionEvaluationState state) [0x00000] in <filename unknown>:0 
  at MindTouch.Deki.Script.Compiler.DekiScriptExpressionEvaluation.Evaluate (MindTouch.Deki.Script.Expr.DekiScriptAccess expr, DekiScriptExpressionEvaluationState state, Boolean evaluateProperties) [0x00000] in <filename unknown>:0 
  at MindTouch.Deki.Script.Compiler.DekiScriptExpressionEvaluation.Visit (MindTouch.Deki.Script.Expr.DekiScriptAccess expr, DekiScriptExpressionEvaluationState state) [0x00000] in <filename unknown>:0 
  at MindTouch.Deki.Script.Expr.DekiScriptAccess.VisitWith[DekiScriptExpressionEvaluationState,Range] (IDekiScriptExpressionVisitor`2 visitor, DekiScriptExpressionEvaluationState state) [0x00000] in <filename unknown>:0 
  at MindTouch.Deki.Script.Compiler.DekiScriptExpressionEvaluation.Visit (MindTouch.Deki.Script.Expr.DekiScriptCall expr, DekiScriptExpressionEvaluationState state) [0x00000] in <filename unknown>:0 
  at MindTouch.Deki.Script.Expr.DekiScriptCall.VisitWith[DekiScriptExpressionEvaluationState,Range] (IDekiScriptExpressionVisitor`2 visitor, DekiScriptExpressionEvaluationState state) [0x00000] in <filename unknown>:0 
  at MindTouch.Deki.Script.Compiler.DekiScriptExpressionEvaluation.Visit (MindTouch.Deki.Script.Expr.DekiScriptReturnScope expr, DekiScriptExpressionEvaluationState state) [0x00000] in <filename unknown>:0 

Given the number of operating systems we support, testing for all the RPMs can be a difficult process. This was a problem in Kilen Woods, as documented in this forum post. To improve this process, we are setting up on the new ESX server a snapshot of common Linux distros (Red Hat, Ubuntu, etc.) which we can quickly fire up, install Deki via RPM, and test the installation procedure. Having somebody go through the "big" new features (a list should be generated) would be immensely helpful.

Be wary of writable resources

Writable resources should be kept in a consistent spot - the fact that FCKeditor was writing the custom configuration into the editors folder made it an untenable solution for multi-tenants, and also required extra configuration. While most UI files store documents in /skins/common/cache, this is considered a "safe" folder to delete at any time, hence we couldn't store that information there. A long-term solution will require finding a permanent store for this folder.

Writable resources also cause problems on the MSI installs, as they may cause IIS to restart - we should be very very careful when creating new writable resources. Any changes we make should also be communicated to the packaging devs.

Communicating open-source vs. commercial

This will be an ongoing work item, but we should have done better about being proactive about communicating open-source versus commercial. The message was locked up among a few members; it should have been more explicitly stated to the whole dev team earlier on in the process.

Shipping delay

While Kilen Woods was expected to be in August, an early announcement, timed with OSCON went out. The August release date was missed, as tying together the open-source and commercial editions of Deki caused the open-source version to be delayed, while the commercial edition kinks were worked out (along with all the compat issues with open-source). If we had treated the commercial developments to the application outside the scope of Kilen Woods, KW could have shipped three weeks earlier (on time). Given that the product activation was originally only supposed to be a part of the MSI, the delay in adding it to the Linux version was acceptable.

This development could not have started earlier, as the business side of things hadn't been clarified - the message was only completed hammered out after OSCON, and the technical implementation (refining exactly how commercial/open-source co-exist) took a few weeks after that (signed binaries are hard!).

SVN syncing to SF.net is a slooow

Is it necessary to sync our temporary branches to SF.net? We could probably just get away with posting the primary /product/deki branch on there, which would make synchronizing a lot faster.

Localization

If there was a big failing in Kilen Woods from a product point, it was the exclusion of localized strings for the new control panel. Due to the new rewrite, it was nearly impossible to carry over the old localization keys to the new ones - hence we trashed them all. However, with 2 release candidates, we should have done a better job of reaching out to the previous translators and coordinating translations.

In the future, we will maintain a listserv for translators, giving them a heads-up on changed keys. We are also going to develop a online tool which will make translating strings much easier.

(Should we look at Ubuntu's sytem?)

Binaries from buildbot

Is it possible to have the buildbot post binaries, so we we can have a flag in updateWiki.sh that will pick up the nightly binaries without having to build?

Blogging

Our blogging: epic fail. Now that we have shipped many features, the engineering team should make a concerted effort to write more blog posts. Ideally, we can put together quick videos to highlight features. A list of bloggable topics should be put together.

Keeping installation documentation current

While a lot of our installation documentation remains up-to-date, they are poorly titled. We should make sure that after each release, the installation instructions are up-to-date.

Changing our release strategy

In the past, we have shipped both features and bug fixes in incremental releases. This will stop being the case - incremental releases will only contain bug fixes and specifically approved refinements.