Building a Discussion Forum

Introduction

This four-part tutorial explains how to build a simple but remarkably functional discussion forum in pure Dekiscript, using only two templates and a modest chunk of code.  If you can make it through all four parts and understand all the code, you'll have a pretty complete understanding of Dekiscript.

In general, this tutorial assumes a reasonable level of familiarity with Dekiscript, though it'll occasionally dive into some more basic material.  Useful more advanced tips may also be sprinkled throughout.

Here's a screenshot of the finished product  (red outline added for clarity):

Credits

The original forum code was created by craigsivils, who therefore inspired this insanity.  Craig has also contributed to its ongoing development.

Additional development, templatification, and tutorial by neilw.

Thanks (or perhaps, not) to SteveB for suggesting the tutorial.

The Tutorial

• Part 1: Basic Structure - This first part introduces the basic structure of the ForumTopicList template we'll be creating.  This is a common Dekiscript pattern that is useful in many applications, and will form the basis for our forums.
• Part 2: Arguments and refinements - Here we add some useful (and highly reusable) argument processing to the template, and then focus on refining the topic list to make it operate more like a real forum.
• Part 3: The ForumTopic Template (not available yet) - Here we'll extend the functionality of our forum list by introducing a template for each topic, and taking advantage of this to put more useful information on the forum topic listing.  The final code presented in part 3 is a usable, fully functioning discussion forum.
• Part 4: Gilding the Lilly (not available yet) - In our interest of exploring every nook and cranny of Dekiscript, for the grand finale we will add some functionality that is not really necessary but very cool.  Check it out!

Building a Discussion Forum, Part 1: Basic Structure

The first part of this tutorial will lay down the basic structure of our ForumTopicList template.  When finished, we'll have a barebones forum structure in place, just enough to whet our appetite.  Our approach is applicable to a wide variety of Deki applications, so this will provide a useful foundation.

Requirements

So what exactly is a discussion forum and what should it do?  Essentially, we'll try to duplicate a useful subset of the functionality of widely used discussion forums such as PHPBB or VBulletin.  For a typical example, just check out the Mindtouch forums.

So here's a quick rundown of our requirements:

1. For now, we want to provide the functionality of a single forum.
2. We should support basic threading.  This means that at the top level, we have a list of topics, and each topic has its own stream of replies.
3. It should be easy to create new topics and replies.
4. Our topic list should be sorted in reverse chronological order (i.e., most recently updated first.)

That's a very high level list, but enough to get us started.

Should we use templates?

Yes!  In general, it is much better to hide re-usable code in a template.  We'll provide as much customizability as we can by passing arguments to the template (see the second part for the gory details.)

So we'll call our first template "ForumTopicList".  As the name suggests, this will generate a list of forum topics (amazing!).  Our basic call, using all default arguments, will be:

{{ template("ForumTopicList") }}

We'll refer to the page on which the above code is inserted as the calling page.

The Basic Structure

Now let's talk about the basic structure that, as previously promised, is very generally applicable to a wide variety of uses.  Deki is going to make our lives easy on one important respect, which is that each page can have a string of comments attached to the bottom.  Therefore, it would make sense for each topic to be its own wiki page, and then the reply mechanism is handled automatically.

Page Hierarchy

We must then decide where to store the topic pages.  We could put them directly underneath the calling page, but then we'd run the risk of the forum topics being intermingled with other, "real" subpages of the calling page.  Instead, we'll create a subpage underneath the calling page called "Forum Topics", and put our topics underneath there.  This has two advantages:

1. Our forum topics are safely sequestered, away from other "content" pages.
2. We know that all pages underneath the "Forum Topics" page are indeed forum topics.  Therefore we don't have to sort through them and make decisions.

We'll refer to the "Forum Topics" page as the "homepage" in our scripts.

The first piece of code we can create, then, is a button to create a new topic:

{{wiki.create("Create New Topic",(page.path.."/Forum_Topics"),nil,true,"Put Your Title Here")}}

Notes:

1. Here I've used the "arg-list" form for calling wiki.create().  I could just as easily have used the "map" form, though it would have been a bit more verbose.  Throughout this tutorial, I'll freely switch back and forth between the two, as the situation dictates.  For more information on this topic, see this FAQ entry.
2. In creating the "path" argument for wiki.create(), we've replaced the space in "Forum Topics" with an underscore.  Different Deki versions may or may not accept spaces in a path, so we'll be safe and use the underscore, which should work in all versions.
3. For now, we've not specified a template for the new topic page.  We'll get to that later.
4. By naming the new page "Put your title here", we're saying that the page title will be the "Subject" of the topic.  This is a natural approach, and from a scripting standpoint it's easy to work with the title of the page, so this works out well overall.

Initializing Variables in a <div block="">

Something looks wrong in the wiki.create() code above.  We've embedded an expression for the path to our homepage in the code.  We're going to need to use that path (lets call it "homepath") elsewhere in the code as well, so we'd like to set it once and then reuse it.  But those double-curly braces define there own scope, so we can't set a variable in there and expect to see it anywhere else.  The solution lies in using HTML statements: any variables set in a "block" attribute are visible anywhere within that element.  In order to make our initialized variables visible to the entire template, we wrap the entire template in a <div> and put our initialization code in a "block" attribute.  Diving into the source editor, it looks like this:

<h1>Template:ForumTopicList</h1>
<div block="var homepath = page.path .. '/Forum_Topics';">
...the rest of the page contents...
</div>

This is a mighty useful technique that I recommend highly.  By putting all common code in the surrounding <div>, you make it easy to find, and the rest of the code inside the the template becomes much cleaner and simpler.  Note that the div code is invisible in the WYSIWYG editor.  We'll be spending most of our time in the source editor anyway.

Our updated button code now looks like this (back in the WYSIWYG editor):

{{wiki.create("Create New Topic",homepath,nil,true,"Put Your Title Here")}}

Much better!

A Quick and Necessary Digression on the Travails of Working in the Source Editor

The forum templates are going to involve a lot of work in the source editor.  Unfortunately, because the source editor is not designed expressly for Dekiscript-inside-HTML, it's not a particularly pleasant place to hang out.  Especially treacherous are misplaced double-quotes and less-than or greater-than signs (<>).

When exiting out of the source view into the WYSIWYG view, the editor parses your HTML and "cleans it up".  That includes removing anything it doesn't like.  No warnings, no opportunity to go back and undo.  This can be a bummer when a misplaced quote causes 90% of your code to be unrecoverably discarded in the transition from source view back to WYSIWYG view.  Therefore, here are a few techniques I've adopted to maintain sanity:

1. Check, double-check, and triple check your code for placement of quotes, less-than, and greater-than signs before going back to WYSIWYG.  Remember that if you wrap your attribute code in double quotes, then you must only use single quotes inside your code.
2. Manually encode these symbols (e.g., "&lt;", "&gt;", and "&quot;") inside your Dekiscript whenever possible.  Don't wait for the source editor to do it.
3. Avoid those symbols in your Dekiscript whenever possible.  For instance, I'd rather use "variable != 0" than "variable > 0" because the latter will eventually end up as the much-less-readable "variable &gt; 0".
4. After editing your code and getting ready to go back to WYSIWYG mode, do a "select all" (ctrl-A) and "copy" (ctrl-C).  That way, if your code gets hosed in the transition, you can paste it back into the source editor and try to find the error.
5. Any time you edit code and exit the source editor, immediately go back in and check that it looks the same as you left it.  Do this always!

There, I've said it.  Onward.

Generating the List of Topics

Finally, let's generate the list of topics.  This will be a table we'll generate with HTML statements.  The format can be anything you want; for now, we'll use a fairly standard forum layout, with columns for Topic, Author, Replies, Last Update, and Views (though I am finding the "Views" column to be of somewhat limited value).

We'll iterate the <tr> element over the subpages of our homepage.  That'll look something like this:

<tr foreach="var p in wiki.getpage(homepath).subpages" if="wiki.getpage(homepath)">
  <td>{{ web.link(p.uri, p.title) }}</td>
  <td>{{ web.link(p.author.uri, p.author.name) }}</td>
  <td>{{ #p.comments }}</td>
  <td>{{ date.format(p.date,'yyyy-M-d H:mm').." by "..web.link(p.author.uri, p.author.name) }}</td>
  <td>{{ p.viewcount }}</td>
</tr>

Note the "if" clause which saves us from a bunch of errors if the homepage hasn't been created yet.

There are lots of problems with this code:

1. It doesn't sort the items by most recent date.
2. It doesn't properly attribute the original author of the topic.  Instead, "p.author" will return the author of the most recent revision.  In fact, the second column lists the same author as the "last Update" column.
3. It also doesn't properly attribute the author of the "Last Update" if that update is a comment.  "p.author" only tells you who made the last change to the page content.
4. If there aren't any topics, it just puts out a null list, which isn't very friendly.
5. The "foreach" clause and "if" clause contain duplicate code ("wiki.getpage(homepath)").  This looks like a good candidate for the initialization code in the <div block="">.

Number 5 is trivial so we'll fix it now.  #1, #3, and #4 will be tackled in part 2.  #2 is a bit thornier; we'll get back to it in part 3.

Results

Final partly-functional code for part 1, with some gussied-up formatting, is presented below, and attached to this page.  See it in action here.

<h1>Template:TopicForumList part1</h1>
<div block="var homepath = page.path..'/Forum_Topics';
var homepage = wiki.getpage(homepath);">
<p>{{wiki.create("Create New Topic",homepath,nil,true,"Put Your Title Here")}}</p>
<table cellspacing="0" cellpadding="3" border="1" class="table" style="border-collapse: separate;">
    <tbody>
        <tr>
            <th valign="top" style="width: 55%;">Topic</th>
            <th valign="top" style="width: 10%; text-align: center;">Author</th>
            <th valign="top" style="width: 5%; text-align: center;">Replies</th>
            <th valign="top" style="width: 25%; text-align: center;">Last Update</th>
            <th valign="top" style="width: 5%;">Views</th>
        </tr>
        <tr if="homepage" foreach="var p in homepage.subpages">
            <td valign="top">{{ web.link(p.uri, p.title) }}</td>
            <td style="vertical-align: top; text-align: center;">{{ web.link(p.author.uri, p.author.name) }}</td>
            <td style="vertical-align: top; text-align: center;">{{ #p.comments }}</td>
            <td valign="top" style="text-align: center;">{{ date.format(p.date,'yyyy-M-d H:mm'); " by "; web.link(p.author.uri, p.author.name) }}</td>
            <td valign="top" style="text-align: center;">{{ p.viewcount }}</td>
        </tr>
    </tbody>
</table>
</div>

Building a Discussion Forum, Part 2: Arguments and Refinements

var homepath = (args.path ?? page.path..'/Forum_Topics');

{{ template("ForumTopicList", { path:path.."/My_Forum_Topics" }) }}

{{ template("ForumTopicList", { path:"./My_Forum_Topics" }) }}

var homepath = (args.path ?? './Forum_Topics');
if (string.startswith(homepath, './') { let homepath = page.path..string.substr(homepath,1); }

var homepage = wiki.getpage(homepath);

var empty = (homepage == nil || #homepage.subpages == 0);

<tr if="!empty" foreach="var p in homepage.subpages">
...
</tr>
<tr if="empty">
  <td colspan="5">(no topics)</td>
</tr>

var topiclist = [];
if (!empty) {
  foreach (var p in homepage.subpages) {
    var lastupdate = <still need to figure this out>;
    var lastauthor = <this too>;
    let topiclist ..= [ { page:p, date:date, author:lastauthor } ];
  }
  let topiclist = list.sort(topiclist, 'date', true);
}

var lastupdate = page.date;
var lastauthor = page.author;
if (#page.comments != 0) {
  var lastcomment = page.comments[#page.comments-1];
  if (date.isafter(lastcomment.date, lastupdate)) {
    let lastupdate = lastcomment.date;
    let lastauthor = lastcomment.author;
  }
}

var topiclist = [];
if (!empty) {
  foreach (var p in homepage.subpages) {
    var lastupdate = p.date;
    var lastauthor = p.author;
    var lasttype = (string.substr(p.editsummary, 0, 12) == 'page created' ? 'new' : 'E');
    if (#p.comments != 0) {
      var lastcomment = p.comments[#p.comments-1];
      if (date.isafter(last.comment.date, lastupdate)) {
        let lastupdate = lastcomment.date;
        let lastauthor = lastcomment.author;
        let lasttype = 'C';
      }
    }
    let topiclist ..= [ { page:p, author:lastauthor, type:lasttype, date:date.format(lastupdate, 's') } ];
  }
  let topiclist = list.sort(topiclist, 'date', true);
}

<h1>Template:ForumTopicList tutorial part2</h1> 
  <div block="var homepath = ($0 ?? args.path ?? './Forum_Topics');
if (string.substr(homepath,0,2) == './') { let homepath = page.path..string.substr(homepath,1); }
var homepage = wiki.getpage(homepath);
var empty = (homepage == nil || #homepage.subpages == 0);
var topiclist = [];
if (!empty) {
  foreach (var p in homepage.subpages) {
    var lastupdate = p.date;
    var lastauthor = p.author;
    var lasttype = (string.substr(p.editsummary, 0, 12) == 'page created' ? 'new' : 'E');
    if (#p.comments != 0) {
      var lastcomment = p.comments[#p.comments-1];
      if (date.isafter(lastcomment.date, lastupdate)) {
        let lastupdate = lastcomment.date;
        let lastauthor = lastcomment.author;
        let lasttype = 'C';
      }
    }
    let topiclist ..= [ { page:p, author:lastauthor, type:lasttype, date:date.format(lastupdate, 's') } ];
  }
  let topiclist = list.sort(topiclist, 'date', true);
}"> 
    <p>{{wiki.create(&quot;Create New Topic&quot;,homepath,nil,true,&quot;Put Your Title Here&quot;)}}</p> 
    <table border="1" cellpadding="3" cellspacing="0" class="table" style="border-collapse: separate; "> 
      <tbody> 
        <tr> 
          <th style="width: 55%; " valign="top">Topic</th> 
          <th style="width: 10%; text-align: center; " valign="top">Author</th> 
          <th style="width: 5%; text-align: center; " valign="top">Replies</th> 
          <th style="width: 25%; text-align: center; " valign="top">Last Comment(C) or Edit(E)</th> 
          <th style="width: 5%; " valign="top">Views</th> 
        </tr> 
        <tr foreach="var t in topiclist" if="!empty"> 
          <td style="font-weight: bold; " valign="top"><font style="font-size: 17px; ">{{ web.link( t.page.uri, t.page.title) }}</font></td> 
          <td style="vertical-align: top; text-align: center; ">{{ web.link(t.page.author.uri, t.page.author.name) }}</td> 
          <td style="vertical-align: top; text-align: center; ">{{ #t.page.comments }}</td> 
          <td style="text-align: center; " valign="top"><font style="font-size: 12px; ">{{ date.format(t.date,'yyyy-M-d H:mm'); if (t.type != 'new') { &quot; by &quot;; web.link(t.author.uri, t.author.name); } &quot; (&quot;;t.type;&quot;)&quot;; }}</font></td> 
          <td style="text-align: center; " valign="top">{{ t.page.viewcount }}</td> 
        </tr> 
        <tr if="empty"> 
          <td colspan="5">(no topics)</td> 
        </tr> 
      </tbody> 
    </table> 
  </div>

Building a Discussion Forum, Part 3: The ForumTopic Template

Building a Discussion Forum, Part 4: Gilding the Lilly

Building a Discussion Forum, Part 5: Coding silly stuff just because

Introduction

This portion of the tutorial demonstrates some advanced use of maps and the xml.text function to retrieve content from subpages.  I am adding the tutorial here because it avoids having to create a complex data structure simply to demonstrate how to aggregate data from that structure.  Special thanks to NeilW for cleaning up the code, and to CRB for helping me with an xpath example.

History

My approach will simply be to go through the template code and explain what is being done.

Processing a Path Argument

The code begins by processing an argument "homepath" with the location of the forum posts.  This is the standard code described in section 2 of the tutorial.

<div block="var homepath='Templates/Basic_Threaded_Discussion_Forum/Forum_Topics';
if (homepath[0] == '/') {
  let homepath = page.path .. homepath;
}
var homepage = wiki.getpage(homepath);
var empty = (homepage == nil || #homepage.subpages == 0);

Initialize List and Map variables

The post count code uses both a list and a map to produce the output.  A map is used because it allows for elements to be dynamically created and modified.  Once the count has been generated in the map, the information is moved to a list to sort the results in descending order.  The variable postcountmap contains the map data and the variable postcountlist contains the list data.

var postcountmap = {};
var postcountlist = [];

Generating the post count numbers

There are three steps to be done for each subpage that is processed.

1. The contents of the subpage is retrieved (Required for step 2)
2. The originator tag is pulled from the contents of the subpage using the xml.text command with an xpath string (argument 2).  The ?? operator is used to provide an alternate value if the originator tag could not be found.
3. The postcountmap is updated to tally one more post for the originator.  The map contents uses the name of the originator as the key and the value is the number of posts by that user.  The ?? operator is also used for the map assignment since the first time a post is found for each user, the map will not have an entry for that user.

if (!empty) { foreach (var p in homepage.subpages)
{
  var entry = wiki.page(p.path);
  var originator = (xml.text(entry, '//*[@id=\'ForumEntryOriginator\']/a') ?? p.author.name);
  let postcountmap ..= { (originator):1+(postcountmap[originator]??0) }; 
}

Sort the output

The post count map is now transfered to a list which is sorted using the list.sort function.  This can be done in one step as shown below.  Since this is the last step in the inital processing, the code block is closed.

let postcountlist = list.sort(map.keyvalues(postcountmap), 'value', true);
}">

Displaying the output

A simple table is used to display the output data.  Neilw's empty trick is also used to handle the situation that there were no posts to count.

    <table cellspacing="0" cellpadding="4" border="1" class="table"> 
      <tbody> 
        <tr> 
          <th valign="top">Author</th> 
          <th valign="top">Posts</th> 
        </tr> 
        <tr foreach="var p in postcountlist" if="!empty"> 
          <td>{{ p.key }}</td> 
          <td>{{ p.value }}</td> 
        </tr> 
        <tr if="empty"> 
          <td colspan="2">(no topics yet)</td> 
        </tr> 
      </tbody> 
    </table><br /> 
  </div>

Putting it all together

The final source code is shown below.  See it in operation here.

<h1>Template:ForumPostCount</h1> 
  <div block="var homepath='Templates/Basic_Threaded_Discussion_Forum/Forum_Topics';
if (homepath[0] == '/') {
  let homepath = page.path .. homepath;
}
var homepage = wiki.getpage(homepath);
var empty = (homepage == nil || #homepage.subpages == 0);
var postcountmap = {};
var postcountlist = [];
if (!empty) { foreach (var p in homepage.subpages)
{
  var entry = wiki.page(p.path);
  var originator = (xml.text(entry, '//*[@id=\'ForumEntryOriginator\']/a') ?? p.author.name);
  let postcountmap ..= { (originator):1+(postcountmap[originator]??0) };
}
  let postcountlist = list.sort(map.keyvalues(postcountmap), 'value', true);
}"> 
    <table cellspacing="0" cellpadding="4" border="1" class="table"> 
      <tbody> 
        <tr> 
          <th valign="top">Author</th> 
          <th valign="top">Posts</th> 
        </tr> 
        <tr foreach="var p in postcountlist" if="!empty"> 
          <td>{{ p.key }}</td> 
          <td>{{ p.value }}</td> 
        </tr> 
        <tr if="empty"> 
          <td colspan="2">(no topics yet)</td> 
        </tr> 
      </tbody> 
    </table><br /> 
  </div>