Introduction

While MindTouch has strong merging capabilities on content stored as MindTouch pages, the ability to collaborate through files is limiting due to the lack of document management tools.

Purpose

File reservation, a lightweight check-in/check-out workflow, forced through UI/social constructs (as opposed to actual file locks) will provide a feature that will allow users to know when a file has been "reserved" to prevent overwrites and multiple uploads.

Intended Audience

Deployments of MindTouch which require simple document management features.

Additional information

Project came in through a professional service contract. This module, once completed, will be available in commercial editions of MindTouch. It will require the Noatak release platform to function fully, although it will be installable on Minneopa (9.08) with some custom modifications.

Status

Functional spec is frozen; feature has been accepted into the Noatak release and is under active development.

Functional Specification

This feature enables the reservation of files attached to MindTouch pages by providing a visual and email notifications to interested users on the system. The purpose is to allow a user to place a reservation on a specific file while they are in the process of editing and/or creating a new version of the document. This process will not lock or prevent other users from accessing the file. Users visiting a page will see a visual cue that another user has reserved the file and may be in the process of editing.

Upon the uploaded a new version of the document the reservation is automatically removed showing the document as available for new reservations.  Implementation of this functionality would not prevent concurrent edits being made by multiple users to a single document but would assist in preventing users from editing documents that were already in use by another user.

Use Cases

User discovers files that they have reserved

  • User goes to a page which lists all files that are reserved by the current user
  • Administrators can discover files that have been checked out within a system to do a "cleanup"

User discovers subscriptions to reserved files they have created

  • User goes to a page which lists all the file reservations they have subscribed to
  • Users can navigate to those pages to cancel subscriptions or cancel the reservation in place

User reserves file

  • User navigates to the "files" section of a page and selects from the dropdown menu of actions, "Reserve"
  • This launches a dialog which asks for confirmation that the user wishes to check out a file, with an input field for explanation
  • A visual notice appears next to the file which shows that this file is checked-out, with the username/timestamp of the file check-out
  • The user is subscribed to the file automatically (user is alerted any future changes are made to that file, including uploads)

 

reserve-3.gif
View for the file for people (does not include the reserver). Note that if you expand the +,
you will see the original file, granting access to all original features. 

reserve-4.gif
View for the person who has reserved the file. They have two actions; "Unreserve" and "Cancel reservation"
The "Unreserve" button's terminology will change before final.

Users want to be notified of a file check-in

  • User sees a file checked-out and wants to be notified for when it gets checked back in.
  • User clicks "Notify me of unreservation"
  • When a file gets unreserved (either through cancellation or new file upload), an email is sent to the user

Users want to cancel notifications for file check-in

  • User sees a checked-out file they have subscribed to notifications for
  • User clicks "Cancel my notification" to cancel the subscription

User lifts reservation on file with changes

  • User complete changes to the binary file, and uploads the file back to the page
  • If the filename matches against the checked-out file, then the file is considered checked-in and notifications for that file are now removed for that user

User (or administrators) lifts reservation on file without changes

  • Original reserver (or administrator) clicks "Cancel reservation" button to remove the reservation
    • If the person who lifted the reservation was not the original reserver, then a message is sent to the reserver indicating the reservation has been lifted.

User lifts reservation on file by submitting any file

  • User clicks the "Unreserve" button, which launches the attach file dialog with one row for a file
  • The uploaded file will set with the filename for the new head revision
  • This unreserves the file and unsubscribes to future file changes

User wants to download a reserved file

  • If a file is reserved, users can click the expand icon to expose the file as it normally displays

User overwrites a reserved file

  • When a file with the same name as the checked-out file is uploaded, the operation succeeds, but a visual UI error message is thrown, indicating to the user that they are overwriting a file that was checked-out, and that their changes will most likely be overwritten in the near future
  • A notification is sent to the original user who has checked-out a file letting them know of the reserved file
  • A warning message will also appear underneath the reservation message notifying the reserver that somebody uploaded a file after the reservation
  • The reservation still remains attached to the last reserved user

reserve-5.gif

Non-goals

  • This feature does not do actual technical file locking ("hard" locks vs. "soft" locks)
  • This feature will not work correctly with the non-web UI in its first iteration (Desktop Suite)
    • We will utilize file properties to ensure future forward compatibility with the Desktop Connector

Technical Specification

Resources: Guerric (UI), Arne (API), Roy (Lead)

Implications in Core & commercial:

  • ADDED: File rename API feature
  • ADDED: PUT file version with rename
  • REMOVED: "Gallery" feature in front-end; we will push other DekiScript implementations
  • REWRITE: File listing to include images and generated by UI plugin
    • Ajax loaded revisions
    • Rename files, attach new file version

UI requirements

  • Integration with PHP hooks to overload file table view
  • Integration with notifications to send the proper notifications; will require API resources for this
  • Addition of dialogs to display confirmation screens
  • Installation module?

 

API requirements

  • A file should never be allowed to have two reservations. It should always respect the older reservation and throw an error.
  • The API side will be created as a DekiExtension
  • File lock info is primarily going to exist out of band from the MindTouch File features
    • Data is stored by service locally for qiery purposes
    • Reason is duplicated as a file property for easy reservation discovery (which implictly provides reserving user and date)
  • The extension will be responsible for generating the notification emails and sending them via the Email Service
  • The extension listens to file events on pub sub in order to determine check-in, overwrites, etc.
    • also checks against MindTouch API periodically to avoid out-of-sync issues due to missed events

Extension Data Types

Reservation
<reservation href="..." >
    <file id="..." href="..." revision="...">
        <user.createdby id="..."/>
    </file>
    <user id="..." href="..." />
    <date.reservation>...</date.reservation>
    <reason>...</reason>
    <subscriptions>
        <user id="..." href="..." />
        ...
    </subscriptions>
</reservation>

Extension Features

GET:reservations

Retrieve reservations in order of reservation date. Only a user with admin rights can query reservations for a user other than themselves.

Query Parameters

Name Type Default Description
user string? none The user to filter reservations by. Can be user id or the string 'current' (optional)
limit int? 100 Max reservations to return
offset int? 0 How many reservations to skip

Response

Response Response Body Description
Ok (200) application/xml A reservation exists and the body is the Reservation document described above
Unauthorized (401) text/html Missing or bad Deki session
Forbidden (403) text/html Authenticated user is not authorized to reserve the file
Not Found (404) text/html There is no reservation in place for the file

Response Body

<reservations>
    <reservation>...</reservation>
    ...
</reservations> 

 

GET:reservations/{fileid}

Retrieve the reservation for a file, if it exists.

Response

Response Response Body Description
Ok (200) application/xml A reservation exists and the body is the Reservation document described above
Unauthorized (401) text/html Missing or bad Deki session
Forbidden (403) text/html Authenticated user is not authorized to reserve the file
Not Found (404) text/html There is no reservation in place for the file

Response Body

The Reservation document from above.

PUT:reservations/{fileid}

Create or modify a reservation.

Query Parameters

Name Type Default Description
reason string none Include the Subscription xml in the response document (optional)

Request Body

The PUT can either be an XDoc or simply the query argument reason. In the case of the query argument usage, there either must be no existing reservation or the existing reservation must be owned by the authenticated user and the query body MUST be empty.

While the body can be the full reservation document, the only part considered is the reason, i.e. the following is the minimum document for the PUT:

<reservation>
    <reason>...</reason>
</reservation> 

Response

Response Response Body Description
Ok (200) application/xml The reservation was stored
Unauthorized (401) text/html Missing or bad Deki session
Forbidden (403) text/html Authenticated user cannot modify the existing reservation
DELETE:reservations/{file}

Cancel or check-in an existing reservation (also cancels all subscriptions). If the current version of the file is by the reserving user and is newer than the version at check-out, a delete triggers a check-out. Note: a check-out will also automatically happen via the file update event. Using delete after an upload exists so that the UI can make sure the reservation is no longer in place before reloading the file list.

Response

Response Response Body Description
Ok (200) application/xml The reservation was deleted (although it might not have existed in the first place)
Unauthorized (401) text/html Missing or bad Deki session
Forbidden (403) text/html Authenticated user cannot cancel the existing reservation
GET:reservations/{fileid}/subscriptions/currentuser

Retrieve the current user's subscription. Exists primarily as a quick way to check whether a particular user is subscribed to a reservation on file.

Response

Response Response Body Description
Ok (200) application/xml A reservation for that user and file exists
Unauthorized (401) text/html Missing or bad Deki session
Forbidden (403) text/html Authenticated user is not authorized to view the reservation
Not Found (404) text/html There is either no reservation or the current user is not subscribed to the reservation

Response Body

<user id="..." href="..."/>
PUT:reservations/{fileid}/subscriptions/currentuser

Subscribe the current user to a reservation

Query Parameters

None

Request Body

None

Response

Response Response Body Description
Ok (200) text/html The subscription was stored
Unauthorized (401) text/html Missing or bad Deki session
Forbidden (403) text/html Authenticated user cannot subscribe to the reservation
Not Found (404) text/html No reservation for {fileid} exists, i.e. no subscription can be added
DELETE:reservations/{fileid}/subscriptions/currentuser
 

Remove the current user from the subscription 

Response

Response Response Body Description
Ok (200) text/html The subscription was removed
Unauthorized (401) text/html Missing or bad Deki session
Forbidden (403) text/html Authenticated user cannot cannot unsubscribe from the reservation

 

GET:subscriptions/{userid}

Retrieve reservations in order of reservation date. {userid} can be either the numeric user id or the string 'current'. Only a user with admin rights can query subscriptions for a user other than themselves.

Query Parameters

Name Type Default Description
limit int? 100 Max reservations to return
offset int? 0 How many reservations to skip

Response

Response Response Body Description
Ok (200) application/xml A reservation exists and the body is the Reservation document described above
Unauthorized (401) text/html Missing or bad Deki session
Forbidden (403) text/html Authenticated user is not authorized to reserve the file
Not Found (404) text/html There is no reservation in place for the file

Response Body

<reservations>
    <reservation>...</reservation>
    ...
</reservations> 

 

Tag page

Files 6

FileSizeDateAttached by 
 checkedout.png
No description
21.89 kB00:59, 25 Jun 2009RoyKActions
 reserve-1.gif
No description
17.4 kB23:57, 3 Sep 2009RoyKActions
 reserve-2.gif
No description
6.85 kB23:57, 3 Sep 2009RoyKActions
 reserve-3.gif
No description
15.36 kB23:57, 3 Sep 2009RoyKActions
 reserve-4.gif
No description
18.14 kB23:57, 3 Sep 2009RoyKActions
 reserve-5.gif
No description
19.47 kB23:57, 3 Sep 2009RoyKActions
Viewing 14 of 14 comments: view all
I would like to see a couple of aspects of this requirement modified (First requirement extract below) to improve the UI and the document management functionality, without changing the "wikiness" of file sharing and management in Mindtouch.

"User checks-in file with changes
User complete changes to the binary file, and uploads the file back to the page
If the filename matches against the checked-out file, then the file is considered checked-in and notifications for that file are now removed for that user"

This solution as it is currently written requires the user to upload a file in the same manner as currently exists, which is to use the 'attach file or image' button and select the file to be uploaded, then if the file has the same name as an existing file it will be saved as a revision, or if the filename does not match an existing file, it is uploaded as a new file. This works fine when there is no real attempt at document management or expectation of such by users, but could quickly become a problem if users intended to check in a checked out (reserved) file as a revision and the filename was slightly different - instead creating a new document. This can often happen in reality where files can have their name changed through appending of a "-1" for example if the file was downloaded via the browser and a file with the same name was previously downloaded. The file would automatically have the name changed in the download process and if the user did not notice and change this before uploading back to Mindtouch they could end up with their changes attached as a new file rather than a revision of the checked out file.

A better solution, I believe, would be to add a "check-in" action in the file action menu (complementing the "check-out" action) that would launch the upload window and then compare the filename against the original document when saving. If the filenames were different, the user should be asked whether the file should be saved as a revision or uploaded as a new file. If the user chose to save as a revision via this "Check-in" action and the filename of the new file was different then Mindtouch would change the name of the file to match the new filename.

The second requirement that I believe should be revised slightly is the following:

"User checks-in file without changes
User goes to the "Actions" menu for that file and clicks "Check-in without changes" (This use case needs more work - "check-in" may be weird verbage if a file can be checked-in with no changes)
This checks the file back in and unsubscribes to future file changes"

I suggest that this action be named "Cancel check-out" rather than "Check-in without changes" as the user in this use case is actually deciding not to make any modifications to the check-out file and intends to un-reserve it so others can check-it out. They do not want or need to upload the file in this case and "Check-in without changes" seems counter-intuitive to what the user intends to do as they don't want to check in anything in this scenario.

Other than those two items, implementing this specification in Mindtouch would provide a very useful improvement and provide basic document management functionality to those users who don't want to split their documentation between Mindtouch and a purpose-built DMS, or do not require the full functionality of a purpose-built DMS. edited 02:25, 9 Jun 2009
Posted 02:23, 9 Jun 2009
@btheato: great suggestions. I agree with both, and will incorporate both into the spec. edited 04:27, 9 Jun 2009
Posted 04:27, 9 Jun 2009
@btheato & @royk: I agree that the way we handle versioning files by filename is unintuitive, but I think we should fix that independently of check-in/check-out, since the reservation system is an extension to file attachment that does not change the existing workflow.
Posted 16:15, 9 Jun 2009
@arnec it's a rather trivial UI implementation for us to add this in, and i think it'll be important to the flow for this to exist. i do agree this problem needs to be solved independently in the core app
Posted 16:26, 9 Jun 2009
@royk Are you sure you can handle it in the UI? I thought the upload passes the file directly to the API. Can you affect it's name in transit?
Posted 16:53, 9 Jun 2009
@arnec yes, we actually set the filename on transit, so it's easy for us to capture that and modify it en route
Posted 17:03, 9 Jun 2009
@RoyK: glad to hear that you like the suggested changes. I look forward to seeing the result in Minneopa

@arnec: If the change I suggested for check-in would require a fundamental change to the file attachment or versioning process, then I agree with you. If this can be implemented simply in the UI, and in a way that won't require extensive unwinding when the file versioning module is re-written, then I think it is worth it for ease of use and intuitiveness sake.
Posted 07:42, 10 Jun 2009
Two more additional comments:
1) Should be possible for a user with certain privilege level (Admin?) to cancel a lock created by someone else. I can't tell if this is included here.
2) I am unclear how this would interact with the Webdav editing feature. Would that have to be disabled, or could it coexist peacefully somehow?
Posted 21:28, 11 Jun 2009
@neilw i don't know if we need to cancel a lock, since it's a soft lock. i would much more rely on the administrator notifying the user to clear up their lock rather than add more functionality to support that use case. (2) that's a great question - using webdav would basically break this feature (same issue with the desktop suite).
Posted 21:57, 11 Jun 2009
I like the screen shot. I'm going to share it with our technology planning group. If you don't hear back, keep going! ;)
Posted 13:49, 26 Jun 2009
@Royk: There is one additional function that I would like to see included with this new feature.

Looking at the screenshot you've added to this page, I realized that there is no way to determine what has changed in the new version of a file. If it is possible within the scope of the work you are doing, I would suggest that you add a field to capture "Notes" or "Summary of changes" to allow users to provide some information about the changes on check-in. Unlike a true DMS system, this should probably be an optional field in Mindtouch so that it is not restrictive.

This small change will make it easier to quickly identify a specific version of a file (e.g., "the version prior to when I accidentally corrupted the calculations in the spreadsheet". After all, this is one of the best use cases for implementing the check-in/out feature for attachments.
Posted 06:08, 15 Jul 2009
We've typically used the description field for this, but a 'changes' field does make sense. Good catch!
Posted 12:00, 15 Jul 2009
In the screenshot/mockup above, when the file is reserved and the notification is displayed, the file specs (size, who uploaded it, the actions menu etc) are not displayed. I think it would be better to have them displayed for consistency, and because they'll be important to some users. Or is that intended to appear when you click the + next to the reservation notice, to force people to notice the reservation on the file?

Re terminology, I'd shy away from "locking" because it's not actually locked. "Check in/out" makes sense to those who have used a DMS before, but when you think of it, it's not like a library either - the file hasn't gone anywhere, you can still access it. I can explain "reservation" more easily to users I think - it's like a reserved table in a restaurant, or a jacket on a seat in a movie theatre. Someone's telling you that they don't want you to sit there - but if you want to, you can do so anyway. (If it's the CEO's jacket on the chair, maybe think about sitting elsewhere, or contacting her before sitting down...)

@RoyK re admin cancelling locks: potential use case when a user has left the organisation. You could reset their password and log in as them to do it, I suppose. If they have many locks then a back-end option for the admin to "release all locks for user RoyK" may be good. No, we shouldn't expect that many users will have a lot of locks... but some will, even if it's not sensible. And if those locks can't be released by the admin then they'll stay there annoying other users with pop-up warnings forever, I presume?

/pedantic: On the screen shot, the second button should be "cancel check-out"
Posted 07:45, 28 Aug 2009
To anybody tracking this - thank you so much for the comments. They have been incredibly informative in the discussion and I presented to the engineering team this feature, where it was accepted into the Noatak release. All your suggestions were taken into account and should be reflected in the spec above. We are starting development on this next week.
Posted 05:56, 4 Sep 2009
Viewing 14 of 14 comments: view all
You must login to post a comment.