MindTouch Deki incorporates Lucene.Net 1.9, a powerful and flexible search engine. 

Pages searched

The search function does respect user permissions; this means you will not see search results for pages/files that you can't view.

Terms

A query is broken up into terms and operators. There are two types of terms: Single Terms and Phrases.

A Single Term is a single word such as "test" or "hello".

A Phrase is a group of words surrounded by double quotes such as "hello dolly".

Multiple terms can be combined together with Boolean operators to form a more complex query (see below).

 

Fields

Lucene supports fielded data. When performing a search you can either specify a field, or use the default field of 'content'. You can search any field by typing the field name followed by a colon ":" and then the term you are looking for.

Deki Wiki defines the following fields:

  • author: name of last person to modify the page or file
  • comments: contents of comments on a page (not available for files)
  • content: contents of the page or file
  • description: contents of the file description (not available for pages)
  • date.edited: date when page or file was last modified (format is "YYYYMMDDhhmmss"; e.g. "20070801171530" corresponds to 5:15:30pm on August 1st, 2007)
  • path: the absolute path to the page (not available for files)
  • size: the size in bytes of a page or file
  • tag: list of all tags on a page (not available for files)
  • title: the title of a page or the filename of a file
  • type: the type of the indexed resource ("wiki" for pages, "document" for documents, and "image" for images)
  • uri: the complete uri for the indexed page or file (deprecated, hence no example. use path: instead.)
  • viewcount: The amount of times a page has been viewed.
  • wordcount: The number of words in a page.

Note: This list is not exhaustive, other fields may be available.

 

Field Examples

If you want to find the document entitled "The Right Way" which contains the text "don't go this way", you can enter:

title:"The Right Way" AND content:go

Note: The field is only valid for the term that it directly precedes, so the query

title:Do it right

Will only find "Do" in the title field. It will find "it" and "right" in the default field (in this case the text field).

To find all pages that have been viewed more than 20 times, use:

type:wiki AND viewcount > 20

To find items in a particular location in your wiki, use path.  To find files stored in a path called Network, use:

path:Network

But that only includes items stored directly in that location, not anything within any subtrees.  To include everything within that path, use:

path:Network OR path:Network/*

 

 

Term Modifiers

Lucene supports modifying query terms to provide a wide range of searching options.

Wildcard Searches

Lucene supports single and multiple character wildcard searches within single terms (not within phrase queries).

To perform a single character wildcard search use the "?" symbol.

To perform a multiple character wildcard search use the "*" symbol.

The single character wildcard search looks for terms that match that with the single character replaced. For example, to search for "text" or "test" you can use the search:

te?t

Multiple character wildcard searches looks for 0 or more characters. For example, to search for test, tests or tester, you can use the search:

test*

You can also use the wildcard searches in the middle of a term.

te*t

Note: You cannot use a * or ? symbol as the first character of a search. (It would require scanning the list of tokens in the index in its entirety to look for those that match the pattern, which could be very expensive.) Lucene 2.1 introduced support for this, turned off by default, but Deki does not expose the support at this time.

Fuzzy Searches

Lucene supports fuzzy searches based on the Levenshtein Distance, or Edit Distance algorithm. To do a fuzzy search use the tilde, "~", symbol at the end of a Single word Term. For example to search for a term similar in spelling to "roam" use the fuzzy search:

roam~

This search will find terms like foam and roams.

Starting with Lucene 1.9 an additional (optional) parameter can specify the required similarity. The value is between 0 and 1, with a value closer to 1 only terms with a higher similarity will be matched. For example:

roam~0.8

The default that is used if the parameter is not given is 0.5.

Proximity Searches

Lucene supports finding words are a within a specific distance away. To do a proximity search use the tilde, "~", symbol at the end of a Phrase. For example to search for a "apache" and "jakarta" within 10 words of each other in a document use the search:

"jakarta apache"~10

Range Searches

Range Queries allow one to match documents whose field(s) values are between the lower and upper bound specified by the Range Query. Range Queries can be inclusive or exclusive of the upper and lower bounds. Sorting is done lexicographically.

mod_date:[20020101 TO 20030101]

This will find documents whose mod_date fields have values between 20020101 and 20030101, inclusive. Note that Range Queries are not reserved for date fields. You could also use range queries with non-date fields:

title:{Aida TO Carmen}

This will find all documents whose titles are between Aida and Carmen, but not including Aida and Carmen.

Inclusive range queries are denoted by square brackets. Exclusive range queries are denoted by curly brackets.

Boosting a Term

Lucene provides the relevance level of matching documents based on the terms found. To boost a term use the caret, "^", symbol with a boost factor (a number) at the end of the term you are searching. The higher the boost factor, the more relevant the term will be.

Boosting allows you to control the relevance of a document by boosting its term. For example, if you are searching for

jakarta apache

and you want the term "jakarta" to be more relevant boost it using the ^ symbol along with the boost factor next to the term. You would type:

jakarta^4 apache

This will make documents with the term jakarta appear more relevant. You can also boost Phrase Terms as in the example:

"jakarta apache"^4 "Apache Lucene"

By default, the boost factor is 1. Although the boost factor must be positive, it can be less than 1 (e.g. 0.2)

 

Boolean Operators

Boolean operators allow terms to be combined through logic operators. Lucene supports AND, "+", OR, NOT and "-" as Boolean operators (Note: Boolean operators must be ALL CAPS).

The OR operator is the default conjunction operator. This means that if there is no Boolean operator between two terms, the OR operator is used. The OR operator links two terms and finds a matching document if either of the terms exist in a document. This is equivalent to a union using sets. The symbol || can be used in place of the word OR.

To search for documents that contain either "jakarta apache" or just "jakarta" use the query:

"jakarta apache" jakarta

or

"jakarta apache" OR jakarta

AND

The AND operator matches documents where both terms exist anywhere in the text of a single document. This is equivalent to an intersection using sets. The symbol && can be used in place of the word AND.

To search for documents that contain "jakarta apache" and "Apache Lucene" use the query:

"jakarta apache" AND "Apache Lucene"

+

The "+" or required operator requires that the term after the "+" symbol exist somewhere in a the field of a single document.

To search for documents that must contain "jakarta" and may contain "lucene" use the query:

+jakarta lucene

NOT

The NOT operator excludes documents that contain the term after NOT. This is equivalent to a difference using sets. The symbol ! can be used in place of the word NOT.

To search for documents that contain "jakarta apache" but not "Apache Lucene" use the query:

"jakarta apache" NOT "Apache Lucene"

Note: The NOT operator cannot be used with just one term. For example, the following search will return no results:

NOT "jakarta apache"

-

The "-" or prohibit operator excludes documents that contain the term after the "-" symbol.

To search for documents that contain "jakarta apache" but not "Apache Lucene" use the query:

"jakarta apache" -"Apache Lucene"

 

Grouping

Lucene supports using parentheses to group clauses to form sub queries. This can be very useful if you want to control the boolean logic for a query.

To search for either "jakarta" or "apache" and "website" use the query:

(jakarta OR apache) AND website

This eliminates any confusion and makes sure you that website must exist and either term jakarta or apache may exist.

 

Field Grouping

Lucene supports using parentheses to group multiple clauses to a single field.

To search for a title that contains both the word "return" and the phrase "pink panther" use the query:

title:(+return +"pink panther")

 

Escaping Special Characters

Lucene supports escaping special characters that are part of the query syntax. The current list of special characters are

+ - && || ! ( ) { } [ ] ^ " ~ * ? : \

To escape these character use the \ before the character. For example to search for (1+1):2 use the query:

\(1\+1\)\:2

 

Tag page
Viewing 4 of 4 comments: view all
I don't understand "Escaping Special Characters." There's no definition of "special characters" are (although there is a list of them) or of what "escaping" means in this context.
Posted 21:08, 25 Nov 2008
It's a programming term which means that special character (i.e. character which have already a specific meaning in context) need to be marked so that they do not interfere. Googling for these terms would probably yield an immediate answer. Also, the list of special characters for lucene is shown in the last paragraph. edited 21:38, 25 Nov 2008
Posted 21:37, 25 Nov 2008
Does anyone know if natural language queries can be made to work with Lucene?
Posted 16:36, 19 Feb 2009
What about digits?
I cannot search for example 192.168.1.2
Posted 23:48, 12 Apr 2009
Viewing 4 of 4 comments: view all
You must login to post a comment.