DWR’s AI Search is designed to locate and rank documents with similar concepts.
Traditional search tools look for specific words within documents. But this approach is susceptible to the shortfall of having unimportant or overly common words or keywords produce undesired matches. Furthermore, traditional searching cannot infer linkages based on additional words in context. The documents may be similar, but not in a meaningful way with respect to the document content.
Concepts, on the other hand, allow the ideas behind the search to be considered. DWR uses an AI approach to analyzes the text in conjunction with how users interact with the metadata to build a network of concepts based on the gist of the text. These conceptual units allow meaningful phrases and information to be directly compared to create a more relevant result based on document semantics (rather than just the presence or absence of specific common words).
DWR can show these concepts and connections to provide the reviewer with a quick overview of the driving factors behind the similarity search. The color and size represent the weighted "significance" of the words. Darker and larger mean more significant. In the above example, "bike", "bicycle", and "fenders" would be the strongest concepts.
Words are always alphabetical (and often stemmed), so combining phrases is left to the viewer. The weighting algorithm is basically what Google uses. It's context independent which means each word is weighted independently overall (and it's why we list them alphabetically). The weighting works with any language but is tuned for English.
By comparing the relative and adaptive strengths of the concepts, we can produce a relative strength indicator (RSI) to allow reviewers to understand why the comparison was made, and how much the retrieved documents match the initial search. A numeric value between zero and one gives the ranking at a glance; the larger the value, the more the documents are like the initial concepts.
AI searches require no training period and can be tailored on the fly. DWR allows the user to generate the concepts by selecting a single document, multiple documents, or even snippets of text. This facilitates faster searches and exploration of the document corpus, and it greatly reduces search time.
Using AI Search within DWR
The new AI Search works in phases: each document as analyzed and a conceptual profile[1] is stored, documents are selected for searching, and the search is compared to all documents to find matches.
This conceptualizing phase occurs automatically after documents are processed. Select Extract Text and choose the Enable AI Searching for the system to vertorize the text for AI searching.
Steps are:
- Select either a single document or multiple documents from the grid. Then right-click in the selected area to invoke the pop-up menu. Select “AI Search” from the menu. The search concepts will be created from the selection and applied to all vectorized documents.
- Once the search is complete, all documents with a positive score will automatically be returned in the Grid. The RSI column will contain the score where 1.0 means exact and 0.0 (or blank) implies no meaningful match.
- The returned results are subject to the current filer tree settings. You can change settings and run additional searches to see more results
- The RSI values are computed for all documents not just the visible once. It is not necessary to re-run the same “AI Search” if you change the filter tree options.
- The “Concept View” will be opened or updated to show a graph of the words and concepts selected. You can zoom in to explore the terms and relationships.
- You can clear the graph and RSI values be selecting the filter tree “Clear” button (next to Search).