xAIgent.

The xAIgent currently works with monolingual documents in English, French, Japanese, German, Spanish, or Korean.

For some applications, you may wish to have a list of keyphrases that covers a whole collection of documents, where each document has been processed individually by the xAIgent. If you have no constraints on the size of the list of keyphrases, you might simply take the union of all of the phrases as your combined list. To reduce the size of the list slightly, you might drop words that have the same stem (e.g., "automobile" and "automobiles"). If you want to substantially reduce the size of the list, then you can assign a normalized score to each keyphrase and select the keyphrases with the highest normalized scores. Doc-Tags is a really good commercial example of managing collections of documents and their extracted keyphrases

Automatic document classification (Taxonomy) is the use of software to sort documents into various pre-defined categories. A similar task is automatic document clustering, in which there are no pre-defined categories, so the software must create the categories by itself. If you wish to use the xAIgent to generate feature vectors, we suggest the following approach: Apply the xAIgent to all of the documents in your sample collection. Take the union of all of the extracted keyphrases as the feature set. For each document and each feature, let the value of the feature be the number of times that the given phrase occurs in the given document (regardless of whether the xAIgent extracted it from the given document). Apply your favourite machine learning algorithm (e.g., decision tree induction, neural network, genetic algorithm, etc.) to the resulting feature vectors. If you want to learn more about automatic document classification and clustering, there is a hypertext Bibliography on Machine Learning Applied to Text http://www.researchgate.net/publication. xAIgent can be used to generate features for use in feature vectors for machine learning algorithms.