A Virtual Robotic Agent that learns Natural Language Commands

A Virtual Robotic Agent that learns Natural Language Commands

J. Kontos, I. Malagardi and M. Bouligaraki

5th European Systems Science Congress. October.

Heraklion Crete. Res Systemica Volume 2 Special Issue. October 2002. ((http:/www.afscet.asso.fr/resSystemica/accueil.html).

(abridged)

Abstract:The present paper presents the design and implementation of a Virtual Robotic Agent that learns Natural Language commands. The system is a motion command understanding system with a machine learning interface for the communication with its user and teacher. The machine learning interface is based on two methodologies that exhibit two basic modes of learning. These two modes are “learning by example” and “learning by doing”. The system described accepts Greek and English as the natural language of communication of the user with the system. The execution of motion commands expressed in natural language is displayed graphically to the user so that he can easily follow the actions of the agent and supervise its learning operation. The system is applied to the study of the communication between a user and a virtual robotic agent, which exists in a virtual environment and accepts commands and knowledge about the objects and the actions possible in this environment. The commands express three kinds of actions. The first kind of action is change of position such as the movement of an object, the second kind is change of state such as the opening or closing of some objects and the third kind is the change of a relation between objects such as the placement of an object on top or inside another object. The two methods of machine learning used by the agent are presented and compared using illustrative scenarios of operation of the system. Keywords: machine learning, natural language commands, virtual robotic agent

1. Introduction
The present paper presents the design and implementation of a motion command understanding system with a learning interface for the communication with its user. The system is related to learning human-robot system as reviewed in [3].This work is part of our NLP project [1],[2]. The system described here accepts Greek and English as the natural language of communication of the user with the system and the execution of motion commands expressed in natural language. The system could be applied to the communication between a user and an artificial agent, which exists in a virtual environment and accepts commands and knowledge about the objects and the actions possible in this environment. The commands are phrased in Greek or English natural language and they express three kinds of actions. The first kind of action is change of position e.g. the movement of an object, the second kind is change of state e.g. the opening or closing of some objects and the third kind is the change of a relation between objects e.g. to placement of an object on top or inside another object. When the system is given a command like: “open the door”, “open the bottle”, “close the box”, “put the book on the desk” etc. which specifies a task, the system has “to understand” the command before attempting to perform the task. Understanding such commands requires understanding the meaning of actions such as “open”, “close”, “put” and the meaning of prepositional words such as “on”. The meanings of the constituents must be combined and the meaning of the sentence as a whole must be obtained taking into consideration knowledge of the environment or “microcosm” where the commands must be performed. The execution of a command by the agent may initially be wrong. The interaction of the agent with the human user results in learning by the agent the correct way of executing a given command. The main contribution of the present paper is based on the ability of the system implemented to learn from its user to understand and execute correctly motion commands that go beyond its initial capabilities. This learning takes place in cases when the system faces the problem of unknown words, of unknown senses of words or underspecified positions of objects. The system was implemented with Prolog which has some simple facilities for computer graphics. Using these facilities the system displays a room with a door, some furniture and some manipulable objects. These are objects such as a table, a door, a desk, a bottle, a box and a book. The bottle, the box and the book are examples of manipulable objects. It is supposed that there is an invisible agent in the room, who can move around and execute the user’s motion commands. These commands may refer directly or indirectly to the movement of specific objects or the change of their state. The agent knows the names of these objects and their position in the room displayed on the screen. The agent also knows how to execute some basic commands.

2. Motion Verbs
Motion may be specified by a verb either directly or indirectly. These verbs we call “motion verbs” and see [5] for a different approach. The simplest way to specify motion of an object is by using a verb that specifies motion directly. An example verb is “move” as used in the sentence “move the box”. This sentence implies that a motion must be executed by the system with the box as the affected object. Indirect specification of motion can be done in two ways: either in terms of geometric goals, or in terms of a force. Indirect specification of motion in terms of a goal involving physical relationship among objects is quite common. Consider the command “put the bottle on the table”. This command requires that a physical object be moved i.e., “the bottle” with a goal to establish the physical relationship of “on” between it and another physical object i.e., “the table”. Performance of such an instruction demonstrates that the goal of establishing a physical relationship drives the motion of the first object. For verbs such as “put” that specify motion in terms of a geometric goal, properties of the objects that participate in the underlying action are of crucial importance. Indirect specification of motion in terms of a force uses verbs such as “push” and “pull”. Objects affected by motion commands may be also specified either directly or indirectly. Direct specification is based on names of objects known to the system such as box, table, etc. Indirect specifications can be accomplished using complex noun phrases such as “the book on the desk”. In [6] the representation of the meaning of motion verbs was addressed. Their ideas have been implemented as a component of a system that accepts natural language commands as input and produces graphical animation as output. They used a fixed lexicon which they represented manually using their representation method. They state that their long -term goal is to investigate how semantic information for motion verbs can be automatically derived from machine readable dictionaries. They also state that at present their system has no feedback from the graphical animation system to the semantic processor. Finally their system has no learning capabilities. Our system exhibits some novel features for treating motion verbs i.e. the creation of its lexicon is accomplished automatically using a machine readable dictionary, learning of the correct interpretation of commands with more than one meaning is accomplished using machine learning by supervision that are techniques based on visual feedback. One source of the multiplicity of meaning of a command is the multiplicity of the senses of a word as recorded in a machine-readable dictionary. Another source is the possibility of an object to be placed on a surface in different ways. When the user submits a command, the agent, in order to satisfy the constraints of the verb’s meaning, may ask for new information and knowledge about objects and verbs, which may be used in the future. A machine-readable dictionary with possibly ambiguous entries is used, which provides the analysis of complex verbs into basic ones. In particular, in the case of Greek about 600 motion verbs were analysed automatically in terms of about 50 basic verbs. Finally every time a command is executed which is amenable to more than one interpretation the system allows the user to observe the graphical output and state its approval or disapproval which helps the system to learn by supervision.

3. System Architecture and Operation
The system is composed of a number of modules each one performing a different task.
These modules are:
 Machine readable dictionary processor
 Lexical processor Syntactic processor
 Semantic processor Basic motion command processor
 Graphics processor
 Learning module
The operation of these modules is supported by a number of databases. These are:
 Machine readable dictionary
 Basic Lexicon
 Stems Base
 Objects Attributes Base
 Knowledge Base
The user enters his commands in natural language, with the keyboard. The commands are imperative sentences with one motion verb which declares the action and some other words like nouns, prepositions or adverbs complementing the main motion verbs. Prior to the syntactic and semantic analysis of the sentence the system checks if each word of the sentence belongs to its lexicon. Stemming is used at this stage because of the complex morphology of the Greek words. When the command contains a word unknown to the system then the system produces a message to the user asking for information about the unknown word and terminates the processing of the present command. After having recognized all the words in a command, the system performs the syntactic analysis of it. If the input sentence is syntactically correct, the system recognizes the motion verb, the object or objects and the adverb or preposition related to the verb. After this the module for “processing of motion commands” tries to satisfy all the constrains and the conditions for the specified motion. This processing requires searching in the knowledge base from where the system retrieves information about the object’s properties (e.g. position, weight, size, state, subparts etc.). At this point, when some information is unavailable or ambiguous, the system interacts with the user in order to acquire the missing knowledge. There are two different types of questions that the system asks. The first type includes questions for which there is no information in the knowledge base and the user must supply it. The second type refers to questions which demand a Yes or No answer. This happens when more than one interpretation of an input command is possible and the system cannot decide which is the correct one. In these cases, the system, using the machine learning mechanism, suggests each time one of the different solutions and requests an answer from the user. The Yes or No answers generate appropriate entries in the knowledge base and can be used next time a similar command is submitted by the user without requesting any more information. This process is based on the “learning by taking advice” technique of machine learning. In the following section some examples of operation of the system implemented will be presented. Prolog predicates that retrieve each object’s position (i.e. it’s coordinates) from the object database have been implemented. Each predicate subsequently calculates the coordinates of the specified points that constitute the shape of the object’s design. A more general predicate redesigns all the objects after the processing of the user’s command. All the knowledge and the current state of each object can be saved in external files, which are available for future use through the menu options of the interface.

4. Examples of Operation of the System
Suppose that the user enters the command “open the door”. The system isolates the words of the command and recognizes the verb “open” and the noun phrase “the door”. The verb “open” appears in the lexicon with a number of different definitions. E.g. in the LDOCE [4] we find among others the senses of “open” a: to cause to become open, b: to make a passage by removing the things that are blocking it. The Greek dictionary we used contains similar sets of senses for this verb and the sense selection mechanism is practically the same for the two languages. The only difference is the wording of the sense selection rules for the two languages where the objects and their properties have different names. The system selects the sense “b” because it knows that a door blocks a passage. The next decision the system has to take concerns the way the opening action is executed. The system finds in the knowledge base that there are two alternative ways of interpreting the verb “open”, using either a “push” or a “pull” basic motion. Then, it selects the first one and asks the user if this is the right one. If the answer is “No”, the system selects the next available interpretation and prompts again the user for an answer. When the answer is “Yes”, a fact is recorded in the knowledge base which denotes that for the verb “open” and the object “door” the appropriate motion is e.g. “pull” in case that the “Yes” answer was given for the “pull” interpretation. The second example refers to the movement of a book that exists in the microcosm of the system. When the command “put the book on the desk” is given, the system searches the knowledge database to find a side of the book that can be used as a base for it. The book has 6 sides and when the system selects one of them it presents graphically the book on the desk having this side as base. Then, it asks the user if the result of the motion is the correct one. When the user enters a “Yes” answer, this is recorded in the knowledge base and the process terminates. When the user enters a “No” answer, the process continues trying sequentially all the available sides of the book until a “Yes” answer is given by the user. The graphical user interface which was implemented, was very helpful during the development. It was easier to see the result on the screen, graphically rather than reading lists of the knowledge database to find the changes that were recorded during the program execution and the machine learning process. 5. The Implementation of the ID3 Algorithm with Logic Programming A program for the implementation of ID3 Algorithm was used for the learnig applications. Indicatively we are referring to some sections of the program which corresponds to the main parts of the ID3 Algorithm. The problem is to determine a decision tree that on the basis of answers to questions about the non-category attributes predicts correctly the value of the category attribute. Usually the category attribute takes only the values {true, false}, or {success, failure}, or {Yes, No}, or something equivalent. In any case, one of its values will mean failure.
The computation of the entropy has been implemented with the following rule:

compute_set_entropy(Data,Entropy):- count_positive(Data,num),length(Data,Dnum), Pp=Pnum/Dnum, Pn=1-Pp, xlogx(Pp,PpLogPp), xlogx(Pn, PnLogPn), Temp=PpLogPp+PnLogPn, Entropy = -Temp.

Were Data is the input file and Entropy is the value of the entropy for the data. The predicate “count_positive” indicates the number of the examples βρίσκει τον αριθμό των παραδειγμάτων στα δεδομένα που ανήκουν στην κατηγορία που πρέπει να αναγνωρίζεται μετά την μάθηση με τους κανόνες:

count_positive([],0). count_positive([dat("P",_)|More],Pnum):-!,count_positive(More,Pnum1),Pnum=Pnum1+1. count_positive([dat("N",_)|More],Pnum):-count_positive(More, Pnum).

The predicate 'length' βρίσκει το συνολικό αριθμό των παραδειγμάτων στα δεδομένα με τους κανόνες: length([],0). length([Dat|Moredat], Dnum):- !, length(Moredat,Dnum1), Dnum=Dnum1+1.

Το γινόμενο 'xlogx' υπολογίζεται με τους κανόνες: xlogx(X,N):- X=0.0E+00, !, N=0. xlogx(X,N):- N=X*log(X).

The predicate select_minimal_entropy δέχεται μία λίστα από τριάδες της μορφής: (attribute, partition-induced-by-that-attribute, resulting-entropy) και βρίσκει την ιδιότητα (attribute) που δίδει τον διαχωρισμό με την μικρότερη εντροπία και αυτό τούτο τον διαχωρισμό δεδομένης της δομής “c=c(attr,partiton,entropy)” με τους κανόνες: select_minimal_entropy([c(Attr,Partition,Entropy)|MorePartitions],BestAttr,BestPartition):- select_minimal_entropy_aux(MorePartitions,c(Attr,Partition,Entropy),BestAttr,BestPartition). select_minimal_entropy_aux([],c(Attr,Partition,_),Attr,Partition). select_minimal_entropy_aux([c(Attr1,Partition1,Entropy1)| MorePartitions], c(_,_,Entropy),BestAttr,BestPartition):- Entropy1


References

Kontos, J., Malagardi, I., and Trikkalidis, D. (1998). NaturalLanguage Interface to an Agent. EURISCON ’98 ThirdEuropean Robotics, Intelligent Systems & ControlConference Athens. Published in Conference Procedings “Advances in Intelligent Systems:Concepts, Tools and Applications” (Kluwer)
Kontos, J., Malagardi, I. (1998). Question Answering andInformation Extraction from Texts. EURISCON ’98Third European Robotics, Intelligent Systems &Control Conference. Athens. Published in ConferenceProcedings “Advances in Intelligent Systems:Concepts, Tools and Applications” (Kluwer). ch. 11,pp. 121-130.
Klingspor, V., Demiris, J., Kaiser, M. (1997). Human-Robot Communication and Machine Learning. AppliedArtificial Intelligence, 11 pp. 719-746.
LONGMAN DICTIONARY OF CONTEMPORARYENGLISH. (1978). The up-to-date learning dictionary. Editor-in-Chief Paul Procter. Longman group Ltd. UK.
Levin, B., (1992). English Classes and Alternations. APreliminary Investigation. The University of ChicagoPress. Chicago London.
Kalita, J., K. and Lee, J., C. (1997). An Informalanalysis of Motion Verbs based on Physical Primitives.Computational Intelligence, Vol. 13, N.1, pp. 87-125.
Kontos, J. and Malagardi, I. (1999). A Learning Natural Language Interface to an Agent. Proceedings of Workshops of Machine Learning ACCAI 99, Chania. Crete. Hellas.
Malagardi, I. (2001). The Acquisition of World Knowledge and Lexical Combination. HERCMA 2001, 5th Hellenic European Research on Computer Mathematics & its Applications Conference. Athens. Hellas.

ΕΠΕΞΕΡΓΑΣΙΑ ΕΝΝΟΙΩΝ ΤΗΣ ΔΩΔΕΚΑΔΕΛΤΟΥ ΕΠΙΓΡΑΦΗΣ ΤΗΣ ΓΟΡΤΥΝΑΣ

ΕΠΕΞΕΡΓΑΣΙΑ ΕΝΝΟΙΩΝ ΤΗΣ ΔΩΔΕΚΑΔΕΛΤΟΥ ΕΠΙΓΡΑΦΗΣ ΤΗΣ ΓΟΡΤΥΝΑΣ ΓΙΑ ΤΗΝ ΑΥΤΟΜΑΤΗ ΑΝΤΛΗΣΗ ΓΝΩΣΕΩΝ

Ιωάννα Μαλαγαρδή & Ιωάννης Κόντος

Η΄ Διεθνές Κρητολογικό Συνέδριο 1996. ΕΚΙΜ, Ηράκλειο Κρήτης (2000), τα, Α2, σελ. 199-213.

1. Εισαγωγή
Η εργασία εμπίπτει στον κλάδο της Πληροφορικής που αφορά στην εφαρμογή μεθόδων της Τεχνητής Νοημοσύνης στην επεξεργασία νομικών κειμένων με ηλεκτρονικό υπολογιστή θεωρουμένων ως πηγές γνώσεων. Η Τεχνητή Νοημοσύνη είναι ο κλάδος της πληροφορικής που αφορά κυρίως την επεξεργασία γνώσεων. Η επεξεργασία αυτή προϋποθέτει τη δημιουργία βάσεων γνώσεων, η οποία μπορεί να γίνει είτε απο ειδικό “μηχανικό γνώσης” είτε από προγράμματα αυτόματης δημιουργίας βάσεων γνώσεων. Μια προσφάτως αναπτυσσόμενη μέθοδος δημιουργίας βάσεων γνώσεων είναι η αυτόματη άντληση γνώσεων από κείμενα. Τα κείμενα που περιέχουν γνώσεις μπορεί να είναι είτε επιστημονικά είτε νομικά αφού και τα δύο είδη κειμένων περιέχουν γνώση υπό μορφή κανόνων. Η άντληση αυτή μπορεί να γίνει είτε με την αυτόματη μετάφραση του γνωσιακού κειμένου σε έναν τυπικό φορμαλισμό παράστασης γνώσης, όπως με λογική, είτε με την επεξεργασία του κειμένου που σκοπεύει στην απάντηση ερωτήσεων με συμπερασμό που προκύπτει ύστερα από λογική ανάλυση, χωρίς προηγούμενη μετάφραση σε τυπικό φορμαλισμό, όπως περιγράφεται στην παρούσα εργασία. Η λογική ανάλυση επιστημονικών και τεχνικών κειμένων χωρίς προηγούμενη μετάφραση σε τυπικό φορμαλισμό έχει προταθεί και περιγραφεί στις εργασίες [J. Kontos 1980, 1982, 1983, 1985, 1992].
Το σύστημα που παρουσιάζεται εδώ και περιγράφεται παρακάτω αναλύει αυτομάτως τις υποθέσεις νομικού κειμένου χωρίς προηγούμενη κωδικοποίηση τους από άνθρωπο ή πρόγραμμα. Όταν η κωδικοποίηση ενός κειμένου γίνεται από άνθρωπο ή πρόγραμμα χρησιμοποιείται ένας τυπικός φορμαλισμός με τη χρήση του οποίου διατυπώνεται το περιεχόμενο του κειμένου. Οι τυπικοί φορμαλισμοί που χρησιμοποιούνται για τέτοιους σκοπούς αποτελούν ειδικές μορφές της κατηγορικής ή κατηγορηματικής λογικής. Στην Τεχνητή Νοημοσύνη έχουν αναπτυχθεί φορμαλισμοί που διευκολύνουν την επεξεργασία με υπολογιστή, όπως τα σημασιακά δίκτυα και τα πλαίσια.
Τα Σημασιακά Δίκτυα εισάγουν την έννοια της παράστασης γνώσης με τη βοήθεια κόμβων και ακμών. Τα Σημασιακά Δίκτυα είναι εικονική παράσταση γνώσης για τη διευκόλυνση του ανθρώπου. Με βάση την εικονική αυτή παράσταση μπορούν να δημιουργηθούν διάφορες εκδόσεις συμβολικής παράστασης γνώσης, χρησιμοποιώντας κάποιον τυπικό συμβολισμό επεξεργάσιμο από υπολογιστή.
Ένα Σημασιακό Δίκτυο αποτελείται από κόμβους και βέλη ή ακμές που φέρουν επιγραφές. Οι κόμβοι παριστάνουν οντότητες η γενικότερα έννοιες. Οι ακμές είναι προσανατολισμένες και παριστάνουν σχέσεις μεταξύ δύο εννοιών που δηλώνονται ως επιγραφές των αντίστοιχων ακμών. Ζεύγη σχετιζόμενων εννοιών παριστάνουν απλά γεγονότα όπως αυτά που στην Κατηγορική Λογική παριστάνονται με συγκεκριμενοποιημένα κατηγορήματα δύο ορισμάτων. Κάθε κόμβος μπορεί να συνδεθεί με οποιοδήποτε αριθμό άλλων κόμβων σχηματίζοντας δίκτυο.
Κάθε πλαίσιο έχει ένα όνομα που καθορίζει την οντότητα που περιγράφει και ένα σύνολο σχισμών που ορίζουν τα στοιχεία της οντότητας. Μια από τις σχισμές καθορίζει το αμέσως ανώτερο πλαίσιο της ιεραρχίας και αντιστοιχεί με το βέλος "είναι" των σημασιακών δικτύων. Τα φύλλα του δέντρου της ιεραρχίας είναι οι συγκεκριμενοποιήσεις που παριστάνουν μοναδικές οντότητες ενώ τα άλλα πλαίσια παριστάνουν ομάδες. Οι σχισμές ενός πλαισίου μπορεί να γεμίσουν και με ονόματα άλλων πλαισίων που καθορίζουν σύνθετες ιδιότητες ενός πλαισίου. Βασικός μηχανισμός λειτουργίας των πλαισίων είναι η κληρονομικότητα τιμών των σχισμών από άλλα πλαίσια, που ανήκουν στην ίδια ιεραρχία ή οντολογία. Οι σχισμές των πλαισίων μπορούν να γεμίσουν και με "κατά συνθήκην" ή "συνήθεις" ή "τυπικές" τιμές. Αυτό σημαίνει ότι δεν είναι απαραίτητο να περιγραφεί μια οντότητα με κάθε λεπτομέρεια και ορισμένες σχισμές να μην πάρουν τιμές από το χρήστη. Οι σχισμές αυτές μπορεί να πάρουν τυπικές τιμές που έχει ορίσει εκ των προτέρων ο σχεδιαστής του συστήματος επεξεργασίας γνώσεων με παράσταση πλαισίων.
Οι τυπικοί αυτοί φορμαλισμοί απαιτούν τη χρήση μονοσήμαντων συμβόλων, ενώ η φυσική γλώσσα και κατά μείζονα λόγο στα νομικά κείμενα εμφανίζει πληθώρα αμφισημιών. Και η μεγάλη δυσκολία έγκειται στο ότι το σύνολο των εννοιών μιας λέξης δεν παραμένει διαχρονικά σταθερό αλλά μεταβάλλεται με τις κοινωνικές συνθήκες. Οι μεταβολές αυτές είναι ευκολότερο να ληφθούν υπόψη με τη μέθοδο που ακολουθείται εδώ με απλή ενημέρωση του υπολογιστικού λεξικού, ενώ σε αντίθετη περίπτωση θα έπρεπε με την αλλαγή μιας έννοιας να επανακωδικοποιηθούν όλα τα κείμενα από την αρχή.
Το σύστημα επεξεργασίας με υπολογιστή νομικού κειμένου που υλοποιήθηκε με μεθόδους Tεχνητής Nοημοσύνης και περιγράφεται στην παρούσα εργασία έχει τη δυνατότητα να επεκταθεί με την ενσωμάτωση γνώσεων για τη δωρική διάλεκτο και πληροφοριών για την εποχή πέραν αυτών που προέρχονται από το κείμενο της Νομοθεσίας. Επίσης υπάρχει η δυνατότητα επέκτασης προς άλλες κατευθύνσεις όπως θα αναφερθεί παρακάτω.
Η υλοποίηση με υπολογιστή έγινε με τη χρήση της γλώσσας λογικού προγραμματισμού Prolog που αποτελεί βασικό εργαλείο λογισμικού της Τεχνητής Νοημοσύνης. Το σύστημα περιέχει υπολογιστικό λεξικό, κανόνες σύνταξης και λογικά γεγονότα που κωδικοποιούν την προϋποτιθέμενη γνώση.

2. Το Κείμενο και μερικά Ιστορικά Στοιχεία του
Η εργασία αποτελεί το πρώτο βήμα της έρευνάς μας σχετικά με την αυτόματη άντληση γνώσεων από ελληνικά νομικά κείμενα. Ως πρώτο κείμενο για την ανάπτυξη των μεθόδων μας επιλέξαμε το κείμενο της Μεγάλης Δωδεκαδέλτου Επιγραφής της Γόρτυνας, που θεωρείται από τους ειδικούς ως ο αρχαιότερος και σημαντικότερος νομοθετικός κώδικας της ελληνικής αρχαιότητας που σώζεται σε πρωτότυπο. Η Επιγραφή της Γόρτυνας αποτελεί το μοναδικό γνωστό αμέσως νομοθετικό κείμενο της ελληνικής αρχαιότητας, εν είδει πλήρους και αυτούσιας κωδικοποίησης σημαντικών θεσμών του δικαίου ελληνικής πόλεως της κλασικής εποχής [Ι. Τριανταφυλλόπουλος, 1968:10].
Δεν θα αναφερθούμε σε ιστορικά στοιχεία σχετικά με την επιγραφή τα οποία είναι ήδη γνωστά και έχουν μελετηθεί. Θα αναφερθούν στοιχεία ,τα οποία έχουν ερευνηθεί από γνωστούς μελετητές της επιγραφής και ελήφθησαν υπ΄ όψη στη δική μας έρευνα.
Η επιγραφή της Γόρτυνας δεν αποτελεί κώδικα με τη σύγχρονη νομοτεχνική σημασία της λέξης. Δεν είναι μεθοδική, συστηματική και εξαντλητική συλλογή και κατάταξη των κανόνων του συνόλου ή ενός τομέα του δικαίου της Γόρτυνας. Ούτε εμφανίζει στο σύνολό της αλληλουχία κατά τη μετάβασή της από το ένα θέμα στο άλλο. Η Επιγραφή περιλαμβάνει τμήματα μόνον του δικαίου που ίσχυε. Η Επιγραφή αποτελεί μάλλον σύνολο οδηγιών προς τον δικαστή για τον τρόπο εφαρμογής της νομοθεσίας. Οι διατάξεις της Επιγραφής αποσκοπούν κυρίως εις τη ρύθμιση σχέσεων του αστικού δικαίου στο μέτρο της επικρατούσας σε εκείνη την εποχή σύγχυσης των ορίων ποινικού και αστικού δικαίου [Σ. Φιοράκης, Ελ. Περάκη 1973:24].
Η σπουδαιότητα της Επιγραφής είναι μεγάλη και ως γλωσσολογικού μνημείου, της πρώιμης μάλιστα, δωρικής διαλέκτου. Η μελέτη της πρωτότυπης γλώσσας της Επιγραφής είναι κάτι που ενδιαφέρει ιδιαίτερα τη δική μας μελέτη και βρίσκεται υπό εξέλιξη.
Για την έρευνα μας ελήφθησαν υπ΄όψη επιστημονικές μελέτες για την Επιγραφή όπως των Ronald Willets, Ελένης Περάκη και Στυλιανού Φιοράκη, F. Bechtel, C.D. Buck καθώς επίσης και πρόσφατες μελέτες που αφορούν την υπολογιστική επεξεργασία σύγχρονων νομικών κειμένων με μεθόδους τεχνητής νοημοσύνης.

3. Γενική περιγραφή της Επεξεργασίας του Κειμένου της Επιγραφής
Η εργασία εστιάζει στην επεξεργασία φράσεων του κειμένου της Επιγραφής της Γόρτυνας με σκοπό τον εντοπισμό και τον σημασιολογικό χαρακτηρισμό των στοιχείων των κανόνων δικαίου του κειμένου. Η σημασιολογική ανάλυση αφορά σχέσεις μεταξύ των εννοιών που περιέχονται σε φράσεις του κειμένου.
Οι σημασιολογικές σχέσεις συνήθως είναι υπονοούμενες και προκύπτουν από τη χρήση προϋποτιθέμενης γνώσης. Η δυσκολία ανακατασκευής της προϋποτιθέμενης βάσης γνώσης των νομοθετών και των αναγνωστών του νομικού κειμένου μιας τόσο παλαιάς εποχής αποτελεί δυσεπίλυτο πρόβλημα. Για το λόγο αυτό αναγκαστικά η εργασία περιορίζεται στη χρήση προϋποτιθέμενης γνώσης που ενυπάρχει και σε ανάλογες σημερινές καταστάσεις.
Οι προτάσεις του κειμένου που επιλέξαμε για επεξεργασία αποτελούν τμήμα των υποθέσεων των κανόνων δικαίου που περιέχει το κείμενο. Ως γνωστό ένας κανόνας δικαίου και στη σημερινή νομοθεσία έχει δύο βασικά τμήματα, την υπόθεση και την απόδοση. Η υπόθεση προσδιορίζει τις συνθήκες που πρέπει να ισχύουν για να εφαρμοστεί ο κανόνας δικαίου και απόδοση προδιαγράφει την απόφαση που μπορεί πάρει ο αρμόδιος δικαστής. Για κάθε εκδικαζόμενη περίπτωση ο βασικός νομικός συλλογισμός στηρίζεται στη συσχέτιση των χαρακτηριστικών της περιπτώσεως με τα χαρακτηριστικά κάποιας υπόθεσης που εμφανίζεται στο ισχύον δίκαιο.
Η υποστήριξη με υπολογιστή της έκδοσης νομικών αποφάσεων που στη διεθνή βιβλιογραφία, δηλαδή, σε ειδικά επιστημονικά περιοδικά και συνέδρια εμπίπτει στον κλάδο που εμφανίζεται με την ονομασία Artificial Intelligence and Law (Τεχνητή Νοημοσύνη και Νόμος), απαιτεί την ανάλυση των υποθέσεων με πρόγραμμα υπολογιστή. Μέχρι τώρα οι περισσότεροι ερευνητές επιχειρούν την ανάλυση αυτή αφού προηγουμένως κωδικοποιήσουν χωρίς τη χρήση υπολογιστή τα κείμενα σε μορφή ώστε η περιεχόμενη σε αυτά πληροφορία να είναι επεξεργάσιμη από τον υπολογιστή. Κατά τη δημιουργία του συστήματος μας ακολούθησε η μέθοδος ανάλυσης του κειμένου χωρίς προηγούμενη μετάφραση σε έναν τυπικό φορμαλισμό, όπως αναφέρεται και παραπάνω. Η δοκιμή του συστήματος μας έγινε με 24 προτάσεις που προέρχονται από υποθέσεις νομικών κανόνων του κειμένου της νομοθεσίας της Γόρτυνας.

4. Το Υπολογιστικό Λεξικό
Το υπολογιστικό λεξικό του συστήματος περιέχει όλες τις λέξεις των φράσεων που επιλέξαμε από το κείμενο για επεξεργασία με υπολογιστή. Οι λέξεις των προτάσεων που χρησιμοποιήθηκαν στις δοκιμές αποτελούν το λεξικό του συστήματος. Οι λέξεις αυτές έχουν ομαδοποιηθεί σε τρεις μεγάλες κατηγορίες ανάλογα με τον αριθμό των χαρακτηριστικών.
Η πρώτη ομάδα έχει ένα μόνο χαρακτηριστικό, δηλαδή το μέρος του λόγου που είναι μια άκλιτη λέξη, όπως “εάν, δεν, δε, όμως κ.α.” και την αντίστοιχη μορφή της στην αρχαία. Η δεύτερη ομάδα αποτελείται από ρήματα και έχει δύο χαρακτηριστικά το ένα δηλώνει τη μεταβατικότητα, ή μη μεταβατικότητα του ρήματος και το άλλο τον αριθμό. Επειδή στα νομικά κείμενα εμφανίζεται μόνον το τρίτο πρόσωπο δεν απαιτείται η παρουσία αντίστοιχου χαρακτηριστικού διότι εννοείται. Η τρίτη ομάδα αποτελείται από άρθρα, ουσιαστικά, επίθετα και μετοχές και έχει τέσσερα χαρακτηριστικά, δηλαδή το είδος τη λέξης, τον αριθμό, την πτώση και το γένος. Σε όλα τα λήμματα του λεξικού εμφανίζεται η νέα και η αρχαία μορφή της λέξης. Ένα τμήμα του υπολογιστικού λεξικού παρουσιάζεται παρακάτω :

l(ean,ai,pa).
l(den,me,pa).
l(de,de,pa).
l(de,omws,pa).
l(i,e,synd).
l(kai,kai,synd).
l(alla,de,synd).
l(pros,epi,proth).
l(ek,es,proth).
l(ek,eks,proth).
l(viasei,kartei_oipei,v,s).
l(viasei,kartei_damasaito,v,s).
l(diazeyhthoun,diakrinontai,iv,p).
l(eggyiithei,andeksetai,iv,s).
l(apothanei,apothanoi,iv,s).
l(nymfefthei,opyiei,v,s).
l(katalipei,katalipoi,v,s).
l(einai,ei,v,s).
l(yparxei,eie,iv,s).
l(yparxoyn,eie,iv,p).
l(zei,dooi,iv,s).
l(lavei,peroi,v,s).
l(afisei,kataleipei,v,s).
l(gennisei,tekoi,iv,s).
l(thelei,leei,v,s).
l(na_ekplironei,tellen,inv,s).
l(ton,ton,pr,s,a,m).l(tin,tan,pr,s,a,f).
l(tis,tis,pr,s,o,m).l(ti,ti,pr,s,a,n).
l(allin,allan,pr,s,a,f).l(allo,allo,pr,s,a,n).
l(aytoy,ayto,pr,s,g,m).
l(oydeis,metis,pr,s,o,m).
l(toy,tas,d,s,g,n).l(twn,twn,d,p,g,n).
l(o,o,d,s,o,m).l(toy,toy,d,s,g,m).
l(eleytheron,eleytheron,e,s,a,m).
l(eleytheran,eleytheran,e,s,a,f).
l(doylin,dolan,e,s,a,f).
l(doylos,dolos,e,s,o,m).
l(anir,aner,e,s,o,m).
l(andros,andros,e,s,g,m).
l(patir,pater,e,s,o,m).
l(patros,patros,e,s,g,m).
l(mitir,mater,e,s,o,f).
l(adelfos,adelpios,e,s,o,m).
l(adelfoi,adelpioi,e,p,o,m).
l(adelfwn,adelpiwn,e,p,g,m).
l(yios,yiys,e,s,o,m).
l(yioi,yieed,e,p,o,m).
l(gyni,gyna,e,s,o,f).
l(oikea,oikea,e,s,o,f).
l(chrimata,kremata,e,p,o,n).
l(diazygioy,kereysios,e,s,g,n).
l(teknwn,teknwn,e,p,g,n).
l(tekna,tekna,e,p,o,n).
l(tekna,tekna,e,p,a,n).
l(oikia,stega,e,s,o,f).
l(yiothetitheis,anpantos,e,s,o,m).
l(elthon,elthon,met,s,o,m).
l(apothanon,apothanon,e,s,o,m).
l(apothanontos,apothanontos,e,s,g,m).
l(diazeygmeni,kereyonsa,e,s,o,f).
l(oikositon,endothidian,ad,s,a,f).
l(ateknon,ateknon,ad,s,a,f).
l(diazeygmeni,kereyonsa,ad,s,o,f).
l(ypaitios,aitios,ad,s,o,m).

Για το συγκεκριμένο κείμενο έγινε προσπάθεια να αποδοθεί η έννοια κάθε αρχαίας λέξης στη Νέα Ελληνική μονολεκτικά, εκτός από δύο περιπτώσεις, όπου ένα ζεύγος αρχαίων λέξεων και συγκεκριμένα το “κάρτει οίπει” και το “κάρτει δαμάσαιτο” αποδόθηκαν με τη νεοελληνική λέξη “βιάζω”, χωρίς όμως να διαταραχθεί η δομική αντιστοιχία μεταξύ των δύο μορφών της γλώσσας.
Τέλος υπάρχει μια περίπτωση όπου το απαρέμφατο του αρχαίου κειμένου αποδίδεται στη Νέα Ελληνική, ως συνήθως, με τη δομή “να+ρήμα”.

5. Συντακτική Ανάλυση
Η συντακτική ανάλυση με υπολογιστή που υλοποιήσαμε στηρίζεται σε πρωτότυπη μέθοδο κατάλληλη για γλώσσες με σχετικά ελεύθερη σειρά λέξεων, όπως η ελληνική. Η μέθοδος στηρίζεται στην αυτόματη μεταγραφή κάθε πρότασης σε έναν αριθμό γεγονότων σε γλώσσα prolog για κάθε λέξη της πρότασης που πρόκειται να αναλυθεί. Τα γεγονότα αυτά έχουν τη μορφή ενός λογικού κατηγορήματος με τρία ορίσματα, όπου το πρώτο όρισμα αντιστοιχεί στον αριθμό της πρότασης που ανήκει η λέξη, το δεύτερο στη θέση της λέξης μέσα στην πρόταση και το τρίτο στη λέξη αυτή καθεαυτή. Στη μέθοδο που ακολουθούμε είναι δυνατόν να εκφραστούν συντακτικοί κανόνες για την αναγνώριση ισοδύναμων συντακτικών δομών με διαφορετικές θέσεις των λέξεων χρησιμοποιώντας έναν μόνον κανόνα. Στις παραδοσιακές μεθόδους επεξεργασίας του λόγου με υπολογιστή οι οποίες ως επί το πλείστον αφορούν την αγγλική γλώσσα κάθε συντακτικός κανόνας αφορά μια συγκεκριμένη σειρά λέξεων, αυτό σημαίνει ότι αν εφαρμόσουμε τέτοιες μεθόδους για την ανάλυση της ελληνικής θα καταλήξουμε σε μια πληθώρα συντακτικών κανόνων λόγω της ελευθερίας της σειράς των όρων που υπάρχει στην ελληνική.
Οι κανόνες που έχουμε αναπτύξει στη φάση αυτή αφορούν υποθετικές προτάσεις. Οι μορφές προτάσεων που μπορούν να αναλυθούν με τους κανόνες του συστήματός μας αποτελούνται από ένα ρήμα και τα ορίσματα του. Αντιμετωπίζεται η περίπτωση του ελλείποντος υποκειμένου του ρήματος και ανάλογα με το σθένος του ρήματος προβλέπεται και ο αριθμός των αντικειμένων. Τα ορίσματα των ρημάτων αναγνωρίζονται από κανόνες που αφορούν τις εξής απλές ή σύνθετες ονοματικές φράσεις :

Αντωνυμία
π.χ. τίς, τί, άλλην, άλλο

Ουσιαστικό
π.χ. ανήρ, μήτηρ, πατήρ, αδελφός, τέκνα, χρήματα

Άρθρο + Ουσιαστικό
π.χ. ο πατήρ, η γυνή, του πατρός, των αδελφών

Άρθρο + Μετοχή
π.χ. ο υιοθετηθείς, ο αποθανών, οι επιβάλλοντες, η διαζευγμένη

Επίθετο + Ουσιαστικό
π.χ. οικόσιτον δούλην, ελευθέρα γυνή, ελευθέρου τέκνου, μητρική περιουσία

Ονοματική Φράση + ή + Ονοματική Φράση
π.χ. ελεύθερον άνδρα ή ελευθέραν γυναίκα, πατήρ ή αδελφός

Ονοματική Φράση + και+ Ονοματική Φράση
π.χ. ανήρ και γυνή

Ουσιαστικό στην Ονομαστική + Ουσιαστικό στη Γενική
π.χ. αδελφοί του πατρός, οι επιβάλλοντες του υιοθετήσαντος, αδελφοί του αποθανόντος

Ουσιαστικό+ Πρόθεση + Άρθρο+Ουσιαστικό
π.χ. υιοί εκ των αδελφών, αδελφός εκ του αυτού πατρός,

Παρουσιάζουμε παραδείγματα κανόνων επεξεργασίας κειμένου σε γλώσσα λογικού προγραμματισμού Prolog.

p(S,np,NP,P):- w(S,N1,D),w(S,N2,N),N2=N1+1,
l(D,_,d,Nu,P,G),l(N,_,e,Nu,P,G),

c(D,N,NP).

p(S,np,NP,P):- w(S,N1,D),w(S,N2,N),N2=N1+1,
l(D,_,d,Nu,P,G),l(N,_,met,Nu,P,G),
c(D,N,NP).

p(S,npc,NPc,o):- w(S,_,No),l(No,_,e,_,o,_),
w(S,_,Na),l(Na,_,e,_,g,_),r(No,Na,R),
concat(No,Na,NPc),write(S,R),nl.

p(S,npq,NP,P,N1):- w(S,N1,E1),l(E1,_,e,_,P,_),
w(S,N2,ek),N2=N1+1,w(S,N3,D),
l(D,_,d,_,g,_),N3=N2+1,
w(S,N4,Pr),l(Pr,_,pr,s,g,_),
N4=N3+1,w(S,N5,E2),l(E2,_,e,_,g,_),
N5=N4+1,c(E1,ek,S1),c(S1,D,S2),
c(S2,Pr,S3),c(S3,E2,NP).

p(S,se,SE,V,_,NP):- a(S,_,V,Ne),l(V,_,v,_),p(S,np,NP,a),
c(Ne,V,CV),c(CV,NP,SE).

p(S,se,SE,V,NPo,NPa):- a(S,_,V,Ne),l(V,_,v,_),
p(S,np,NPo,o),p(S,np,NPa,a),
c(NPo,Ne,S1),c(S1,V,S2),c(S2,NPa,SE).

p(S,se,SE,V,NPo,INV):- a(S,_,V,Ne),l(V,_,v,_),p(S,np,NPo,o),
w(S,_,INV),l(INV,_,inv,_), c(NPo,Ne,S1),c(S1,V,S2),c(S2,INV,SE).

p(S,se,SE,V,NPo,NPc):- w(S,_,V),l(V,_,v,_),
p(S,np,NPo,o),p(S,npc,NPc,o),
c(NPo,V,S1),c(S1,NPc,SE).

p(S,se,SE,V,NPc,_):- a(S,_,V,NE),l(V,_,iv,_),
p(S,npc,NPc,o),c(V,NPc,S1),c(NE,S1,SE).

6. Γνώση του Κόσμου για την Υποστήριξη της Ανάλυσης
Όπως έχουμε και σε άλλες μας εργασίες επισημάνει η γνώση του κόσμου είναι απαραίτητη για την ανάλυση φυσικής γλώσσας με υπολογιστή [Ι. Κόντος, 1982, 83], [Ι. Μαλαγαρδή 1995α, 1995β, 1996]. Η γνώση που χρησιμοποιείται στην παρούσα εφαρμογή χωρίζεται σε δύο κύρια μέρη:

α) στην κατηγοριοποίηση των ενεργειών και
β) στον καθορισμό των υπονοούμενων σχέσεων μεταξύ ουσιαστικών.
Οι ενέργειες εκφράζονται με ρήματα που στον συγκεκριμένο μικρόκοσμο ομαδοποιούνται ως εξής :

1) αδικήματα : βιάσει, λάβει (κάρτει οίπει, κάρτει δαμάσαιτο, πέροι)
2) ύπαρξη : ζει, υπάρχει, αποθάνει (δόοι, ει, είε, αποθάνοι)
3) γενικές δράσεις : αφήσει, γεννήσει, διαζευχθούν, εγγυηθεί, καταλείπει, νυμφευθεί (καταλείπει, τέκοι, διακρίνονται, ανδέκσεται, καταλίποι, οπυίει)

Τα είδη σχέσεων που υπονοούνται μεταξύ ουσιαστικών στη βάση γνώσης του συγκεκριμένου μικρόκοσμου είναι συγγένεια και ευθύνη.

1) συγγένεια : αδελφός πατρός (αδελπιός πατρός)
2) ευθύνη : υπαίτιος διαζυγίου (αίτιος ει τας κερεύσιος)

Η γνώση αυτή των υπονοούμενων σχέσεων χρησιμοποιείται κατά την ανάλυση ονοματικών φράσεων της μορφής ουσιαστικό στην ονομαστική + ουσιαστικό στη γενική, επίθετο στην ονομαστική + ουσιαστικό στη γενική. Οι σχέσεις αυτές εμφανίζονται στο εξαγόμενο εφόσον συμβεί να αποτελέσουν μέρος της απάντησης κάποιας από τους τύπους ερωτήσεων που περιγράφονται παρακάτω.
Σε προηγούμενη μας εργασία [Ι. Μαλαγαρδή, 1996] μελετήθηκαν ονοματικές φράσης με γενική, λαμβάνοντας υπόψη την κατηγοριοποίηση του Α. Τζάρτζανου, η οποία, ως γνωστό, αφορά κυρίως σημασιολογικές σχέσεις μεταξύ των συστατικών της ονοματικής φράσης. Στη συνέχεια έγινε υλοποίηση με υπολογιστή για τον προσδιορισμό της υπονοούμενης σχέσης μεταξύ των συστατικών της ονοματικής φράσης με βάση τις ιεραρχίες των εννοιών.
Παρουσιάζουμε επίσης μέρος της παραπάνω γνώσης κωδικοποιημένης ως μια βάση γνώσης. Η ανθρώπινη γνώση κωδικοποιείται ως μία βάση γνώσης με γεγονότα (facts) και με κανόνες (rules). Με τα παρακάτω γεγονότα κωδικοποιείται μέρος της γνώσης που χρησιμοποιήσαμε.
adik(viasei).adik(lavei).
yparx(zei).yparx(yparxoyn).yparx(yparxei).yparx(apothanei). drasi(diaseyhthoun).drasi(afisei).drasi(gennisei).drasi(nymfemthei).
drasi(katalipei). drasi(eggyiithi).
r(adelfoi,patros,syggeneia). r(ypaitios,diazygioy,efthyni).

Τα γεγονότα αυτά γράφονται χρησιμοποιώντας τα εξής κατηγορήματα :

adik καθορίζει την κατηγορία των ρημάτων που δηλώνουν αδικήματα.
yparx καθορίζει την κατηγορία των ρημάτων που δηλώνουν ύπαρξη.
drasi καθορίζει την κατηγορία των ρημάτων που δηλώνουν δράση που δεν αποτελεί αδίκημα.
r καθορίζει σχέσεις μεταξύ εννοιών που εκφράζονται με ουσιαστικά.

7. Παραδείγματα Ερωτήσεων που απαντά το Σύστημα

Αναφέρονται εδώ ορισμένες βασικές μορφές ερωτήσεων που μπορεί να απευθύνει ο χρήστης στο σύστημα και να λάβει απαντήσεις που σχετίζονται με το νομικό συμπερασμό καθόσον προκύπτουν από την επεξεργασία υποθέσεων. Οι απαντήσεις στις ερωτήσεις τύπου 2, 3, 4 στηρίζονται στη χρησιμοποίηση γνώσης της κατηγοριοποίησης των ενεργειών.

Ερώτηση τύπου 1 : Να βρεθούν όλες οι δομές που εμφανίζονται στο κείμενο και προβλέπονται από τους κανόνες (q).

q:-p(S,P,X,_),write(S," has"),nl,write(P,"=",X),nl, fail.

Eρώτηση τύπου 2 : Nα βρεθούν όλες οι προτάσεις που αναφέρονται σε αδικήματα και να προσδιοριστούν οι θεματικοί ρόλοι που είναι ρόλοι του μικρόκοσμου (Δράστης, Θύμα) (ta).

ta:-p(S,se,X,V,Y,A),adik(V),write(S,"=",X),nl,
write("DRASTIS=",Y),nl,write("THYMA=",A),nl,
readchar(_),fail.

Ερώτηση τύπου 3 : Να βρεθούν όλες οι δράσεις και να προσδιοριστεί ο δράστης και το αντικείμενο (td).

td:-p(S,se,X,V,Y,A),drasi(V),write(S,"=",X),nl,
write("DRASTIS=",Y),nl,write("ANTIK=",A),nl,
readchar(_),fail.

Ερώτηση τύπου 4 : Να βρεθούν όλες οι προτάσεις που αφορούν ύπαρξη και να προσδιοριστεί ο πάσχων (ty).

ty:-p(S,se,X,V,Y,_),yparx(V),write(S,"=",X),nl,
write("PASCHON=",Y),nl,readchar(_),fail.

Ερώτηση τύπου 5 : Να βρεθούν όλες οι προτάσεις που αναφέρονται σε συνδυασμό συγκεκριμένης ενέργειας και ουσιαστικών που παίζουν το ρόλο υποκειμένου ή αντικειμένου (q1).

q1:-p(S,se,X,apothanei,gyni,_),write(S,"=",X),nl,fail.
%(sentno,np/npc/npq/se,phrasi,rima,ypok,antik)

Ερώτηση τύπου 6 : Να αναλυθεί πρόταση της οποίας δίνεται ο αριθμός. Χρησιμοποιείται για τεχνικούς λόγους (εκσφαλμάτωση) (q2).

q2:-write("DOSE ARITHMO PROTASIS"),nl,readint(N),
p(N,se,X,_,_,_),nl,write(X),fail.

Ερώτηση τύπου 7 : Να βρεθεί βάσει του ρήματος της υπόθεσης που αντιστοιχεί στις συνθήκες ενός κανόνος δικαίου η απόδοση που αντιστοιχεί στην ποινή ή στο νομικό αποτέλεσμα (q3, q4, q5).

q3:-p(S,se,X,apothanei,_,_),write(S,"=",X),nl,
a(S,A),write(A),nl,fail.
%(sentno,np/npc/npq/se,phrasi,rima,ypok,antik)

q4:-p(S,se,X,yparxei,_,_),write(S,"=",X),nl,
a(S,A),write(A),nl,readchar(_),fail.
%(sentno,np/npc/npq/se,phrasi,rima,ypok,antik)

q5:-p(S,se,X,viasei,_,_),write(S,"=",X),nl,
a(S,A),write(A),nl,readchar(_),fail.
%(sentno,np/npc/npq/se,phrasi,rima,ypok,antik)

Παρουσιάζουμε μερικά παραδείγματα ανακτηθέντων κανόνων δικαίου της Νομοθεσίας. Η νεοελληνική μετάφραση ακολουθεί τη συντακτική δομή του αρχαίου κειμένου για τη διευκόλυνση της παράλληλης ανάλυσης των δύο κειμένων.

(ΙΙ 2) Εάν τις ελεύθερον ή ελευθέραν βιάσει, εκατόν στατήρας θα καταβάλει .

Αί κα τόν ελεύθερον ε τάν ελευθέραν κάρτει οίπει, εκατόν στατέρανς καταστασεί.

(ΙΙ 11) Οικόσιτον δούλην εάν βιάσει, δύο στατήρας θα καταβάλει.

Ενδοθιδίαν δόλαν αί κάρτει δαμάσαιτο, δύο στατέρανς καταστασεί.

(VΙΙΙ 51.)Εάν όμως μήτηρ δεν υπάρχει, πλησίον των εκ μητρός συγγενών της να ανατρέφεται.

Αί δέ μάτερ μέ είε, παρ τοίς μάτροσι τράπεθαι.

(XI 6) Αν δε αποθάνει ο υιοθετηθείς γνήσια τέκνα μη καταλιπών, να μεταβαίνει η περιουσία στους επιβάλλοντας του υιοθετήσαντος.

Αί δ’ αποθάνοι ο ανπαντός γνέσια τέκνα μέ καταλιπόν, παρ’ τόνς τό ανπαναμένο επιβάλλοντανς ανκορέν τά κρέματα.

8. Επίλογος
Η παρούσα εργασία αποτελεί το πρώτο βήμα της έρευνάς μας σχετικά με την αυτόματη άντληση γνώσεων από ελληνικά νομικά κείμενα. Ως πρώτο κείμενο για την ανάπτυξη των μεθόδων μας επιλέξαμε το κείμενο της Μεγάλης Δωδεκαδέλτου Επιγραφής της Γόρτυνας, που θεωρείται από τους ειδικούς ως ο αρχαιότερος και σημαντικότερος νομοθετικός κώδικας της ελληνικής αρχαιότητας που σώζεται σε πρωτότυπο. Έγινε η προσπάθεια μιας νέας προσέγγισης της Νομοθεσίας, βασισμένη στην ανάλυση κειμένου με ηλεκτρονικό υπολογιστή. Βέβαια ακούγεται κάπως περίεργα η ανάλυση ενός αρχαιότατου κειμένου με μεθόδους που μας προσφέρει η σύγχρονη τεχνολογία. Η υλοποίηση όμως μας απέδειξε ότι είναι εφικτή η επεξεργασία από μετάφραση στη Ν. Ελληνική που είχε γίνει ακολουθώντας όσο ήταν δυνατόν τη συντακτική δομή του αρχαίου κειμένου.
Η επεξεργασία που γίνεται με το σύστημα αφορά υποθέσεις των νομικών κανόνων που περιέχει το κείμενο. Η χρήση ενός τέτοιου συστήματος για υποστήριξη του νομικού συλλογισμού στηρίζεται στην ταύτιση μιας ειδικής περίπτωσης με κάποια από τις υποθέσεις που αναλύθηκαν. Μια περίπτωση ορίζεται από το Δράστη το Θύμα και την Πράξη εφόσον είναι τιμωρητέα, δηλαδή αδίκημα. ΄Αλλα είδη περιπτώσεων αντιστοιχούν σε υποθέσεις που δεν περιγράφουν αδικήματα, αλλά συνθήκες με κάποιες νομικές συνέπειες, όπως είναι περιπτώσεις κληρονομιάς και υιοθεσίας. Όταν το σύστημα αναγνωρίσει μια τέτοια ταύτιση μπορεί αυτομάτως να μας πληροφορήσει για το περιεχόμενο της απόδοσης του εφαρμοζομένου κανόνα δικαίου. Η επέκταση του συστήματος για νεότερη νομοθεσία εφόσον θα είναι δυνατή η συλλογή χαρακτηριστικών περιπτώσεων θα μας δώσει τη δυνατότητα μιας ρεαλιστικότερης εφαρμογής.
Η εργασία αυτή είναι ελπιδοφόρο πρώτο βήμα για την επέκταση της έρευνας μας προς διάφορες κατευθύνσεις, όπως :
1) H επεξεργασία αρχαίου κειμένου από το πρωτότυπο, χωρίς τη βοήθεια μετάφρασης, ελπίζοντας ότι άλλα κείμενα της αρχαίας ελληνικής γραμματείας θα μας είναι περισσότερο οικεία.
2) H επέκταση της γραμματικής της συντακτικής ανάλυσης σε άλλα γλωσσικά φαινόμενα που δεν καλύπτονται από το παρόν σύστημα.
3) Η μελέτη των τυχόν πλεονεκτημάτων της αρχαίας ελληνικής σε σχέση με άλλες γλώσσες σε ότι αφορά στην επεξεργασία με υπολογιστή.

9. Βιβλιογραφία
Bechtel, F. (1963).Die griechischen Dialekte. Berlin 1921-4.
Buck, C. D. (1955). The Greek Dialects. Chicago.
Kontos, J. (1980). Syntax-Directed Processing of Texts with Action Semantics. Cybernetica, 23, 2 pp. 157-175.
Kontos, J. (1982). Syntax-Directed Plan Recognition with a Microcomputer. Microprocessing and Microprogramming. 9, pp. 227-279.
Kontos, J. (1983). Syntax-Directed Fact Retrieval from Texts with a Micro-Computer. Proc. MELECON '83, Athens.
Kontos, J. (1985). Natural Language Processing of Scientific/Technical Data, Knowledge and Text Bases.Proceedings of ARTINT Workshop. Luxemburg.
Kontos, J. (1992). ARISTA: Knowledge Engineering with Scientific Texts. Information and Software Technology, Vol. 34, No 9, pp 611-616.
Κόντος, Ι. (1996). Τεχνητή Νοημοσύνη και Λογομηχανική. Εκδόσεις Ε. Μπένου.
Liddell, H. G., Scott, R. Μέγα Λεξικόν της Ελληνικής Γλώσσης.
Μαλαγαρδή, I. (1995a). Συγκριτική Ανάλυση "να" και "για να" Δομών της Νέας Ελληνικής με Αντίστοιχες Δομές της Γερμανικής και Εφαρμογή στη Μηχανική Μετάφραση. Αδημοσίευτη Διδακτορική Διατριβή. Πανεπιστήμιο Αθηνών.
Malagardi I., (1995b) The resolution of the subject ambiguity in sentences with "ya na" using domain knowledge, and related problems in machine tranlation. To appear inthe 2nd. International Congress on Greek Linguistics. Salzburg. September 1995.
Μαλαγαρδή, Ι. (1996). Προσδιορισμός με Υπολογιστή της Υπονοούμενης Σχέσης μεταξύ των Συστατικών Ονοματικών Φράσεων σε Υπογλώσσες. 17η Συνάντηση Εργασίας ΑΠΘ, Θεσσαλονίκη.
Πανταζίδου. Ι (1892). Λεξικόν Ομηρικόν. Αθήνα.
Περάκη, Ε. (1973). Η Μεγάλη Δωδεκάδελτος Επιγραφή της Γόρτυνος. Εισαγωγή Σ. Φιοράκη. Δικηγορικός Σύλογος Ηρακλείου.
Rissland, E. L. (1990). Artificial Intelligence and Law : Stepping Stones to a Model of Legal Reasoning. Yale Law Journal. Vol. 99, pp.1956- 1981.
Τζάρτζανος, A. (1989). Νεοελληνική Σύνταξις. Τομος B΄ , Αφοί Κυριακίδη, Θεσσαλονίκη.
Τριανταφυλλόπουλος, Ι. (1968). Αρχαία Ελληνικά Δίκαια.
Willets, R. F. (1967). The Law Code of Gortyn, Kadmos : Supplement I. Berlin 1967.

Motion Verbs and Vision

Motion Verbs and Vision

Ioanna Malagardi and John Kontos

HERCMA 2007 8th Hellenic European Research on Computer Mathematics & its Applications Conference. Athens

On-line publication: http://www.aueb.gr/pympe/hercma/proceedings2007

 (abridged)

Abstract--In the present paper we aim at the use of the analysis of the definitions of the Motion Verbs for the application to the understanding and description of action sequences such as those recorded in a video. The main points of computer processing of verbs of motion involve that the definitions are given as input to a system that produces an output that gives a grouping of the verbs and synthesized definitions of these verbs using primitives. In the system presented here the input action sequence is analyzed using the semantics of primitive motion verbs and the way they combine for the synthesis of complex verbs that summarize the action sequence. A future application of this work could be in the automatic text generation of descriptions of motion images obtained by artificial vision systems. These texts may be helpful for people with vision disabilities.
Index Terms – Cognitive Vision, Moving Images, Motion Verbs, Semantic Ontology, Video

1 INTRODUCTION
In the present paper we aim at the use of the analysis of the definitions of the Motion Verbs for the application to the understanding and description of action sequences such as those recorded in a video. Motion Verbs are analyzed using primitive verbs as described below using definition chains. A primitive motion verb can be classified according to pictorial criteria that may be obtained by the comparative analysis of a sequence of images. This classification can be inherited by non primitive verbs in accordance with their dependence on primitive verbs.
A variety of approaches have been proposed for the processing of action sequences recorded in a video. Most of these approaches refer to one or more of three levels of representation, namely, image level, logic level and natural language level. Motion verbs are useful for the third level representation.
In previous work [1] and [2] we presented a system of programs that concerns the processing of definitions of 486 verbs of motion as they are presented in a dictionary. This processing aimed at the exploitation of dictionaries for Natural Language Processing systems. A recent proposal for the organization of a Machine Readable Dictionary is based on the structure and development of the Brandeis Semantic Ontology (BSO), a large generative lexicon ontology and lexical database. The BSO has been designed to allow for more widespread access to Generative Lexicon-based lexical resources and help researchers in a variety of natural language computational tasks [3].
The semantic representation of images resulting from knowledge-assisted semantic image analysis e.g. [4] can be used to identify a primitive motion verb describing the sequence of a few images. Other efforts for the semantic annotation and representation of image sequences aim at the building of tools for the pre-processing of images and are briefly present below.

2 RELATED WORK
Cho et al. [5] propose to measure similarity between trajectories in a video using motion verbs. They use a hierarchical model based on is_a and part_of relation in combination with antonym relations. The ontological knowledge required by their system is extracted from WordNet.
They created five base elements to represent motion of moving objects namely “approach”, “go to”, “go into”, “depart” and “leave”. They use motion verbs to represent moving of objects as high level features from the cognitive point of view. According to this paper the problem of bridging the gap between high level semantics and no level video features is still open. The method proposed in the present paper is a novel contribution towards the solution of the bridging problem mentioned above.
The University of Karlsruhe group Dahlkamp, and Nagel, [6], [7], is developing a system for cognitive vision applied to the understanding of inner-city vehicular road traffic scenes. They argue that an adequate natural language description of developments in a real–word scene can be taken as a proof of “understanding what is going on”. In addition to vehicle manoeuvres the lane structure of inner-city roads and road intersections are extracted from images and provide reference for both the prediction of vehicle movements and the formulation of textual descriptions. Individual actions of an agent vehicle are associated to verb phrases that can be combined with a noun phrase referring to the agent vehicle to construct a single sentence in isolation. The next step is to concatenate individual manoeuvres into admissible sequences of occurrences. Such knowledge about vehicular behaviour is represented internally as a situation graph formed by situation nodes connected by prediction edges. They organize situation nodes not only according to their temporal concatenation but also according to a degree of conceptual refinement. For example an abstract situation node called “cross” (for cross an intersection) is refined into a sub graph that consists of a concatenation of three situation nodes, namely, (1) drive_to_intersection, (2) drive_on_intersection, (3) live_intersection. Such a refinement can take place recursively. A subordinate situation node inherits all predicates from its superordinate situation nodes. A path through a directed situation graph tree implies that the agent executes the actions specified in the most detailed situation node reached at its point in time during traversal semicolon, that is, such a path implies the behavior associated with a concatenation of actions encountered along such a path.
Using these situation graphs trees the above mentioned system generates a list of elementary sentences describing simply events. These sentences are analogues to the sentences we input to our system which instead of traffic scenes analyses office scenes and recognizes higher level event structures.
Recently learning systems have been developed for the detection and representation of events in videos. For example A. Hakeem, M. Shah [8] who propose an extension of CASE representation of natural languages that facilitates the interface between users and the computer. CASE is a representation proposed by Fillmore [9]. They propose two critical extensions to CASE that concern the involvement of multiple agents and the inclusion of temporal information.
M. Fleishman at al. (2006) [10] present a methodology to facilitate learning temporal structure in event recognition. They modeled complex events using a lexicon of hierarchical patterns of movement, which were mined from a large corpus of unannotated video data. These patterns act as features for a tree kernel based Support Vector Machine that is trained on a small set of manually annotated events. To distinct types of information are encoded by these patterns. First, by abstracting to the level of events as opposed to lower level observations of motion, the patterns allow for the encoding of more fine grained temporal relations than traditional HMM approaches. Additionally, hierarchical patterns of movement have the ability to capture global information about an event.

3 GREEK MOTION VERB PRIMITIVES
The main points of computer processing of verbs of motion involve that the definitions are given as input to a system of Prolog programmes and an output is produced that gives a grouping of the verbs and synthesized definitions of these verbs.
The set of 486 verb entries related to motion and were used as input to a system that produced groups of them on the basis of chains of their definitions. The verb at the end of a chain was used as the criterion of verb grouping. Prior to using these chains it was necessary to eliminate the cyclic parts of the definition chains which were also automatically detected by the system. The definitions of the verbs in each definition are in turn retrieved from the lexicon and in this way chains of definitions are formed. These chains end up in circularities that correspond to reaching basic verbs. The elimination of circularity that occurs in certain chains requires the choice of suitable verb as terminal the chain. The choice for each case of elimination of circularity requires the adoption of some “ontology”.
The results of automatic grouping were compared with groupings in Greek, German and English language that were done manually. The construction of chains was then applied to the automatic construction of definitions using a small number of verbs that appeared in at the end of the definition chains and were named “basic” representing primitive actions. The English translation of some of the primitive Greek motion verbs obtained by our system are: Touch, take, put, stir, raise, push, walk that are used for the system presented here in this paper in order to make it intelligible to a wider audience.

4 UNDERSTANDING AND DESCRIPTION OF MOVING IMAGES
The choice of one or more primitive verbs for the automatic description of a motion sequence is based on the abstract logical description of this sequence. The abstract logical description is supposed to contain declarations of the position and state of different entities depicted in the images from which the action sequence is extracted. The comparative logical analysis of the semantic representation of images resulting from knowledge-assisted semantic image analysis is used to identify a primitive motion verb describing the sequence of a few images. The synthesized definitions of more complex motion verbs together with other domain knowledge is used to generate text that describes a longer part of the action sequence with these verbs. A system that we implemented for the description in English of action sequences is described below.

5 SYSTEM DESCRIPTION
The system consists of an input module that accepts formal descriptions of action sequences. These sequences are analyzed by a primitive action recognizer module that provides input to a complex verb recognizer and finally an output module generates the sequence description. The system was implemented in Prolog and two examples of its operation are given below. The three main modules of our system are briefly described below giving indicative Prolog rules that are used for the accomplishment of its basic function. Finally a simple example is given of how the system could be augmented in order to be able to answer natural language questions about the evolution of the action sequence that was input. This constitutes an image grounded human computer interface for multimodal natural language question answering systems. Such an interface could be a test of whether “Cognitive Vision” is achieved. An early system for the generation of visual scenes with a multimodal interface was reported in [11].

5.1 The Input Module
The Input Module accepts a sequence of facts representing images using a predicate herewith named “frame” that constitute abstract descriptions of the images. The “frame” predicate records information concerning the time of taking the image, location of the acting agent and the state of the agent and all other entities of interest.
The following are examples of rules defining the semantics of the primitive verbs “take” and “put” and which are used for the recognition of the occurrence of primitive actions by combining successive image formal descriptions:

THE TAKE RULE:

take(A,X,L1,C5):-frame(T1,L,_,_,L1,_),
frame(T2,L,_,_,hand,_),T2=T1+1,L1<>"hand",
entities(A,_,X,_),
c(" the ",A,C1),c(C1," took the ",C2),
c(C2,X,C3),c(C3," from the ",C4),c(C4,L1,C5).

THE PUT RULE:

put(A,X,D):-frame(T1,L,_,_,hand,_),
frame(T2,L,_,_,D,_),T2=T1+1,D<>"hand",!,
entities(A,_,X,_),write(" the ",A," put the ",X," at the ",D).

Where :

c(X,Y,Z):-concat(X,Y,Z) that constructs the concatenation Z of the strings X and Y.

The positions of the objects in the microcosm are stated as below:

position(desk,1).
position(door,3).
position(bookcase,6).

The possible states of the door and the book are given by: state(opened,open).

The states of the agent are classified as stationary and moving. E.g. the stationary states are defined by:

stationary(sitting).
stationary(standing).

5.2 The Complex Verb Recognizer Module
The Complex Verb Recognizer Module uses the semantics of the complex verbs used in the action sequence descriptions expressed in terms of the primitive verbs used for the low level description of the actions depicted by short sequences of images. The following is an example of a rule defining the semantics of complex verb such as “transport” that is used by the Complex Verb Recognizing Module.

THE TRANSPORT RULE

transport(A,X,L1,L2):-write(" because "),nl,
take(A,X,L1,C),!,write(C),
write(" and "),nl,put(A,X,L2),
L1<>L2,write(" it follows that "),nl,
write("The ",A," transported the ",X," from the ",L1, " to the ",L2),nl.

5.3 The Action Sequence Description Generation Module
The output of our system is a sentence describing briefly the input action sequence and an explanation giving the reasons that support this description using primitive verbs. The operation of this module is closely related to the operation of the complex verb recognizer module and generates a single sentence description together with an explanation that supports the description generation.

6 EXAMPLES USED FOR THE EVALUATION OF THE SYSTEM

6.1 The First Example of Action Sequence Description
A simple example is presented her that was used for the evaluation of the feasibility of our approach. Consider the microcosm of an office environment. A video taken of an agent acting on such an environment may depict the agent approaching a book case in another room taking a book from it and placing the book on her desk. The sequence of images may show the following sequence of actions:

1. The agent is sitting at her desk.
2. The agent is getting up and walking to the door of her room.
3. The agent opens and goes through the door of her room.
4. The agent approaches the bookcase.
5. The agent takes a book from the bookcase.
6. The agent approaches her desk and puts the book on it.
7. The agent sits at her desk and opens the book.

This action sequence may be finally described by the sentence “The agent transported a book from the bookcase on her desk”.
The above action sequence is represented first as a set facts using the “frame” predicate as follows:

frame(1,1,sitting,closed,bookcase,closed).
frame(2,1,standing,closed,bookcase,closed).
frame(3,2,walking,closed,bookcase,closed).
frame(4,3,walking,closed,bookcase,closed).
frame(5,3,standing,open,bookcase,closed).
frame(6,3,walking,open,bookcase,closed).
frame(7,4,walking,open,bookcase,closed).
frame(8,5,walking,open,bookcase,closed).
frame(9,6,standing,open,bookcase,closed).
frame(10,6,standing,open,hand,closed).
frame(11,5,walking,open,hand,closed).
frame(12,4,walking,open,hand,closed).
rame(13,3,walking,open,hand,closed).
frame(14,2,walking,open,hand,closed).
frame(15,1,standing,open,hand,closed).
frame(16,1,standing,open,desk,closed).
frame(17,1,sitting,open,desk,open).

6.2 The Second Example of Action Sequence Description
The second example concerns the same microcosm as above but with a different action sequence. This action sequence of the second example may be described by the sentence
“The agent transported a book from her desk to the bookcase”. This action sequence is represented as a set of facts using the “frame” predicate as follows:

frame(1,1,sitting,closed,desk,closed).
frame(2,1,standing,closed,hand,closed).
frame(3,2,walking,closed,hand,closed).
frame(4,3,walking,closed,hand,closed).
frame(5,3,standing,open,hand,closed).
frame(6,3,walking,open,hand,closed).
frame(7,4,walking,open,hand,closed).
frame(8,5,walking,open,hand,closed).
frame(9,6,standing,open,hand,closed).
frame(10,6,standing,open,bookcase,closed).
frame(11,5,walking,open,bookcase,closed).
frame(12,4,walking,open,bookcase,closed).
frame(13,3,walking,open,bookcase,closed).
frame(14,2,walking,open,bookcase,closed).
frame(15,1,standing,open,bookcase,closed).
frame(16,1,standing,open,bookcase,closed).
frame(17,1,sitting,open,bookcase,open).

entities(agent,door,book,book).

7 QUESTION ANSWERING FROM ACTION SEQUENCES
A future expansion of our system is the implementation of question answering module that may answer questions about the evolution of action in an action sequence. For example when the question "when is the door opened?" the time is given as output. The processing of such a question can be accomplished by rules like:

q1:-q(1,Q),f(Q,when,R1),f(R1,is,R2),f(R2,the,R3), f(R3,door,R4),f(R4,QS,""),state(QS,S),
ans(S,T),write("The time is ",T),nl.
ans(S,T):-frame(T,_,_,S,_,_).

Where:

f(X,W,Z):-fronttoken(X,W,Z) that puts the first word of X in W and the rest in Z.

The processing of such questions involves the syntactic and semantic analysis of the questions. The semantic analysis is grounded on the input formal representations of the images depicting an action sequence.

8 DESCRIPTION OF VISUALIZATIONS OF BRAIN FUNCTIONS
Using modern technology some cognitive functions of the human brain can now be visualized and observed in real time. One example is the observation of the reading process of a human using a MEG (Magnetoencephalogram) [12]. The MEG is obtained by a system that collects magnetic signals from over 200 points on the scull of a human that are processed by computer to give values for the electrical excitation at different areas inside the brain. These deduced excitations are supposed to correspond to activations of the corresponding point of the brain during the performance of a cognitive function. A strong advantage of a MEG system is its time resolution which is about 4msecs and provides the capability of detailed observations.Some Brain MEG Data from an experiment during which reading and saying a word is performed by a human is given in Table 1. The Data were provided by Prof. Andreas Papanikolaou, University of Texas. These data result when a human is reading aloud a word projected to him while being monitored by a MEG system. This human is supposed to perform the cognitive functions of first reading silently the word, then processing it for recognition and finally saying it aloud.

N TIME X Y Z BRAIN AREA

VISUAL

1 256.71 -3.23 -3.80 6.92
2 260.64 -2.90 -3.34 6.71
3 264.58 -2.60 -3.05 6.51
4 268.51 -2.31 -2.84 6.33
5 272.44 -2.03 -2.63 6.18
6 276.37 -1.78 -2.45 6.04
7 280.31 -1.53 -2.27 5.86
8 284.24 -1.28 -2.08 5.61
9 288.17 -0.85 -1.76 5.08
10 343.22 -3.23 -2.67 3.13
11 347.15 -3.26 -2.74 3.25
12 351.09 -3.64 -3.04 3.18
13 355.02 -3.80 -3.24 3.09
14 358.95 -3.80 -3.35 2.97

SPEECH

15 370.75 1.51 5.78 6.02
16 374.68 1.60 5.62 5.82
17 378.61 1.75 5.64 5.67
18 382.54 1.83 5.70 5.53
19 386.48 1.91 5.75 5.40
20 390.41 2.03 5.85 5.31
21 394.34 2.18 5.94 5.23

MOTION

22 465.12 -1.35 3.74 7.89
23 469.05 -1.46 3.35 7.35
24 472.98 -1.50 2.83 6.84
25 555.56 -0.59 1.57 7.65
26 559.49 -0.69 1.93 7.60
27 563.42 -0.65 1.89 7.19
28 567.36 -0.68 1.85 6.94
TABLE 1.: The MEG data from one experiment.

Every cognitive action consists of a number of point activations N. The activations of the different brain areas during the above cognitive actions are supposed to be as follows:

The Visual (V) area activated for silent reading of the word.
The Speech (S) area activated for word processing
The Motion (M) area activated for saying the word.

We may use motion verbs to describe the dynamics of the real time visualization of such cognitive phenomena considering the MEG point activations as elementary events. Using a logical representation of these events high level descriptions of the cognitive actions observed can be described in natural language using the system presented in the present paper. Such a description will require the use of an anatomical database that provides the correspondence of the numerical coordinates of the points of activation with the medical names of the anatomical regions of the brain that these points lie. Example descriptions are: “Activation of the speech area follows activation of the vision area” and “Activation of the motion area follows activation of the speech area”.

9 CONCLUSION
In the present work we aim at the use of the analysis of the definitions of the Motion Verbs for the application to the understanding and description of action sequences such as those included in a video.
The evaluation of the feasibility of useful performance of our system was presented using two examples of the processing of action sequences and explaining how the output descriptive sentence is generated by the system and an example of work in progress for the description of brain MEG imaging sequences.
A future application of this work could be in the automatic text generation of descriptions of motion images obtained by artificial vision systems. These texts may be helpful for people with vision disabilities.
Finally it had shown how the system could be augmented in the direction of multimodal question answering.

ACKNOWLEDGMENT
We thank Prof. Andreas Papanikolaou, University of Texas for the provision of the MEG data.


REFERENCES
[1] J. Kontos, I. Malagardi and M. Pegou, “Processing of Verb Definitions from Dictionaries” 3rd International Conference nf Greek Linguistics pp. 954-961, 1997. Athens (in Greek).
[2] I. Malagardi, “Grouping of Modern Greek Verbs related to Motion using their Definitions” Journal of Glossologia, Athens Greece. 11-12 2000. pp. 282-294 (in Greek).
[3] J. Pustejovsky, C. Havasi, R. Saur, P, Hanks, and A. Rumshisky, “Towards a generative lexical resource: The Brandeis Semantic Ontology” Submitted to LREC 2006, Genoa.
[4] P. Panagi, S. Dasiopoulou, G.Th. Papadopoulos, I. Kompatsiaris and M.G. Strintzis, “A Genetic Algorithm Approach to Ontology-Driven Semantic Image Analysis” 3rd IEE International Conference of Visual Information Engineering (VIE), K-Space Research on Semantic Multimedia Analysis for Annotation and Retrieval special session, 2006. Bangalore, India.
[5] M. Cho, C. Choi and P. Kim, “Measuring Similarity between Trajectories using Motion Verbs in Semantic Level”, ICACT2007, pp. 511- 515, 2007, Korea.
[6] H.-H. Nagel, “Steps toward a Cognitive Vision System” AI Magazine 25(2), pp.31-50, 2004.
[7] H. Dahlkamp, H.-H. Nagel, A. Ottlik, P. Reuter, “A Framework for Model- Based Tracking Experiments in Image Sequences. International Journal of Computer Vision. 73(2), pp. 139-157, 2007.
[8] A. Hakeem, M. Shah, “Learning, detection and representation of multi- agent events in videos” Artificial Intelligence, 2007 Elsevier, (in press).
[9] C.J. Fillmore, “The case for CASE”, in : E. Bach, R. Harms (Eds), Universals in Linguistic Theory, Holt, Rinehart and Winston, New York, pp. 1-88, 1968.
[10] M. Fleischman, P. Decamp and D. Roy, “Mining temporal patterns of movement for video content classification”, International Multimedia Conference. Proceedings of the 8th ACM international workshop on Multimedia information retrieval. Poster Session. 2006. Santa Barbara, California USA.
[11] J. Kontos, I. Malagardi and D. Trikkalidis, “Natural Language Interface to an Agent”. EURISCON ’98 Third European Robotics, Intelligent Systems & Control Conference Athens. Published in Conference Proceedings “Advances in Intelligent Systems: Concepts, Tools and Applications” (Kluwer) pp.211-218, 1998.
[12] R. Salmelin, “Clinical Neurophysiology of Language: The MEG Approach” (Invited Review), Clinical Neurophysiology, ELSEVIER Ireland Ltd, 118, pp. 237-254, 2007.

QUESTION ANSWERING FROM PROCEDURAL SEMANTICS TO MODEL DISCOVERY

QUESTION ANSWERING FROM PROCEDURAL SEMANTICS TO MODEL DISCOVERY

Prof. John Kontos and Dr. Ioanna Malagardi

Encyclopedia of Human Computer Interaction, Edited by Dr. Claude Ghaoui. Idea Group Inc.Hershey, USA (2005)

(abridged)

ABSTRACT
A series of systems that can answer questions from various data or knowledge sources are briefly described and future developments are proposed. The line of development of ideas starts with procedural semantics and leads to human-computer interfaces that support researchers for the study of causal models of systems. The early implementation of question answering systems was based on procedural semantics. Deductive systems appeared later that produce answers implicit in the original database used for answering. Some systems were developed that instead of databases they used collections of texts for extracting the answers. Finally it is described how the information extracted from scientific and technical texts is used by modern systems for the answering of questions concerning the behaviour of causal models using appropriate linguistic and deduction mechanisms. It is predicted that the perfection of such systems will revolutionize the discovery practice of scientists and engineers.

HISTORICAL INTRODUCTION
Question Answering (QA) is one of the branches of Artificial Intelligence (AI) that involves the processing of human language by computer. QA systems accept questions in natural language and generate answers often also in natural language. The answers are derived from databases, text collectons or knowledge bases. The main aim of QA systems is to generate a short answer to a question rather than a list of possibly relevant documents. As it becomes more and more difficult to find answers on the WWW using standard search engines, the technology of QA systems will become increasingly important. A series of systems that can answer questions from various data or knowledge sources are briefly described below. These systems provide a friendly interface to the user of information systems that is particularly important for users who are not computer experts. The line of development of ideas starts with procedural semantics and leads to interfaces that support researchers for the discovery of parameter values of causal models of systems under scientific study. QA systems historically developed roughly during the 1960-1970 decade (Simmons, 1970). A few of the QA systems that were implemented during this decade are:

The BASEBALL System (Green et al, 1961)
The FACT RETRIEVAL System (Cooper, 1964)
The DELFI Systems (Kontos and Papakontantinou, 1970; Kontos and Kossidas, 1971)

The BASEBALL System
This system was implemented in the Lincoln Laboratory and it was the first QA system reported in the literature according to the references cited in the first book with a collection of AI papers (Feigenbaum and Feldman, 1963). The inputs were questions in English about games played by baseball teams. The system transformed the sentences to a form that permits search of a systematically organized memory store for the answers.Both the data and the dictionary were list structures and questions were limited to a single clause.

The FACT RETRIEVAL System
The system was implemented using the COMIT compiler-interpreter system as programming language. A translation algorithm was incorporated into the input routines. This algorithm generates the translation of all information sentences and all question sentences into their logical equivalents.

The DELFI System
The DELFI system answers natural language questions about the space relations between a set of objects. These are questions with unlimited nesting of relative clauses that were automatically translated into retrieval procedures consisting of general purpose procedural components that retrieved information from the database that contained data about the properties of the objects and their space relations.

The DELFI II System
The DELFI II system (Kontos and Kossidas, 1971) was an implementation of the second edition of the system DELFI augmented by deductive capabilities. In this system the procedural semantics of the questions are expressed using macro-instructions that are submitted to a macro-processor that expands them with a set of macro-definitions into full programs. Every macro-instruction corresponded to a procedural semantic component. In this way a program was generated that corresponded to the question and could be compiled and executed in order to generate the answer. DELFI II was used in two new applications. These applications concerned the processing of the database of the personnel of an organization and the answering of questions by deduction from a database with airline flight schedules using the rules:

If flight F1 flies to city C1 and flight F2 departs from city C1 then F2 follows F1.
If flight F1 follows flight F2 and the time of departure of F1 is at least two hours later than the time of arrival of F2 then F1 connects with F2.
If flight F1 connects with flight F2 and F2 departs from city C1 and F1 flies to city C2 then C2 is reachable from C1.

Given a database that contains the data:

F1 departs from Athens at 9 and arrives at Rome at 11
F2 departs from Rome at 14 and arrives at Paris at 15
F3 departs from Rome at 10 and arrives at London at 12

If the question “Is Paris reachable from Athens?” is submitted to the system then the answer it gives is “yes”, because F2 follows F1 and the time of departure of F2 is three hours later than the time of arrival of F1. It should be noted also that F1 departs from Athens and F2 flies to Paris.

If the question “Is London reachable from Athens?” is submitted to the system then the answer it gives is “no”, because F3 follows F1 but the time of departure of F3 is one hour earlier than the time of arrival of F1. It should be noted here that F1 departs from Athens and F3 flies to London.

BACKGROUND

The SQL QA Systems
In order to facilitate the commercial application of the results of research work like the one described so far it was necessary to adapt the methods used to the industrial data base environment. One important adaptation was the implementation of the procedural semantics interpretation of natural language questions using a commercially available database retrieval language. The SQL QA systems implemented by different groups including the author’s followed this direction by using SQL (Structured Query Language) so that the questions can be answered from any commercial database system.
The domain of an illustrative application of our SQL QA system involves information about different countries. The representation of the knowledge of the domain of application connected a verb phrase like “exports” or “has capital” to the corresponding table of the database that the verb is related to. This connection between the verbs and the tables provided the facility of the system to locate the table a question refers to using the verbs of the question. During the analysis of questions by the system an ontology related to the domain of application may be used for the correct translation of ambiguous questions to appropriate SQL queries. Some theoretical analysis of SQL QA systems has appeared recently (Popescu et al, 2003) and a recent system with a relational database is described in Samsonova et al [2003].

QA from Texts Systems
Some QA systems use collections of texts instead of databases for extracting answers. Most such systems are able to answer simple “factoid” questions only. Factoid questions seek an entity involved in a single fact. Some recent publications on QA from texts are Diekema [2003], Doan-Nguyen and Kosseim [2004], Harabagiu et al [2003], Kosseim et al [2003], Nyberg et al [2002], Plamondon and Kosseim [2002], Ramakrishnan [2004], Roussinof and Robles-Flores [2004] and Waldinger et al [2003]. Some future directions of QA from texts are proposed in Maybury [2003]. An international competition between question answering systems from texts has been organized by NIST (National Institute of Standards and Technology (Voorhees, 2001).
In what follows it is described how the information extracted from scientific and technical texts may be used by future systems for the answering of complex questions concerning the behaviour of causal models using appropriate linguistic and deduction mechanisms. An important function of such systems is the automatic generation of a justification or explanation of the answer provided.

The ARISTA System
The implementation of the ARISTA system is a QA system that answers questions by knowledge acquisition from natural language texts and it was first presented in Kontos [1992]. The ARISTA system was based on the representation independent method also called ARISTA for finding the appropriate causal sentences from a text and chaining them by the operation of the system for the discovery of causal chains.
This method achieves causal knowledge extraction through deductive reasoning performed in response to a user's question. This method is an alternative to the traditional method of translating texts into a formal representation before using their content for deductive question answering from texts. The main advantage of the ARISTA method is that since texts are not translated into any representation formalism retranslation is avoided whenever new linguistic or extra linguistic prerequisite knowledge has to be used for improving the text processing required for question answering.
An example text that is an extract from a medical physiology book in the domain of pneumonology and in particular of lung mechanics enhanced by a few general knowledge sentences was used as a first illustrative example of primitive knowledge discovery from texts (Kontos, 1992). The ARISTA system was able to answer questions from that text that require the chaining of causal knowledge acquired from the text and produced answers that were not explicitly stated in the input texts.

The use of Information Extraction
A system using information extraction from texts for QA was presented in Kontos and Malagardi [1999]. The system described had as ultimate aim the creation of flexible information extraction tools capable of accepting natural language questions and generating answers that contained information either directly extracted from the text or extracted after applying deductive inference. The domains examined were oceanography, medical physiology and ancient Greek law (Kontos and Malagardi, 1999). The system consisted of two main subsystems. The first subsystem achieved the extraction of knowledge from individual sentences that was similar to traditional information extraction from texts (Cowie and Lehnert, 1996; Grishman, 1997) while the second subsystem was based on a reasoning process that combines knowledge extracted by the first subsystem for answering questions without the use of a template representation.

QUESTION ANSWERING FOR MODEL DISCOVERY

The AROMA System
A modern development in the area of QA that points to the future is our implementation of the AROMA (ARISTA Oriented Model Adaptation) system. This system is a model-based QA system that may support researchers for the discovery of parameter values of procedural models of systems by answering “What if” questions. (Kontos et al, 2002). The concept of “What if” questions are considered here to involve the computation data of describing the behaviour of a simulated model of a system.
The knowledge discovery process relies on the search for causal chains that in turn relies on the search for sentences containing appropriate natural language phrases. In order to speed up the whole knowledge acquisition process the search algorithm described in Kontos and Malagardi [2001] was used for finding the appropriate sentences for chaining. The increase in speed results because the repeated sentence search is made a function of the number of words in the connecting phrases. This number is usually smaller than the number of sentences of the text that may be arbitrarily large.

The Knowledge Extraction Subsystem
This subsystem integrates partial causal knowledge extracted from a number of different texts. This knowledge is expressed in natural language using causal verbs such as “regulate”, “enhance” and “inhibit”. These verbs usually take as arguments entities such as entity names and process names that occur in the texts that we use for the applications. In this way causal relations are expressed between the entities, processes or entity-process pairs.
The input texts are submitted first to a preprocessing module of the subsystem that converts automatically each sentence into a form that shows word data with numerical information concerning the identification of the sentence that contains the word and its position in that sentence. This conversion has nothing to do with logical representation of the content of the sentences. It should be emphasized that we do not deviate from our ARISTA method with this conversion. We simply annotate each word with information concerning its position within the text. This form of sentences is then parsed and partial texts with causal knowledge are generated.

The Causal Reasoning Subsystem
The output of the first subsystem is used as input to the second subsystem that combines causal knowledge in natural language form to produce answers and model data by deduction not mentioned explicitly in the input text. The operation of this subsystem is based on the ARISTA method. The sentence fragments containing causal knowledge are parsed and the entity-process pairs are recognized. The user questions are processed and reasoning goals are extracted from them. The answers to the user questions that are generated automatically by the reasoning process contain explanations in natural language form. All this is accomplished by the chaining of causal statements using prerequisite knowledge such as ontology to support the reasoning process.

The Simulation Subsystem
The third subsystem is used for modelling the dynamics of a system specified on the basis of the texts processed by the first and second subsystem. The data of the model such as structure and parameter values are extracted from the input texts combined with prerequisite knowledge such as ontology and default process and entity knowledge. The solution of the equations describing the system is accomplished with a program that provides an interface with which the user may test the simulation outputs and manipulate the structure and the parameters of the model.

FUTURE TRENDS
The architecture of the AROMA system is pointing to future trends in the field of QA by serving among other things the processing of “What if” questions. These are questions about what will happen to a system under certain conditions. Implementing systems for the answering “What if” questions will be an important research goal in the future (Maybury 2003) .
Another future trend is the development of systems that may conduct an explanatory dialog with their human user by answering “Why” questions using the simulated behaviour of system models. A “Why question” seeks the reason for the occurrence of certain system behavior.
The work on model discovery QA systems paves the way towards important developments and justifies effort leading to the development of tools and resources aiming at the solution of the problems of model discovery based on larger and more complex texts. These texts may report experimental data that may be used to support the discovery and adaptation of models with computer systems.

CONCLUSION
A series of systems that can answer questions from various data or knowledge sources were briefly described. These systems provide a friendly interface to the user of information systems that is particularly important for users that are not computer experts. The line of development of systems starts with procedural semantics systems and leads to interfaces that support researchers for the discovery of model parameter values of simulated systems. If these efforts for more sophisticated human-computer interfaces succeed then a revolution may take place in the way research and development is conducted in many scientific fields. This revolution will make computer systems even more useful for research and development.

REFERENCES
Cooper, W. S. (1964). Fact Retrieval and Deductive Question–Answering Information Retrieval Systems. Journal of the ACM, Vol. 11, No. 2, pp. 117-137.
Cowie J., and Lehnert, W., (1996). Information Extraction. Communications of the ACM. Vol. 39, No. 1, pp. 80-91.
Diekema, A., R. (2003). What do You Mean? Finding Answers to Complex Questions. New Directions on Question Answering. Papers from 2003 AAAI Spring Symposium. The AAAI Press. USA, pp. 87-93.
Doan-Nguyen, H. and Kosseim, L. (2004). Improving the Precision of a Closed-Domain Question Answering System with Semantic Information. Proceedings of Researche d’ Information Assistee Ordinateur (RIAO-2004). pp. 850-859. Avignon, France. April 2004.
Feigenbaum, E., A. and Feldman, J. (1963). Computers and Thought. McGraw Hill. New York.
Green B. F. et al. (1961). BASEBALL: An Automatic Question Answerer. Proceedings of the Western Joint Computer Conference 19, pp. 219-224.
Grishman R., (1997). Information Extraction: Techniques and Challenges. In Pazienza, M. T. Information Extraction. LNAI Tutorial. Springer, pp. 10-27.
Harabagiu, S., M., Maiorano, S., J. and Pasca, M., A. (2003). Open-domain textual question answering techniques. Natural Language Engineering, Vol. 9 (3). Pp. 231-267.
Kontos J. and Papakonstantinou G., (1970). A question-answering system using program generation. In Proceedings of the ACM International Computing Symposium, Bonn, Germany, pp. 737-750.
Kontos J. and Kossidas A., (1971). On the Question-Answering System DELFI and its Application. Proceedings of the Symposium on Artificial Intelligence, Rome, Italy, pp. 31-36.
Kontos J. (1992) ARISTA: Knowledge Engineering with Scientific Texts. Information and Software Technology, Vol. 34, No 9, pp. 611-616.
Kontos J. and Malagardi I. (1999). Information Extraction and Knowledge Acquisition from Texts using Bilingual Question-Answering. Journal of Intelligent and Robotic Systems, Vol 26, No. 2, pp. 103-122, October.
Kontos J. and Malagardi I. (2001) A Search Algorithm for Knowledge Acquisition from Texts. HERCMA 2001, 5th Hellenic European Research on Computer Mathematics & its Applications Conference, Athens, Greece, pp. 226-230.
Kontos J., Elmaoglou A. and Malagardi I. (2002). ARISTA Causal Knowledge Discovery from Texts. Proceedings of the 5th International Conference on Discovery Science DS 2002, Luebeck, Germany, pp. 348-355.
Kontos, J., Malagardi, I., Peros, J. (2003). “The AROMA System for Intelligent Text Mining” HERMIS International Journal of Computers mathematics and its Applications. Vol. 4. pp.163-173. LEA.
Kontos, J. (2004). Artificial Intelligence. Chapter in Book “Cognitive Science: The New Science of the Mind”. Gutenbeng. Athens 2004. pp. 43-153 (in Greek).
Kosseim, L., Plamondon, L. and Guillemette, L., J. (2003). Answer Formulation for Question-Answering. Proceedings of the Sixteenth Conference of the Canadian Society for Computational Studies of Intelligence. (AI’2003). Lecture Notes in Artificial Intelligence no 2671, pp. 24-34. Springer Verlag. June 2003. Halifax Canada.
Maybury, M., T. (2003). Toward a Question answering Roadmap. New Directions on Question Answering. Papers from 2003 AAAI Spring Symposium. The AAAI Press. USA, pp. 8-11.
Nyberg, E. et al. (2002). The JAVELIN Question-Answering System at TREC 2002. NIST Special Publication:11th Text Retrieval Conference (TREC), 2002.
Plamondon, L. and Kosseim, L. (2002). QUANTUM: A Function-Based Question Answering System. Proceedings of The Fifteenth Canadian Conference on artificial Intelligence (AI’2002). Lecture Notes in artificial Intelligence no. 2338, pp.281-292. Springer Verlag. Berlin . May 2002. Calgary, Canada.
Popescu, A., Etzioni, O. and Kautz, H. (2003). Towards a Theory of Natural Language Interfaces to Databases. Proceedings of IUI’03, Miami, Florida, USA, pp. 149-157.
Ramakrishnan, G. et al. (2004). Is Question Answering an Acquired Skill? (2004). WWW2004. May 2004, New York, USA.
Roussinof, D., Robles-Flores, J., A. (2004). Web Question Answering: Technology and Business Applications. Proceedings of the Tenth Americas Conference on Information Systems. New York. August 2004.
Samsonova, M., Pisarev, A. and Blagov M. (2003). Processing of natural language queries to a relational database. Bioinformatics. Vol. 19 Suppl. 1. pp. i241-i249
Simmons R., F. (1970). Natural Language Question-Answering Systems: 1969. Computational Linguistics, Vol. 13, No 1, January, pp. 15-30.
Voorhees E., M. (2001). The Trec Question answering Track. Natural Language Engineering, Vol. 7, Issue 4.pp.361-378.
Waldinger, R. et al. (2003). Deductive Question Answering from Multiple Resources. New Directions in Question Answering, AAAI 2003.

Terms and Definitions

Ontology: A structure that represents taxonomic or meronomic relations between entities.
Question Answering System: A computer system that can answer a question posed to it by a human being using pre-stored information from a database, a text collection or a knowledge base.
Procedural Semantics: A method for the translation of a question by a computer program into a sequence of actions that retrieve or combine parts of information necessary for answering the question.
Causal Relation: A relation between the members of an entity-process pair where the first member is the cause of the second member that is the effect of the first member.
Causal Chain: A sequence of instances of causal relations such that the effect of each instance but the last one is the cause of the next one in sequence.
Model: A set of causal relations that specify the dynamic behavior of a system.
Explanation: A sequence of statements of the reasons for the behavior of the model of a system.
Model Discovery: The discovery of a set of causal relations that predict the behavior of a system.
“What if” question: A question about what will happen to a system under given conditions or inputs.
“Why” question: A question about the reason for the occurrence of certain system behavior.

Cosmetic Surgery

QUESTION ANSWERING AND Rhetoric ANALYSIS

QUESTION ANSWERING AND Rhetoric ANALYSIS of Biomedical Texts in the AROMA System

 JOHN KONTOS, IOANNA MALAGARDI and JOHN PEROS

7th Hellenic European Research on Computer Mathematics & its Applications Conference. Athens.

Abstract--Question answering with intelligent knowledge management of biomedical texts and analysis of rhetoric relations in the AROMA system is presented. The development of the AROMA system aims at the creation of an intelligent tool for the support of the discovery and adaptation of biomedical models based on data extracted from natural language texts. The system operation includes three main functions namely question answering and text mining and simulation. The question answering function generates model based answers and their explanations. The operation of AROMA allows the exploitation of rhetoric relations between a “basic” text that proposes a model of a biomedical system and parts of the abstracts of papers that present experimental findings supporting the model. An important use of AROMA concerns the comparison of experimental data with the model proposed in the basic text. The AROMA system consists of three subsystems. The first subsystem extracts knowledge including rhetoric relations from biomedical texts. The second subsystem answers questions with causal knowledge extracted by the first subsystem and generates explanations using rhetoric relation knowledge in addition to other knowledge. The third subsystem simulates the time-dependent behavior of a model from which textual descriptions of the waveforms are generated automaticaly.
Index Terms-- rhetoric relations, intelligent biomedical text mining, knowledge discovery from text, simulation, question answering, explanation, p53, mdm2.

I. INTRODUCTION

The new AROMA (Automatic Rhetoric Organizer for Model Analysis) system is presented in the present paper and an illustrative example of application is described. This system is an intelligent computer tool for question answering and text mining [5] of biomedical knowledge including the management of rhetoric knowledge. Parts of the older version of the system were presented in [6], [7], [8], [9], [10].
The development of the AROMA system aims at the creation of an intelligent tool for the support of the discovery and adaptation of biomedical models based on data extracted from natural language texts. The system operation includes two main functions namely question answering and text mining. The question answering function generates model based explanations of the answers. The expanded operation of AROMA allows the exploitation of rhetoric relations between a basic text that proposes a model of a biomedical system and parts of the text of papers that present experimental data supporting the model.
In a typical application of the system a theoretical text presenting the model of a system under scientific investigation is taken as the “basic” one and a computerized method is used for the analysis of the relationship of the model to the texts of the papers that provide experimental support of the model or theory proposed by it. Text mining is applied both to the basic paper and the supporting papers in order to extract the knowledge fragments that have to be rhetorically related and computationally processed. An important aspect of the application of the system is the answering of natural language questions for the computer aided comparison of new experimental data with the predictions of the model proposed in the “basic” paper. The friendly man-machine interface of the system interface is capable of answering such questions and generating explanations that help the user in tracing the support of the model by experimental and background domain knowledge.
It is envisaged that the AROMA System may prove useful for the support of the discovery activity of research scientists with the intelligent management of scientific knowledge mined from texts. Text mining differs from data mining [2] in that it uses unstructured texts for the collection of knowledge rather than structured sources like databases. We define intelligent text mining as the process of creating novel knowledge from texts that is not stated in them by combining sentences with deductive reasoning. An early implementation of this kind of intelligent text mining was reported by us in [4]. A review of different kinds of text mining including intelligent text mining is presented in [11].
There are two possible methodologies of applying deductive reasoning to texts. The first methodology is based on the translation of the texts into some formal representation of their “content” with which deduction is performed [12]. The advantage of this methodology depends on the simplicity and availability of the required inference engine but its serious disadvantage is the need for reprocessing all the texts and storing their translation into a formal representation every time something changes in their domain. In the case of scientific texts what may change is some part of the background knowledge such as the ontology used for deducing new knowledge. The second methodology eliminates the need for translation of the texts into a formal representation because an inference engine capable of performing deductions “on the fly”, i.e. directly from the original texts, is implemented.
A disadvantage of this second methodology is that a more complex inference engine than the one needed by the first one must be built. The implementation of such inference engines has however proved feasible for causal knowledge. The strong advantage of the second methodology is that the translation into a formal representation is avoided. In [4] we proposed the second methodology and the method we developed was therefore called ARISTA i.e. Automatic Representation Independent Syllogistic Text Analysis. The ARISTA method is used in the AROMA system for its basic reasoning functions.
An early attempt was also made by us in [4] to implement a model-based question answering system using the scientific text as a knowledge base describing a qualitative model. One of our early application examples concerned medical text mining related to human respiratory system mechanics [4]. Biomedical text mining is now recognized as a very important field of study [3] and [17] particularly for molecular biology. The system GeneWays [16] is a typical example of a biomedical text analysis system for the extraction of molecular pathway data. The idea of combining text mining with simulation and question answering was pursued further by our group as reported in [7], [8], [9] and [10] as well as in the present paper.
The general architecture of our AROMA system consists of three subsystems namely the Knowledge Extraction Subsystem, the Causal Reasoning Subsystem and the Simulation Subsystem. These subsystems are briefly described below.

II. THE GENERAL ARCHITECTURE OF THE AROMA SYSTEM

A. The Knowledge Extraction Subsystem

This subsystem integrates partial causal knowledge extracted from a number of different texts. This knowledge is expressed in natural language using causal verbs such as “regulate”, “enhance” and “inhibit”. These verbs usually take as arguments entities such as protein names and gene names that occur in the biomedical texts that we use for the present applications. In this way causal relations between entities and processes are expressed. A lexicon containing words such as causal verbs and stop words are used by this subsystem. An output file is produced by the system that contains parts of sentences collected from the original sentences of different input texts. These output file is used for reasoning by the second subsystem. The input files used for this subsystem in the example of p53-mdm2 dynamics contain texts downloaded from MEDLINE. The operation of the subsystem is based on the recognition of noun phrases and verb groups and their relations.

B. The Causal Reasoning Subsystem

The output of the first subsystem is used as input to the second subsystem that combines background knowledge with causal knowledge in natural language form to produce by automatic deduction conclusions not mentioned explicitly in the input text. The operation of this subsystem is based on the ARISTA method [4] and results in the recognition of causal relations on-the-fly of the form “causes(process1, entity1, process2, entity2, manner)”.
The pair (process1, entity1) stands for the cause, the pair (process2, entity2) stands for the effect and “manner” stands for the kind of causality i.e. whether it is positive or negative. The sentence fragments containing causal knowledge are analyzed and the entity-process pairs are recognized. The user questions are analyzed and reasoning goals are extracted from them. The answers to the user questions are generated automatically by a reasoning process together with explanations in natural language form. This is accomplished by the chaining “on the fly” of causal statements using background knowledge such as an ontology to support the reasoning process.

C. The Simulation Subsystem

The third subsystem is used for modelling the dynamics of the biomedical system specified in the “basic” text. The characteristics of the model such as structure and parameter values are extracted from the input texts combined with background knowledge such as ontology and default process and entity knowledge [9]. Considering the p53-mdm2 example two coupled first order differential equations were used as the approximate mathematical model of the biomedical system in rough correspondence with the model proposed in [1]. A basic characteristic of the behaviour of such a system is the occurrence of oscillations for certain values of the parameters of the equations. Two finite difference equations that approximate the differential equations system of the model are:

Δx= a1*x + b1*y + c1*x*y (1)
Δy= a2*y + b2*delay(d,x) (2)

where Δx means the difference between the value of the variable x at the present time and the value of the variable x at the next time instant and delay(d,x) at equation (2) stands for the value that x had d units of time before present time. Time is taken to advance in discrete steps. The symbols x and y are the variables that represent the concentrations of the proteins p53 and mdm2 respectively. The symbols a1, b1, c1, a2, b2 are the coefficients of the equations that represent the parameters of the biomedical system. It is noted that multiplicative term c1*x*y renders equation (1) non-linear. This non-linearity causes the appearance of the oscillations to differ from simple sine waves. The solution of these equations is accomplished with a Prolog program that provides an interface for manipulating the parameters of the model. An important module of the simulation subsystem is one that generates text describing the behaviour of the variables of the model on true. The above information concerning the connection of the parts of the differential equations with the parts of the biomedical system being modelled is formalized using rhetorical relations explained below.

III. THE RHETORIC RELATIONS

Research on the rhetoric or discourse analysis of biomedical texts has started only recently [15]. In [15] an annotation scheme is proposed for a rhetorical analysis of biology articles using the “zoning” method. This method characterizes parts of a scientific text using an annotation scheme to identify these parts or “zones”.
In our system we apply the rhetoric relations approach [13, 14] in contrast with the zoning approach. This approach connects parts of sentences and other textual fragments using rhetoric relations.
About 50 rhetoric relations have been theoretically defined in [13] but only 4 were computationally defined in [14] namely “contrast”, “cause-explanation-evidence”, “elaboration” and “condition” that were defined at a much coarser level of granularity for practical reasons. It is however proposed to enrich this approach of analysis by using a few more rhetoric relations necessary for the representation of the content of scientific texts related to models of systems.
We distinguish between two kinds of purely textual rhetoric relations namely internal to the “basic” paper (symbolized as “inr”) and external to it (symbolized as “exr”). A third kind of relations (symbolized as mbr) is proposed that formalizes the appearance of the waveforms of the time behaviour of the model variables. These waveforms are treated as “numerical narratives” with a “rhetoric” structure of events equivalent to the “pattern structure” of the waveforms where peaks and valleys or maxima and minima play the role of events. All these kinds of relations are briefly described below and illustrated using examples from [1] where “coerel” and “varrel” are of the inr kind, “parbib” and “entfra” are of the exr kind and “behrel” and “tfollows” are of the mbr kind and are defined below.
The formal representation of the rhetoric relations used by our system consists of a single predicate rr(PAR_1,…,PAR_n) where the arguments PAR_1 to PAR_n stand for n parameters that define a rhetoric relation between two rhetorically related “objects”. These objects may be of different kinds such as sentences or other text fragments, equations, equation variables, physical quantities, citations, references or parts of waveforms representing the time behavior of model variables. The formal representation:

rr(relation_name,relation_kind,

first_object_kind,first_object_identifier,

second_object_kind,second_object_identifier).

is used below for the illustration of some rhetorical relations used by our system.

a. Internal Relations (inr)

1) The relation name “coerel” stands for relations between a coefficient “c” of a mathematical model and the corresponding entity property or parameter “ep” of the biomedical system modeled.

Or formally: “rr(coerel,inr,coefficient,c,parameter,ep).

2) The relation name “varrel” stands for relations between a variable “x” of an equation of the model and a physical entity “e” or an entity property “ep” of the biomedical system modeled.

Or formally: “rr(varrel,inr,variable,x,entity,e) and “rr(varrel,inr,variable,x,entityp,ep).

b. External Relations (exr)

3) The relation name “parbib” stands for relations between a parameter “p” of the biomedical system and a bibliographic reference “r” to a paper that contains experimental data that support the inclusion and possibly the numerical value of this parameter.

Or formally: rr(parbib,exr,parameter,p,reference,r).

4) The relation name “entfra” stands for relations between an entity “e” of a biomedical system and a text fragment labeled “f” of a reference text labeled “r” and is symbolized by “r_f” that presents the experimental data that support the inclusion and possibly the numerical values of the properties of the entity “e”.

Or formally: rr(entfra,exr,ent,e,fragment,r_f).

c. Model Behavior Relations (mbr)

5) The relation name “behrel” stands for relations between some coefficient “c” of the model the time behavior of a variable “v” of the model.

Or formally: “rr(behrel,mbr,coefficient,c, variable,v).

6) The relation name “tfollows” stands for time ordering relations between a part in position “wp1” of a waveform representing the time variation of a variable v1 of the model such as a peak or a valley symbolized as wp1_v1 and a part in position “wp2” of a waveform representing the time variation of variable v2 and symbolized as wp2_v2. The kind of object “waveform part” is symbolized as “wpart”.

Or formally: rr(tfollows,mbr,wpart,wp1_v1,wpart,wp2,p2_v2)

IV. SOME RHETORIC RELATIONS IN THE P53-MDM2 EXAMPLE

In the p53-mdm2 example the rhetoric relations extracted concern the text [1] that proposes a mathematical model for the interaction of the proteins p53 and mdm2 as well as the MEDLINE abstracts of papers related to the model proposed by [1] either used or not as references by [1]. These abstracts were downloaded from MEDLINE and contain knowledge of experimental results concerning the interaction of the proteins p53 and mdm2. These proteins are involved in the life cycle of the cell and interact through a negative feedback system. Some rhetoric relations found in the text [1] are:

r1:rr(coerel,inr,coefficient, sourcep53, parameter,synthesis_rate_of_the_p53_protein)
extracted from the sentence: “Here the coefficient sourcep53 specifies the synthesis rate of the p53 protein.”
r2:rr(coerel,inr,coefficient,activity,parameter,p53's_sequence-specific_DNA_binding activity”
extracted from the sentence: “The coefficient activity can include p53's sequence-specific DNA binding activity”
r3:rr(varrer,inr,variable, degradation(t),entity,rate of degradation)
extracted from the sentence: “The variable degradation(t) measures the rate of degradation”
r4:rr(coerel,inr,coefficient, p1,entity, rate_of_p53-independent_mdm2_transcription)
and
r5:rr(parbib,exr,entity, mdm2,reference,26)
extracted from the sentence: “Here the coefficient p1 denotes the rate of p53-independent mdm2 transcription and translation (24)”

The abstracts of two papers presenting experimental data supporting the qualitative model proposed by [1] named with the labels “32” and “92” can be used to illustrate the question answering process of the AROMA system and the use of external rhetoric relations in the explanations generated by the question answering process.

The first abstract labeled as “32” as listed in [1] consists of six sentences from which two are selected by the first subsystem of AROMA and from which the following two sentence fragments are extracted automatically:

1.“The p53 protein regulates the mdm2 gene”
2.“regulates both the activity of the p53 protein”

These fragments are then automatically transformed to Prolog facts in order to be processed by the second subsystem as shown below:

t(“32_5”, “The p53 protein regulates the mdm2 gene”).
t(“32_6”, “regulates both the activity of the p53 protein”).

The labels 32_5 and 32_6 denote that these fragments are extracted from the sentences 5 and 6 of the text with label 32.

The second abstract labeled by the number “92” that was found independently of [1] consists of seven sentences from which two are selected by the first subsystem of AROMA from which the following two sentence fragments are extracted automatically:

3.“The mdm2 gene enhances the tumorigenic potential of cells”
4.“The mdm2 oncogene can inhibit p53_mediated transactivation”
and expressed in the form of Prolog facts as:
t(“92_3”, “The mdm2 gene enhances the tumorigenic potential of cells”).
t(“92_7”, “The mdm2 oncogene can inhibit p53_mediated transactivation”).

The labels 92_3 and 92_7 denote that these fragments are extracted from the sentences 3 and 7 of the text with label 92.

Using the above sentence fragments of the p53-mdm2 example our system discovers the causal negative feedback loop by appropriate chaining of the relevant sentence fragments and represents it as:

p53 +causes mdm2 -causes p53

Where

+causes means “causes increase” and
-causes means “causes decrease or inhibition”.

This is effected by answering the question:

Is there a process loop of p53?
This question is internally represented as the Prolog goal:

“cause(P1,p53,P2,p53,S)”
where P1 and P2 are two process names that the system extracts from the texts and characterize the behavior of the entity p53. S stands for the overall effect of the feedback loop found i.e. whether it is a positive or a negative feedback loop. In this case S is found equal to “-” since a positive causal connection is followed by a negative one.
The short answer automatically generated by our system is:
Yes.
The loop is p53 activity –causes p53 production.

By a short answer we mean a simple answer not connected to any explanation of the reasoning followed for the derivation of the answer.
The long answer automatically generated by our system together with an appropriate explanation is as follows:

The QUESTION is:
“Is there a process loop of p53 ? ”

Represented internally in Prolog as:
cause(P1,p53,P2,p53,S).

USING INFERENCE RULE IR4a
since the DEFAULT entity of is

with rhetoric relations:

rr(entfra,exr,entity,p53,fragment,92_7)
rr(entfra,exr,entity,mdm2,fragment,92_7)

USING INFERENCE RULE IR4b
with rhetoric relations:

rr(entfra,exr,entity,p53,fragment,32_5)
rr(entfra,exr,entity,mdm2,fragment,32_5)

the EXPLANATION is:

since is a kind of p53 protein -causes p53

because

p53 protein +causes gene of mdm2
and
oncogene of mdm2 -causes p53 mediated transactivation of p53.

It should be noted that the combination of sentence fragments (92_7) and (32_5) in a causal chain that forms a closed negative feedback loop is based on two facts of background knowledge.

This background knowledge is inserted manually in our system as Prolog facts and can be stated as:

the DEFAULT entity of is or default(p53_mediated, p53).

is a kind of or kind_of(oncogene, gene).

The above analysis of the text fragments of the example is partially based on the following background linguistic and domain knowledge which is manually inserted as Prolog facts:

Linguistic Knowledge:

kind_of(“the”,“determiner”).
kind_of(“is”,“copula”).
kind_of(“of”,“preposition”)
kind_of(“activated”,“causal_connector”).
kind_of(“inhibits”,“causal_connector”).
kind_of(“regulated”,“causal_connector”).

Domain Knowledge:

kind_of(“protein”,“entity”).
kind_of(“DNA”,“entity”)
kind_of(“p53”,“entity”).
kind_of(“Mdm2”,“entity”).
kind_of(“damage”,“process”).
kind_of(“expression”,“process”).
kind_of(“increase”,“process”).
kind_of(“activity”,“process”).

The above knowledge base fragment contains both linguistic and domain knowledge to support the analysis of the sentences occurring in the corpus. In practice these two parts of knowledge are handled differently by the inference rules of the reasoning module.

V. AUTOMATIC DESCRIPTION OF THE BEHAVIOR OF THE MODEL VARIABLES

The system can generate automatically descriptions of the the time behaviour of the concentration of the two proteins that are represented by the two variables of the model using the “tfollows” rhetoric relation. More details may be found in [9] an [10]. An example is shown below of an automatically produced description of the numerical results produced by the solution of the model equations where T stands for time and protein names capitalized as P53 and MDM2 for displaying emphasis only:

“A peak of P53 at T=3.6 P53=28830 is followed by a peak of MDM2 at T=6.4 MDM2=16550 which is followed by a valley of P53 at T=8.6 P53=-6100 which is followed by a valley of MDM2 at T=14.6 MDM2=9360 which is followed by a peak of P53 at T=17 P53=670”.

VI. CONCLUSIONS

In the present paper we presented the question answering function of our AROMA system with intelligent knowledge management and rhetoric analysis of biomedical texts related to the modeling of a biomedical system. The AROMA system that we have developed consists of three main subsystems. The first subsystem achieves the extraction of knowledge from texts that is related to the structure and the parameters of the biomedical system simulated. The second subsystem is based on a reasoning process that answers questions by combining causal knowledge extracted by the first subsystem with background knowledge and generates explanations of the reasoning followed. The third subsystem is a system simulator written in Prolog that generates the time behavior of the model’s variables. An important feature of the system presented here is its ability for model based non-factoid question answering with the use of rhetoric relation recognition and intelligent causal knowledge extraction from scientific texts with explanation generation and automatic generation of textual descriptions of the dynamic behavior of the model of a biomedical system.

References

[1] Bar-Or, R. L. et al (2000). Generation of oscillations by the p53-Mdm2 feedback loop: A theoretical and experimental study. PNAS, vol. 97, No 21 pp. 11250-11255.
[2] Barrera J. et al (2004). An environment for knowledge discovery in biology. Computers in Biology and Medicine, vol 34 pp 427-447.
[3] Hirshman L. et al (2002). Accomplishments and challenges in literature data mining for biology. Bioinformatics, vol. 18, 12, pp 1553-1561.
[4] Kontos, J. (1992). ARISTA: Knowledge Engineering with Scientific Texts. Information and Software Technology. vol. 34, No 9, pp 611-616.
[5] Kontos, J. and Malagardi, I. (1999). Information Extraction and Knowledge Acquisition from Texts using Bilingual Question-Answering. Journal of Intelligent and Robotic Systems, vol 26, No. 2, pp. 103-122.
[6] Kontos, J. and Malagardi, I. (2001). A Search Algorithm for Knowledge Acquisition from Texts. Proceedings of HERCMA 2001, 5th Hellenic European Research on Computer Mathematics & its Applications Conference. Athens.
[7] Kontos, J. et al (2002a). ARISTA Causal Knowledge Discovery from Texts Discovery Science 2002 Luebeck, Germany. Proceedings of the 5th International Conference DS 2002 Springer Verlag. pp 348-355.
[8] Kontos, J. et al (2002b). System Modeling by Computer using Biomedical Texts. Res Systemica, volume 2, Special Issue, October 2002. (http:/www.afscet.asso.fr/resSystemica/accueil.html).
[9] Kontos J. et al (2003a). The Simulation Subsystem of the AROMA System HERCMA 2003. Proceedings of 6th Hellenic European Research on Computer Mathematics & its Applications Conference, Athens.
[10] Kontos J. et al (2003b). The AROMA System for Intelligent Text Mining. HERMIS, vol 4, pp 163-173.
[11] Kroetze J. A. et al (2003). Differentiating Data- and Text-Mining Terminology. Proceedings of SAICSIT 2003, pp 93-101.
[12] Kuehne S. E. and Forbus K. D. (2004). Capturing QP-relevant Information from Natural Language Text. Proceedings of the 18th International Workshop on Qualitative Reasoning, August, Evanston, Illinois, USA.
[13] Mann, W. C. and Thompson, S. A. (1988). Rhetorical structure theory: toward a functional theory of text organization. Text, 8(3), pp 243-281.
[14] Marcu, D. & Echihabi, A. (2002). An unsupervised approach to recognizing discource relations. ACL 2002.
[15] Mizuta, Y. & Collier N. (2004). An Annotation Scheme for Rhetorical Analysis of Biology Articles. Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC 2004). Lisbon, Portugal.
[16] Rzhetsky, A. et al (2004). GeneWays: a system for extracting, analyzing, visualising and integrating molecular pathway data. Journal of Biomedical Informatics. vol 37, pp. 43-53.
[17] Shatkay, H. and Feldman R. (2003). Mining the Biomedical Literature in the Genomic Era: An Overview. Journal of Computatinal Biology, 10 (3) 821-855.