Coreference Resolution Tools : A first look

2010-09-28 17:44:08 » Natural Language Processing

refer-a-friend

Coreference is where two or more noun phrases refer to the same entity.   This is an integral part of natural languages to avoid repetition, demonstrate possession/relation etc.

Eg:  Harry wouldn’t bother to read “Hogwarts: A History” as long as Hermione is around.  He knows she knows the book by heart.

The different types of coreference includes:
Noun phrases: Hogwarts A history <- the book
Pronouns : Harry <- He
Possessives : her, his, their
Demonstratives:  This boy

Coreference resolution or anaphor resolution is determining what an entity is refering to.  This has profound applications in nlp tasks such as semantic analysis, text summarisation, sentiment analysis etc.

In spite of extensive research, the number of tools available for CR and level of their maturity is much less compared to more established nlp tasks such as parsing.  This is due to the inherent ambiguities in resolution.

The following are some of the tools currently available.   Almost all tools come with bundled sentence deducters, taggers, parsers, named entity recognizers  etc as setting them up all would be tedious.

Tools Demo
Illinois Coreference Package http://cogcomp.cs.illinois.edu/demo/coref/
CherryPicker
Natural Language Synergy Lab http://nlp.i2r.a-star.edu.sg/demo_coref.html
Bart
JavaRAP http://www-appn.comp.nus.edu.sg/%7Erpnlpir/cgi-bin/JavaRAP/JavaRAPdemo.html
GuiTAR
OpenNLP
Reconcile
ARKref http://www.ark.cs.cmu.edu/ARKref/#ex

Let us try using the following sentence from one of the presentations on BART as input. I'm using the demo app wherever possible and where not, I'm installing the same on my local machine.

Queen Elizabeth set about transforming her husband, King George VI, into a viable monarch. Lionel Logue, a renowned speech therapist, was summoned to help the King overcome his speech impediment.

The following equivalence sets need to be identified.

QE: Queen Elizabeth, her
KG: husband, King George VI, the King, his
LL: Lionel Logue, a renowned speech therapist

The results are as follows.

Tools Result Comments
Illinois Coreference Package Lionel Logue(0)
a renowned speech |therapist|(2)
his(8)

Queen Elizabeth(3)
the |King|(5)
King(4)
King |George VI|(7)
transforming her |husband|(6)
her(1)

a viable |monarch|(9)
The ‘his’ and ‘her’ are wrongly matched to the wrong entities.  his is matched to Logue and her is matched to King George
CherryPicker <COREF ID=”1″>Queen</COREF> <COREF ID=”2″>Elizabeth</COREF> set about transforming <COREF ID=”3″ REF=”2″>her</COREF> <COREF ID=”4″>husband</COREF>, <COREF ID=”5″>King</COREF> <COREF ID=”6″ REF=    “5″>George VI</COREF>, into a viable monarch.
<COREF ID=”9″>Lionel Logue</COREF>, a renowned speech <COREF ID=”10″>therapist</COREF>, was summoned to help the <COREF ID=”7″ REF=”5″>King</COREF> overcome <COREF ID=”8″ REF=”5″>his</COREF> sp eech impediment.

Queen
    her
Elizabeth
Husband
King
    George VI
    King
    his
Lionel Logue
Therapist

It is mostly ok, except that Queen Elizabeth is split into two Entities Queen and Elizabeth .  Other than that, it is one of the best results.  Notably it matches King to King George VI and hence, his is correctly mapped to King George VI
Natural Language Synergy Lab 0 Queen Elizabeth set about transforming 0 her husband , King George VI , into a viable monarch. Lionel Logue , a renowned speech therapist , was summoned to help the King overcome 0 his speech impediment .
BART
{person Queen Elizabeth } set about transforming {np {np her } husband } , {person King George VI } , into {np a viable monarch } . {person Lionel Logue } , {np a renowned {np speech } therapist } , was summoned to help {np the King } overcome {np {np his } {np speech } impediment } .

Coreference chain 1

{person Queen Elizabeth }
{np her }
{np a viable monarch }
{np the King }
{np his }

Coreference chain 2

{person Lionel Logue }
{np a renowned {np speech } therapist }

Coreference chain 3

{np speech }
{np speech }
JavaRAP ********Anaphor-antecedent pairs*****
(0,0) Queen Elizabeth <– (0,5) her,
(1,12) the King <– (1,15) his

********Text with substitution*****
Queen Elizabeth set about transforming <Queen Elizabeth’s> husband, King George VI, into a viable monarch.
Lionel Logue, a renowned speech therapist, was summoned to help the King overcome <the King’s> speech impediment.

It has attempted only the pronoun resolution and that has been done well.
GuiTAR Failed
OpenNLP (TOP (S (NP#6 (NNP Queen) (NNP Elizabeth)) (VP (VBD set) (PP (IN about) (S (VP (VBG transforming) (NP (NP (NML#6 (PRP$ her)) (NN husband)) (, ,) (NP (NNP King) (person (NNP George) (NNP VI)))) (, ,) (PP (IN into) (NP (DT a) (JJ viable) (NN monarch))))))) (. .)) )
(TOP (S (NP#1 (NP (person (NNP Lionel) (NNP Logue))) (, ,) (NP#1 (DT a) (JJ renowned) (NN speech) (NN therapist))) (, ,) (VP (VBD was) (VP (VBN summoned) (S (VP (TO to) (VP (VB help) (S (NP (DT the) (NNP King)) (VP (VBN overcome) (NP (NML#1 (PRP$ his)) (NN speech) (NN impediment))))))))) (. .)) )

Lionel Logue
a renowned speech therapist
Queen Elizabeth
her husband

Reconcile <NP NO=”0″ CorefID=”1″>Queen Elizabeth</NP> set about transforming <NP NO=”2″ CorefID=”3″><NP NO=”1″ CorefID=”1″>her</NP> husband</NP>, <NP NO=”3″ CorefID=”3″>King George VI</NP>, into <NP NO=”    4″ CorefID=”4″>a viable monarch</NP>. <NP NO=”5″ CorefID=”6″>Lionel Logue</NP>, <NP NO=”6″ CorefID=”6″>a renowned speech therapist</NP>, was summoned to help <NP NO=”7″ CorefID=”6″>the King</NP    > overcome <NP NO=”9″ CorefID=”9″><NP NO=”8″ CorefID=”6″>his</NP> speech impediment</NP>.

Queen Elizabeth
her
her husband
King George VI
A Viable monarch
Lionel Logue
a renowned speech therapist
the king
his

the king has been wrongly attributed to Lionel Logue, which resulted in his also to be wronlt atttributed.
ARKref [Queen Elizabeth]1 set about transforming [[her]1 husband , [King George VI]2 ,]2 into [a viable monarch] .
[Lionel Logue , [a renowned speech therapist]6 ,]6 was summoned to help [the King]8 overcome [[his]8 speech impediment] .
One of the best results.  only info lacking is linking ‘the King’ to “king George VI”

_As a side note, cherry picker fails with the following error.

cherrypicker1.01/tools/crf++/.libs/lt-crf_test: error while loading shared libraries: libcrfpp.so.0: cannot open shared object file: No such file or directory

To proceed, we need to download CRF++ .  Install it.

Then we need to modify line no 18 in cherrypicker.sh file
tools/crf++/crf_test -m modelmd $1.crf > $1.pred
to
crf_test -m modelmd $1.crf > $1.pred

open a new terminal:
run
sudo ldconfig_

now cherrypicker should work

ARKref and Cherrypicker seem to be the best options available right now.

Are there any other coreference resolution systems that have not been looked at?  Can we add more about the above tools?  Please post your comments.

comments powered by Disqus