Bilingual Terminology Mining - IIT Bombay

Bilingual Terminology Mining - IIT Bombay

Bilingual Terminology Mining G U I D E : P R O F. P U S H PA K B H AT T A C H A R Y YA B Y: M U N I S H M I N I A ( 0 7 D 0 5 0 1 6 ) P R I YA N K S H A R M A ( 0 7 D 0 5 0 1 7 ) Content Introduction Multilingual Terminology Mining Chain Term Extraction Term Alignment Direct Context-Vector Method Translation of Lexical Units Linguistic Resources

Comparable Corpora Bilingual Dictionary Conclusion Introduction Text mining research generally adopts big is beautiful approach Justified by the need of large amount of data in order to make use of statistic or stochastic methods[1] Hypothesis : The quality rather than the quantity of the corpus matters more in terminology mining Web is used as a Comparable Corpus

The comparability of the corpus should not only be based on the domain or the sub-domain, but also on the type of discourse 1 : Manning and Schtze, 1999 Multi lingual Terminology Mining Chain Architecture : Source language document s WEB Terminology Extraction

Target language document s Terminology Extraction Lexical alignment process Terms to be translated Bilingual Dictionary Translated terms

Architecture A comparable corpora is taken as input Output is a list of single- and multi-word candidate terms along with their candidate translations Processes involved : Term Extraction Term Alignment Direct Context-Vector Method Translation of lexical units

Term Extraction Terminological units extracted are MW terms whose syntactic patterns, expressed using POS tags, correspond either to a canonical or a variation structure For French main patterns are NN N Prep N et N adj For Japanese main patterns are N N N Suff Adj N Pref N

Variants handled are Morphological for both French and Japanese Syntactical for French Compounding for Japanese Variants handled in French Morphological Variant : Morphological modification of one of the components of base form Syntactical Variant : the insertion of another word into the components of the base form Compounding Variant : the agglutination of another word to one of the components of base form Example : scrtion dinsuline (insulin secretion)

Base form : N Prep N pattern Morphological variant : scrtions dinsuline (insulin secretions) Syntactic variant : scrtion pancratique dinsuline (pancreatic insulin secretion) Syntactic variant : scrtions de peptide et dinsuline (insulin and peptide secretion) Variants handled in Japanese Example the MWT (insulin secretion) appears in following form:

Base form : N N pattern : Compounding variant : agglutination of a word at the end of the base form : (insulin secretion ability) Term Alignment It aligns source MWTs with target SWTs or MWTs Direct Context-Vector method: Collect all lexical units in the context of lexical unit i in a window of size n words around i For each lexical unit of source and target language

Obtain a context-vector Vi, which gathers the set of cooccurrences units j associated with the number of times that j and i occur together Normalize context vector Mutual information Log-likelihood Term Alignment: Direct Context-Vector Method

Using Bilingual Dictionary, translate the lexical units of the source context-vector For a word to be translated, compute the similarity between the translated context-vector and all target vectors through vector distance Candidate translations of a lexical units are the target lexical units closest to the translated context-vector acc. to the vector distance Term Alignment: Translation Translation of lexical units Depends on the coverage of bilingual dictionary If bilingual dictionary provides several translations for a lexical unit, consider all of them but weight the different translations by their frequency in the target language

For a MW, possible translations are generated by using compositional method If it is not possible to translate all compositions of MW, MWT is not taken into account in the translation process Composition methods for French and Japanese For Japanese Fatigue chronique (chronic fatigue) for fatigue four translations are possible : two translations for chronique: We generate all combinations[2] of translated elements and select those which refer to an existing MWT in the target language

For French For a multi-word of length n[3], produce all the combinations of MW Unit elements of length less than or equal to n Syndrome de fatigue chronique (chronic fatigue disease) yields the four possible combinations: [Syndrome de fatigue chronique] [Syndrome de fatigue] [chronique] [Syndrome] [fatigue chronique] [Syndrome] [fatigue] [chronique] A direct translation of subpart of the MW is done if present in bilingual dictionary

90% of the candidate terms provided by term extraction are composed of only two content words, so limiting to the combination 4th 2: Grefenstette, 1999 3: Robitaille et al.,2006 Comparable Corpora Are sets of texts in different languages, that are not translations of each other Share some characteristics or features : topic, period, media, author, discourse One of the clearest is ICE -- the International Corpus of English=1, ; (Greenbaum1991) Corpora of around one million words in each of many varieties of English around the world Bilingual Dictionary

A bilingual dictionary or translation dictionary is a specialized dictionary used to translate words or phrases from one language to another. Bilingual dictionaries can be Unidirectional, meaning that they list the meanings of words of one language in another Bidirectional, allowing translation to and from both languages. Conclusion More frequent a term and its translation, the

better is the quality of alignment The discourse categorization of documents allows lexical acquisition to increase precision Including discourse, results in candidate translations of better quality even if the corpus size is reduced by half Gives rise to data sparsity problem Data sparsity problem can be partially solved by using comparable corpora of high quality

References http://en.wikipedia.org/wiki/Bilingual_diction ary http://www.ilc.cnr.it/EAGLES/corpustyp/node 21.html Bilingual terminology mining - using brain, not brawn comparable corpora [Emmanuel Morin, Beatrice Daille, Koichi Takeuchi, and Kyo Kageura 2007] Automatic Extraction of Bilingual Terms from Comparable Corpora in a Popular Science Domain X. Saralegi, I. San Vicente, A. Gurrutxaga Thank You

Recently Viewed Presentations

  • GLOBAL SYSTEM FOR MOBILE (GSM) The Mobile Station

    GLOBAL SYSTEM FOR MOBILE (GSM) The Mobile Station

    Radio link protocol (RLP), specified in GSM link access protocol over the radio link called . LAPDm. LAPD, the link access protocol (LAP) adapted from . ISDN D channel. Message transfer part (MTP), the protocols used for signaling transport on...
  • Web Formalism and the IR limit of 1+1

    Web Formalism and the IR limit of 1+1

    There is an A category of branes/boundary conditions, with amplitudes for emission of BPS particles from the boundary governed by solutions to the MC equation. (Using just IR data we can define an L - algebra and there are ``interaction...
  • Financial Modeling - OpenCourseWare

    Financial Modeling - OpenCourseWare

    In each class we will develop a model or portion thereof. ... Moody's KMV 101. Crystal Ball 101. Matlab 101. Morningstar Encorr 101 or Direct 101. 4] Project ... While in class you will work on Financial Modeling. Students working...
  • Care Act 2014 Guidance

    Care Act 2014 Guidance

    Who are we safeguarding? An adult who: Has need for care and support . Is experiencing, or at risk of, abuse or neglect. As a result of those care and support needs is unable to protect themselves from either the...
  • Local Control Accountability Plan Development

    Local Control Accountability Plan Development

    gned and consistent towards ensuring students are ready for college and careers. Differentiate the performance of schools and distri. cts in reliable and meaningful ways so they receive appropriate support and assistance. Improve performance across the systems
  • Guidelines for Writing Technical Documents

    Guidelines for Writing Technical Documents

    Guidelines for Writing Technical Documents Computer Science 312 Technical Documents Technical documents are written materials that are used to convey factual information. IM, Email, Memos, Specifications, Documentation, Publications Know your audience. This determines how explicit you must be. State facts...
  • ネット上の匿名性 - 九州大学サイバー ...

    ネット上の匿名性 - 九州大学サイバー ...

    Things like analogizing placesoften going from past posts are often done. Same in reality and net. Do not act nuisance or illegal acts in real society or on the net. Do not write criminal acts or similar things. Criminal acts...
  • British Sign Language (BSL) - level 1

    British Sign Language (BSL) - level 1

    Conversation Ask 2-3 questions - follow up Widely vocabulary - Age Birthday Live LEARNING OUTCOMES Aim: To be able to maintain basic social conversations in BSL in the following domains: describing people, colours and clothes Objectives: At the end of...