custom stemmer with elasticsearch / tire / rails

Tag: ruby-on-rails , elasticsearch , tire , porter-stemmer , snowball Author: situozixun Date: 2014-04-23

Im' searchinkg to ass new stemmer to elastisearch to use with tire / rails

I've found java file (https://github.com/emilis/PolicyFeed/blob/master/src/search/java/org/tartarus/snowball/ext/LithuanianStemmer.java) I've created a jar from this file I've put it in elasticsearch's lib folder

here my rails file

tire.settings :analysis => {
                :filter => {
                  "lt_stemmer" => {
                    "type" => "stemmer",
                    "name" => "lithuanian",
                    "rules_path" => "lt_stemmer.jar"
                  }
                },
                :analyzer => {
                  "lithuanian" => {
                  "type" => "snowball",
                  "tokenizer" => "keyword",
                  "filter" => ["lowercase", "lt_stemmer"]
                },
            },
  } do
  mapping do
    indexes :titre_lt, :analyzer => "lithuanian"

  end

I succeed them to create index and index data, but when I test, it seems it doesn't use the rule in my jar file.

curl -XGET 'localhost:9200/lituanieindex/_analyze?analyzer=lithuanian' -d 'smulki?, dalini?, piln? krovini? pervežimas nuosavais arba partneri? vilkikais su standartin?mis 92 m3 puspriekab?mis ir 120 m3 autotraukiniais;'



{"tokens":[{"token":"smulki?","start_offset":0,"end_offset":7,"type":"<ALPHANUM>","position":1},{"token":"dalini?","start_offset":9,"end_offset":16,"type":"<ALPHANUM>","position":2},{"token":"piln?","start_offset":18,"end_offset":23,"type":"<ALPHANUM>","position":3},{"token":"krovini?","start_offset":24,"end_offset":32,"type":"<ALPHANUM>","position":4},{"token":"pervežima","start_offset":33,"end_offset":43,"type":"<ALPHANUM>","position":5},{"token":"nuosavai","start_offset":44,"end_offset":53,"type":"<ALPHANUM>","position":6},{"token":"arba","start_offset":54,"end_offset":58,"type":"<ALPHANUM>","position":7},{"token":"partneri?","start_offset":59,"end_offset":68,"type":"<ALPHANUM>","position":8},{"token":"vilkikai","start_offset":69,"end_offset":78,"type":"<ALPHANUM>","position":9},{"token":"su","start_offset":79,"end_offset":81,"type":"<ALPHANUM>","position":10},{"token":"standartin?mi","start_offset":82,"end_offset":96,"type":"<ALPHANUM>","position":11},{"token":"92","start_offset":97,"end_offset":99,"type":"<NUM>","position":12},{"token":"m3","start_offset":100,"end_offset":102,"type":"<ALPHANUM>","position":13},{"token":"puspriekab?mi","start_offset":103,"end_offset":117,"type":"<ALPHANUM>","position":14},{"token":"ir","start_offset":118,"end_offset":120,"type":"<ALPHANUM>","position":15},{"token":"120","start_offset":121,"end_offset":124,"type":"<NUM>","position":16},{"token":"m3","start_offset":125,"end_offset":127,"type":"<ALPHANUM>","position":17},{"token":"autotraukiniai","start_offset":128,"end_offset":143,"type":"<ALPHANUM>","position":18}]}

what do I do wrong ?

thanks for help