A multilingual tokenizer to split a string into tokens

If no language is specified, the library will default to English.To specify a language use its two character ISO 639-1 code.Pragmatic Tokenizer will unescape any HTML entities.
A multilingual tokenizer to split a string into tokens #ruby #rubydeveloper #rubyonrails https://rubyonrails.ba/single/a-multilingual-tokenizer-to-split-a-string-into-tokens

Nezir Zahirovic

Contractor Ruby On Rails (8+ years) / MCPD .Net / C# / Asp.Net / CSS / SQL / (11 years)

related articles