Combines expressions in single word. For instance, climate change
will
become climatechange
.
tokenization(x, token)
a vector of characters. Typically a vector of titles/abstracts in
which expressions listed in token
must be replaced.
a vector of characters. Words expression to convert in single words
x <- "funrar: An R package to characterize functional rarity"
tokenization(x, token = c("R package", "functional rarity"))
#> [1] "funrar: An Rpackage to characterize functionalrarity"