Relaciones entre palabras
Ir al Capítulo anterior.
Ir al Case study: Netflix.
Este script asume que se han instalado los paquetes igraph y ggraph.
Hasta ahora hemos considerado las palabras como unidades individuales y hemos analizado sus relaciones con el tono o sentimiento del documento. Sin embargo, muchos análisis de texto interesantes se basan en las relaciones entre palabras, ya sea las que tienden a presentarse inmediatamente a otras, o cuáles tienden a coexistir dentro de los mismos documentos.
Cargar librerías
library(dplyr)
library(tidyr)
library(tidytext)
library(forcats)
library(ggplot2)Leer datos
climate_text <- read.csv(file = "data/climate_text.csv")Tokenización por secuencias de palabras (bigramas)
climate_text %>% tibble()## # A tibble: 593 × 4
## station show show_date text
## <chr> <chr> <chr> <chr>
## 1 MSNBC Morning Meeting 2009-09-22 13:00:00 the interio…
## 2 MSNBC Morning Meeting 2009-10-23 13:00:00 corporation…
## 3 CNN CNN Newsroom 2009-12-03 20:00:00 he says he …
## 4 CNN American Morning 2009-12-07 11:00:00 especially …
## 5 MSNBC Morning Meeting 2009-12-08 14:00:00 lots more c…
## 6 MSNBC Countdown With Keith Olbermann 2009-12-10 06:00:00 so they're …
## 7 CNN Sanjay Gupta MD 2009-12-12 12:30:00 let me ask …
## 8 CNN The Situation Room With Wolf Blitzer 2009-12-16 21:00:00 other impor…
## 9 MSNBC Countdown With Keith Olbermann 2009-12-19 01:00:00 let democra…
## 10 MSNBC The Rachel Maddow Show 2010-01-08 04:00:00 you know th…
## # ℹ 583 more rows# En este caso, cada token corresponde a una secuencia de dos palabras (bigrama)
bigram_climate <- climate_text %>%
select(-show_date) %>%
unnest_tokens(input = text, output = bigram, token = "ngrams", n = 2) %>%
tibble()
bigram_climate## # A tibble: 40,483 × 3
## station show bigram
## <chr> <chr> <chr>
## 1 MSNBC Morning Meeting the interior
## 2 MSNBC Morning Meeting interior positively
## 3 MSNBC Morning Meeting positively oozes
## 4 MSNBC Morning Meeting oozes class
## 5 MSNBC Morning Meeting class raves
## 6 MSNBC Morning Meeting raves car
## 7 MSNBC Morning Meeting car magazine
## 8 MSNBC Morning Meeting magazine slick
## 9 MSNBC Morning Meeting slick and
## 10 MSNBC Morning Meeting and sensuous
## # ℹ 40,473 more rowsMuchos bigramas no son informativos (e.g., “of the”, “on the”, “is a”, …). Esto motiva aplicar nuevamente filtros con stop words. En este paso obtendremos las combinaciones de bigramas, identificando aquellos que se producen con más frecuencia.
bigram_climate %>%
count(bigram, sort = TRUE)## # A tibble: 18,336 × 2
## bigram n
## <chr> <int>
## 1 climate change 1584
## 2 change is 237
## 3 of the 178
## 4 on climate 132
## 5 on the 121
## 6 of climate 118
## 7 in the 115
## 8 is a 94
## 9 about climate 93
## 10 this is 84
## # ℹ 18,326 more rowsbigram_counts <- bigram_climate %>%
select(-show) %>%
separate(col = bigram, into = c("word1", "word2")) %>%
# Eliminar stop words en ambas posiciones
filter(!word1 %in% stop_words$word) %>%
filter(!word2 %in% stop_words$word) %>%
# Evitar "global warming"
filter(word1 != "global" ) %>%
# Combinaciones más frecuentes
count(word1, word2, sort = TRUE) %>%
rename(weight = n)
bigram_counts## # A tibble: 3,046 × 3
## word1 word2 weight
## <chr> <chr> <int>
## 1 climate change 1584
## 2 donald trump 40
## 3 al gore 39
## 4 change climate 34
## 5 president obama 25
## 6 james cameron 19
## 7 sea level 19
## 8 white house 17
## 9 causing climate 16
## 10 level rise 16
## # ℹ 3,036 more rowsUna vez se ha determinado la frecuencia de bigramas, podemos definir la red de frecuencia.
library(igraph)##
## Attaching package: 'igraph'## The following object is masked from 'package:tidyr':
##
## crossing## The following objects are masked from 'package:dplyr':
##
## as_data_frame, groups, union## The following objects are masked from 'package:stats':
##
## decompose, spectrum## The following object is masked from 'package:base':
##
## union# Crear un grafo no dirigido con los pares query-subject
g <- bigram_counts %>%
filter(weight > 10) %>%
graph_from_data_frame(directed = FALSE)
# Visualización
library(ggraph)
set.seed(2025)
a <- grid::arrow(type = "closed", length = unit(.15, "inches"))
ggraph(g, layout = "fr") +
geom_edge_link(aes(edge_alpha = weight), show.legend = FALSE,
arrow = a, end_cap = circle(.07, 'inches')) +
geom_node_point(color = "lightblue", size = 5) +
geom_node_text(aes(label = name), vjust = 1, hjust = 1) +
theme_void() 
Common bigrams in climate change narratives, showing those that occurred more than 10 times and where neither word was a stop word.
It may take some experimentation with ggraph to get your networks into a presentable format like this, but the network structure is useful and flexible way to visualize relational tidy data.
bigram_MSNBC <- climate_text %>%
select(-show_date) %>%
unnest_tokens(input = text, output = bigram, token = "ngrams", n = 2) %>%
filter(station == "MSNBC") %>%
select(-show) %>%
separate(col = bigram, into = c("word1", "word2")) %>%
# Eliminar stop words en ambas posiciones
filter(!word1 %in% stop_words$word) %>%
filter(!word2 %in% stop_words$word) %>%
# Evitar "global warming"
filter(word1 != "global") %>%
# Combinaciones más frecuentes
count(word1, word2, sort = TRUE) %>%
rename(weight = n) %>%
tibble() ## Warning: Expected 2 pieces. Additional pieces discarded in 1248 rows [44, 45, 172, 173,
## 314, 315, 356, 357, 386, 387, 416, 417, 447, 448, 460, 461, 593, 594, 598, 599,
## ...].bigram_FOX <- climate_text %>%
select(-show_date) %>%
unnest_tokens(input = text, output = bigram, token = "ngrams", n = 2) %>%
filter(station == "FOX News") %>%
select(-show) %>%
separate(col = bigram, into = c("word1", "word2")) %>%
# Eliminar stop words en ambas posiciones
filter(!word1 %in% stop_words$word) %>%
filter(!word2 %in% stop_words$word) %>%
# Evitar "global warming"
filter(word1 != "global") %>%
# Combinaciones más frecuentes
count(word1, word2, sort = TRUE) %>%
rename(weight = n) %>%
tibble() ## Warning: Expected 2 pieces. Additional pieces discarded in 688 rows [16, 17, 24, 25, 32,
## 33, 83, 84, 161, 162, 165, 166, 203, 204, 227, 228, 236, 379, 380, 415, ...].# Crear un grafo no dirigido con los pares query-subject
g_msnbc <- bigram_MSNBC %>%
filter(weight > 10) %>%
graph_from_data_frame(directed = FALSE)
g_fox <- bigram_FOX %>%
filter(weight > 6) %>%
graph_from_data_frame(directed = FALSE)
# Visualización
set.seed(2025)
a <- grid::arrow(type = "closed", length = unit(.10, "inches"))
ggraph(g_msnbc, layout = "fr") +
geom_edge_link(aes(edge_alpha = weight), show.legend = FALSE,
arrow = a, end_cap = circle(.07, 'inches')) +
geom_node_point(color = "red3", size = 5) +
geom_node_text(aes(label = name), vjust = 1, hjust = 1) +
theme_void() +
labs(
title = "Red de relaciones MSNBC",
subtitle = "Basado en pesos de conexión",
caption = "Fuente: Datos de MSNBC")
ggraph(g_fox, layout = "fr") +
geom_edge_link(aes(edge_alpha = weight), show.legend = FALSE,
arrow = a, end_cap = circle(.07, 'inches')) +
geom_node_point(color = "dodgerblue", size = 5) +
geom_node_text(aes(label = name), vjust = 1, hjust = 1) +
theme_void() +
labs(title = "Red de relaciones MSNBC",
subtitle = "Basado en pesos de conexión",
caption = "Fuente: Datos de MSNBC")
options(repr.plot.width = 14, repr.plot.height = 7)
library(gridExtra)##
## Attaching package: 'gridExtra'## The following object is masked from 'package:dplyr':
##
## combinegrid.arrange(ggraph(g_msnbc, layout = "fr") +
geom_edge_link(aes(edge_alpha = weight), show.legend = FALSE,
arrow = a, end_cap = circle(.07, 'inches')) +
geom_node_point(color = "red3", size = 5) +
geom_node_text(aes(label = name), vjust = 1, hjust = 1) +
theme_graph() +
labs(title = "Red de relaciones MSNBC",
subtitle = "Basado en pesos de conexión"),
ggraph(g_fox, layout = "fr") +
geom_edge_link(aes(edge_alpha = weight), show.legend = FALSE,
arrow = a, end_cap = circle(.07, 'inches')) +
geom_node_point(color = "dodgerblue", size = 5) +
geom_node_text(aes(label = name), vjust = 1, hjust = 1) +
theme_graph() +
labs(title = "Red de relaciones MSNBC",
subtitle = "Basado en pesos de conexión"),
ncol = 2)
In other analyses, we may want to work with the recombined words. For example, we can look at the tf-idf (Chapter 3) of bigrams across news stations. These tf-idf values can be visualized within each station, just as we did for words.
bigram_climate## # A tibble: 40,483 × 3
## station show bigram
## <chr> <chr> <chr>
## 1 MSNBC Morning Meeting the interior
## 2 MSNBC Morning Meeting interior positively
## 3 MSNBC Morning Meeting positively oozes
## 4 MSNBC Morning Meeting oozes class
## 5 MSNBC Morning Meeting class raves
## 6 MSNBC Morning Meeting raves car
## 7 MSNBC Morning Meeting car magazine
## 8 MSNBC Morning Meeting magazine slick
## 9 MSNBC Morning Meeting slick and
## 10 MSNBC Morning Meeting and sensuous
## # ℹ 40,473 more rowsbigram_tf_idf <- bigram_climate %>%
count(station, bigram) %>%
bind_tf_idf(bigram, station, n) %>%
arrange(desc(tf_idf))
bigram_tf_idf## # A tibble: 21,163 × 6
## station bigram n tf idf tf_idf
## <chr> <chr> <int> <dbl> <dbl> <dbl>
## 1 MSNBC against climate 22 0.00114 1.10 0.00126
## 2 MSNBC cameron about 19 0.000988 1.10 0.00109
## 3 MSNBC fight against 19 0.000988 1.10 0.00109
## 4 MSNBC his fight 19 0.000988 1.10 0.00109
## 5 MSNBC james cameron 19 0.000988 1.10 0.00109
## 6 CNN will get 10 0.000947 1.10 0.00104
## 7 FOX News change if 9 0.000842 1.10 0.000925
## 8 FOX News they will 9 0.000842 1.10 0.000925
## 9 CNN because romney 8 0.000757 1.10 0.000832
## 10 CNN change activists 8 0.000757 1.10 0.000832
## # ℹ 21,153 more rowsbigram_tf_idf %>%
group_by(station) %>%
slice_max(tf_idf, n = 10) %>%
mutate(word = reorder_within(bigram, tf_idf, station)) %>% #ordena dentro de cada grupo
ungroup() %>%
ggplot(aes(x = bigram, y = tf_idf, fill = station)) +
geom_col(show.legend = FALSE) +
facet_wrap(~ station, scales = "free_y") +
coord_flip() +
labs(title = "tf-idf values per station",
y = "tf-idf", x = NULL)
These words are, as measured by tf-idf, the most important to each news station.
This is the point of tf-idf; it identifies words that are important to one document within a collection of documents.