I’m about to BARE MY SOUL to the internet. Well, the soul of my teenage self. Get ready!

Live Journal, you may remember, was(/is - it does still exist) was a blogging site before we really knew what blogging was. It was both a place to put diary entries and those quizzes that got passed around in the day. It was also used for FanFic and community gathering. I did NOT use my (main) account for fan gathering. (Though I did write some excellent/terrible fanfics. Didn’t everyone have a Lord of Rings self-insert character?)

I don’t really remember when I started this idea, but I thought it would be fun to see just how emo I was in 2006. Let’s go!

Step 0: Load the libraries

library(tidyverse)
library(lubridate)
library(tidytext)
library(hunspell)
library(ggrepel)
library(cowplot)

Step 1: Clean the data

I downloaded all my past Live Journal entities to a folder on my desktop in the same CSV format, so that I could easily load them in for analysis. I am pleasantly surprised that Live Journal made it so easy to download my history like this! I did have to click the same button a ton of time - but I did get all my data.

The next step was to take every journal and separate out the individual words using ‘tidytext.’

lj_words <- lj_df %>%
  select(itemid, eventtime, logtime, subject, current_music, current_mood, event) %>%
  mutate(across(c(eventtime, logtime), ymd_hms),
         year = year(logtime),
         month = month(logtime),
         ) %>%
  mutate(event = str_remove_all(event, "'")) %>%
  unnest_tokens(word, event, token = "words", format = "html", strip_url = FALSE) 

Not included in this blog post, for privacy of my teenage friends, I also cleaned and changed names of my friends and locations to clean the data and protect their privacy. For example, instead of the name “Linda” you may see “nameofsister”.

I was (and continue to be) terrible at spelling words correctly and also terrible at checking what I’ve typed after the fact. I use ‘Hunspell’ here in an attempt to fix some of the most common issues. Does this spell check get everything? No! But alas, I am a terrible speller and we move on in life.

lj_words_spell_check <- lj_words_protect %>%
  anti_join(my_stop_words, by = "word") %>%
  count(word) %>%
  rowwise() %>%
  mutate(spell_check = hunspell(word)) %>%
  filter(length(spell_check) >= 1) %>%
  mutate(suggest = hunspell_suggest(spell_check)) 

lj_correct <- lj_words_spell_check %>%
  filter(length(suggest) > 0) %>%
  mutate(suggest_pick = pluck(suggest, 1)) %>% # just pick the first one because I am lazy
  ungroup() %>%
  unnest(suggest_pick) %>%
  select(word, suggest_pick) 

lj_words_corrected <- lj_words_protect %>%
  left_join(lj_correct, by = "word") %>%
  mutate(word = coalesce(suggest_pick, word)) %>%
  unnest_tokens(output = "word", input = word) # used because sometimes the correction is actually 2+ words now 

Step 2: Now we move on to analysis!

The data is clean, or at least as clean as it is going to get today.

Word counts

I start with TF-IDF. The goal here is to see what I was talking about each year and how it may differ as I got older. As a reminder, I have changed the names of all my friends and family for privacy. That way you don’t know who “nameofbestfriend” is and why I stopped mentioning “nameofbestfriend” in 2006. (We had a bit of a falling out at the end of HS.)

tfidf <- lj_words_corrected %>%
  count(year, word) %>% 
  tidytext::bind_tf_idf(word, year, n) %>%
  anti_join(stop_words, by = "word") %>%
  group_by(year) %>%
  top_n(n = 10, wt = tf_idf) %>%
  ungroup() %>%
  filter(n >= 2) 

Look at 2009 - clearly my only entries were my Norse myth college class. I remember I put a few of my class papers on my Live Journal.

We can look at the differences between TF-IDF and a regular word count, while accounting for stop words.

wordcount <- lj_words_corrected %>%
  count(year, word) %>%
  anti_join(stop_words, by = "word") %>%
  group_by(year) %>%
  top_n(n = 10, wt = n) %>%
  ungroup() %>%
  filter(n >= 3) 

Sentiment

Next I look at sentiment. I remember using live journal to be super angsty. I assumed that I would largely see negative sentiment and words across the years.

df_plot <- lj_words_corrected %>%
  left_join (get_sentiments("bing"), by = "word" ) %>%
  rename(bing_sentiment = sentiment) %>%
  left_join (get_sentiments("nrc"), by = "word" ) %>%
  rename(nrc_sentiment = sentiment) %>%
  pivot_longer(cols = c(bing_sentiment, nrc_sentiment), 
               names_to = "sentiment_type", values_to = "sentiment") %>%
  count(sentiment_type, year, sentiment) %>%
  filter(! is.na(sentiment) ) %>%
  rename(count = n ) %>%
  group_by(sentiment_type, year) %>%
  mutate(total = sum(count)) %>%
  ungroup() %>%
  mutate(percent = count / total,
         year_month = ymd(str_c(year, "01", "01", sep = "-"))
         ) 

Instead, it seems my words were largely more positive then negative. (Outside of 2009 - which is either largely from my anxiety attacks that year or Norse mythology is just super depressing.) Not as ansgty as I remember!

Ah, but did I mark my “current mood” / how are you feeling” part as positive as my words are? You be the judge.

End :)

So there you have it. Was teenage Erin as emo as I thought? Maybe not! Or maybe I wrote all the most emo journals in my physical diary. The world will never know (because those diaries have been lost).