Follies in Project Design

Text Mining and Analysis
Sep 27, 2021

For this project, I want to, after all of the poems are finished, conduct some text mining and analysis on them to see if any insights on my own writing can shine through.

Text mining is a process done on unstructured text automatically by artificial intelligence using natural language processing (or NLP). It’s very useful in discovering patterns in a piece of text to help better understand it. Forms of text mining can be word frequency (how often certain words are used), collocation (finding sequences of words that often are written near/next to each other), and concordance (finding the context in which certain words are used). Advanced text mining techniques like sentiment analysis (analyzing the associated emotions/moods with words) or intent detection are also common.

I think it would be really enlightening to conduct some of these analyses on my poetry to get more familiar with the patterns of my own writing style. It’d be useful to see which words, moods, etc. that I fall-back on too often in order to help better diversify my work and make it less homogenous. I tend to use my writing as a form of venting -- I release the (usually negative) emotional build-up I have through my poetry as a way of coping with the stress of daily life. Because of this, my pieces trend towards similar themes and feelings, to the point where I worry that the body of my work is so uniform that reading one poem would be reading them all. I’m going to choose to focus on primarily word frequency, collocation, and sentiment analysis to hopefully better develop my writing and work past some of these issues (plus, I just think it’d be fun!).