Rap Over Time: A Textual Data Analysis

Written By: Amy Huang, Lindsey Weiskopf, Matthew Ottomano, and Rennah Weng

Before we dive into the project, check out our final visualizations here:

Rap Over Time

Introduction

Our project explores the relationship between time, words, and hip-hop. We chose rap as an axis for analysis because of its prolific prose, its reactionary nature to the artist’s current state of the world, and the rapid changes in the rap industry over the past 4 decades. Our investigation was guided by research and the contextualization of the history of hip-hop, capturing time as authentically as possible. With the context to understand our data, the viewer can explore each moment through the words of rap.

Related Work & Inspirations

• This ThePudding project analyzes unique word counts by hip-hop artists. Extending this, we decided to explore rap artists’ dictions across different eras.
• This FiveThirtyEight project shows the relation between hip-hop and politics, inspiring us to parse semantic themes in rap.
• Hip-Hop Word Count shows what rap lyrics have captured about rap culture over time.

Methodology

Our methodology can be defined by three stages:

1. Research

• We conducted preliminary academic research in order to gain a better understanding of hip-hop culture. This scholarship guided our data exploration.
• From our research and review of existing textual analysis, we posed the research question, “How has rap changed over time?” We referenced the book, RAP: A Juxtaposition of the Eras, by V.L Collins Jr, to split our data into four eras: Foundation, Golden, Millennial, and Zoomer.

2. Data Collection

• We found the 10 most popular artists of each era and their most popular album. Then, we used Genius to collect rap lyrics of every song in each artist’s album.
• We used Python libraries to tokenize the lyrics, remove stop words, calculate word frequency, and compiled our data into CSV files, which were later converted into JSON files. Finally, we used SQL to rank words on total word counts and then split the count internally by era.

3. Understanding Data

• We conducted Exploratory Data Analysis on our dataset of rap lyrics across four eras. We used a semantic parser and word association model that we trained using rap lyrics and applied the trained model to our dataset for our semantic analysis.

Design

Our design was primarily guided by the goal of contextualized exploration. We wanted to first present the viewer with background information presented in a simple and fun way. We also wanted to keep our color schemes and consistent throughout our visualizations to emphasize how we chose to separate our data. We initially attempted a more narrative visualization, but we decided that exploratory visualizations better fit our goals while omitting certain biases from our presentation of data.

Implementation

We started our visualizations by looking through the Observable Gallery to gain inspiration. After collecting, cleaning, and exploring our data, we adapted the searchable and zoomable treemap, and the chord diagram templates to represent our data. We created the timeline from scratch.

For the Chord Diagram, we used the python library, Gensim, to train a model on rap songs throughout the eras. We then used this model to get semantically similar words to themes researched such as love, god, and family. Finally, we scanned our data for these words and summed their occurrences into categories like love, god, family, etc. When we were using this data in our chord diagram, Observable was mistaking a really long processing process to an infinite loop due to poor latency. To solve this, we normalized the weights in our matrices.

Check out our final visualizations here:

Rap Over Time

Discussion

After several iterations of our final visualizations, we’re satisfied with our encodings and the progress that we have made throughout this process. Audiences loved our searchable treemap that allows them to explore interesting words like “money” and “police” and see their distribution across eras. However, they prefer to have search suggestions to quickly explore words that we’ve found interesting. Audiences experienced confusion in the chord diagram and were curious about the decision for the various themes of the chord diagram. They asked for further documentation and for the inclusion of additional themes. We want to emphasize that our main goal is to present exploratory visualizations that lead the readers to explore various findings in rap lyrics and link them with historical context to make their own interpretations!

Future Work

An interesting visualization to look at is a sentence tree, in which we match certain words to where they appear in sentences to provide more context.

In addition, we recognize that there are different sub-cultures within hip-hop and several more influential artists. We tried to include a mix of the different artists, however, being able to isolate different sub-genres and analyze them by word and semantics can provide useful insights into the differences within hip-hop.

Acknowledgments

THANK YOU to Professor Chang, Aviva, and Rounak for listening to our concerns and helping us find solutions.

We are grateful for our classmate’s feedback during our in-progress critique, whose comments propelled us forward to a more thoughtful design process.

References

• https://pudding.cool/projects/vocabulary/index.html
• https://observablehq.com/@d3/zoomable-treemap
• https://observablehq.com/@d3/treemap
• https://observablehq.com/@d3/stratify-treemap
• https://observablehq.com/@didoesdigital/2020-timeline
• Rap : the lyrics (entire book)
• Contextualizing Hip Hop Sonic Cool Pose in Late Twentieth– and Twenty–first–century Rap Music
• Geography of American rap: rap diffusion and rap centers
• Rap and Race: It’s Got a Nice Beat, but What about the Message?
• Changing Images of Violence in Rap Music Lyrics: 1979–1997
• “One Day It’ll All Make Sense”: Hip-Hop and Rap Resources for Music Librarians
• Social Context and Musical Content of Rap Music, 1979–1995
• The Relationship between African Self-Consciousness, Cultural Misorientation, Hypermasculinity, and Rap Music Preference
• https://www.rapanalysis.com
• “Never Shoulda Been Let out the Penitentiary”: Gangsta Rap and the Struggle over Racial Identity
• Women of Color in Hip Hop: The Pornographic Gaze
• Religion and Rap Music: An Analysis of Black Church Usag
• 2 “Go back the other way, we’ll stop and eat at Dick’s”: SEATTLE HIP HOP IN THE 1980S
• Chapter 4: RAP, RACE, AND RESISTANCE
• Rap’s Unruly Body: The Postmodern Performance of Black Male Identity on the American Stage
• 3 “We got the talent, microphones, turntables, and crates”: SEATTLE HIP HOP IN THE 1990
• The Fine Art of Rap
• Understanding Rap as Rhetorical Folk-Poetry
• The Rap on Chicano and Black Masculinity: A Content Analysis of Gender Images in Rap Lyrics
• The Words Have Changed But the Ideology Remains the Same: Misogynistic Lyrics in Rap Music
• Black Nationalism and Rap Music
• CONTEMPORARY URRAN MUSIC: Controversial Messages in Hip-Hop and Rap Lyrics
• AMERICAN STUDIES 111: HIP-HOP CULTURE COMING SOON TO A CLASSROOM NEAR YOU?
• Introduction: “Reading” Hip-Hop Discourse in the Twenty-First Century
• Beyond Keeping It Real: OutKast, the Funk Connection, and Afrofuturism
• Deeper than Rap: Expanding Conceptions of Hip-hop Culture and Pedagogy in the English Language Arts Classroom
• Jay Z’s American Gangster
• Introduction: Hip Hop in History: Past, Present, and Future
• Shake It, Baby, Shake It: Consumption and the New Gender Relation in Hip-Hop
• 5 Roads to Zion: Hip Hop’s Search for the City Yet to Come
• Gangstas, Thugs, and Hustlas: Identity and the Code of the Street in Rap Music
• Metaphorical Conceptions in Hip-Hop Music
• Rhythm: Talking That Talk
• The Aesthetics of Rap
• Good kid, m.A.A.d city: Kendrick Lamar’s Autoethnographic Method
• CONCLUSION: “Back when Empire Way became MLK”
• “Elevate My Mind”: Identities for Women in Hip Hop Love Songs
• SIX For the Texas Bama Femme: A Black Queer Femme-inist Reading of Beyoncé’s “Sorry”
• A “Man’s Woman”? Contradictory Messages in the Songs of Female Rappers, 1992–2000
• Challenging the Contradictions of Chicanismo in Chicano Rap Music and Male Culture
• Hip Hop in the United
• “Fight the Power”: Hip Hop and Civil Unrest in Spike Lee’s Do the Right Thing
• Tupac in their Veins: Hip-Hop Alteño and the Semiotics of Urban Indigeneity
• Hip Hop History: From Streets to the Mainstream
• The History of Hip-Hop: 1925 to Now

Thank you for reading!