Google apparently cares more about giving the best search results than punishing spammers, as it’s returning lyrics site Rap Genius to its high rankings for searches after it was exiled for SEO spam 10 days ago. What looked like a death sentence for Rap Genius’ traffic has turned into a slap on the wrist. Today Rap Genius detailed what it did wrong, and how it ditched the spammy links to get back in Google’s good graces.
Previously on “Rap Genius’ SEO blunders”, the startup had raised $15 million from Andreessen Horowitz to annotate the web. It’s site hosts lyrics, religious texts, legal documents, poems, and news and allows users to add explanations of what they mean. The Rap Genius founders are known as braggadocious rabble-rousers, and they showed off their ridiculousness on stage in an interview with me at TechCrunch Disrupt New York embedded below. There they discussed doing study drugs like Adderall while naked to make sure the stayed home and focused on building the site.
Rap Genius steadly rose to the top of many search result pages thanks to links from bloggers and being venture funded so it doesn’t have to show ads like the aggressive pop-ups and ringtone scams that pollute competing lyrics sites like AZlyrics and MetroLyrics.
But in a sketchy failed attempt at growth hacking, Rap Genius started the “Rap Genius Blog Affiliate” program where it would promote anyone’s blog post through social media in exchange for the blogger inserting sets of links to Rap Genius lyrics into their posts. For example, it asked email filtering startup founder John Marbach to add links to Rap Genius pages for all of Justin Bieber’s new songs in hopes of scamming its way to the top of searches for Bieber lyrics.
The problem is that Google prohibits sites from gaming its search engine ranking algorithm by having links to them added to unrelated web pages and blog posts — which is exactly what Rap Genius was doing. Marbach published the instructions Rap Genius had sent him, which tipped off Google’s search spam czar Matt Cutts who said his team would investigate.
Despite an apology from Rap Genius, we detailed how Google destroyed Rap Genius’ search engine result page rankings, burying them on the fifth or sixth page of results for lyric searches and even searches for “Rap Genius” where they used to rank high. The punishment dealt out on Christmas had a devastating impact on Rap Genius’ traffic since a signficant amount of it comes from Google searches. Quantcast says Rap Genius fell from around 700,000 uniques a day to around 100,000.
At the time, Rap Genius told TechCrunch “We are working with Google right now to resolve this….We’re working on it as fast as we can, and expect to be back on Google very soon.”
Negotiations appear to have panned out well, as today Rap Genius announced “Rap Genius is back on Google. It takes a few days for things to return to normal, but we’re officially back! First of all, we owe a big thanks to Google for being fair and transparent and allowing us back onto their results pages.”
In its lengthy blog post, Rap Genius explains how it initially begged music bloggers to link to it when appropriate. But then the founders Mahbod Moghadam, Tom Lehman, and Ilan Zechory admit “We overstepped, and we deserved to get smacked”, in reference to the shady Blog Affiliate program. “We apologize to Google and our fans for being such morons”, they wrote, showing they sure don’t come from the Snapchat “never say sorry” school of crisis management.
Rap Genius goes on to detail how it got back on Google. The search engine had handed down a “manual action” where it directly manipulated search results to push down Rap Genius URLs as punishment. The reason was for “Unnatural links to your site” that Google explains as “a pattern of unnatural artificial, deceptive, or manipulative links pointing to your site.”
To fix this, Rap Genius had to either have all the spammy links removed, tagged as “nofollow”, or disavowed. But there were hundreds of thousands of these links scattered around the web. So Rap Genius contacted the webmasters it knew, and built a scraper to find the rest of the links. Those it couldn’t have removed or tagged “nofollow” were fed into Google’s Disavow tool that prevents them from influencing search result rankings.
In a move that demonstrates why a bunch of rowdy Yale guys prone to telling tech luminaries like Mark Zuckerberg to fellate them got $15 million from Andreessen, Rap Genius detailed how it built a highly efficient, parellelized scraper. With tools like Nokogiri, Typhoeus, Heroku, and some serious hacking, it created a scraper that found all the links in just 15 minutes. The code snippets the technical details included in the post are surely an attempt to raise Rap Genius’ status amongst engineers it might try to hire.
In the end, it fetched over 177,000 URLs to find and fix or remove spammy links to its site. And apparently that was enough to get Google to restore their SEO standing.
Of course, it likely didn’t hurt that Rap Genius is funded by Andreessen Horowitz, one of the most powerful and well-connected venture capital firms in Silicon Valley. A bootstrapped company without such advantages might not have gotten off as easy, which some could construe as Google playing favorites.
We’ll check back to see whether Rap Genius regains all of its SEO juice, or has any lingering penalty, but Rap Genius’ site is now the top result for searches of “rap genius”, and it’s also again appearing amongst the top results for searches like “Kanye West Blood On The Leaves Lyrics”. Rap Genius is also planning to try to ween itself off such dependence on Google with the release an iOS app next week.
In the end, Google is putting its users first. Though Rap Genius’ tactics may have been deplorable, they provide a much better lyrics site experience than most of their competitors. Sites like AZlyrics and Metrolyrics are covered with scammy ads for $9.99 a month ringtone subscriptions, and are suspected of also engaging in dubious SEO practices. That’s not a music player in that screenpic of AZLyrics above, it’s a deceptive link to another site.
Meanwhile, Rap Genius provides accurate lyrics as well as explanations of what lyrics mean so you can decode obscure metaphors. It also provides SoundCloud embeds so you can hear songs, links to YouTube and Spotify, and even notes about where a song’s samples come from.
Google could have kept Rap Genius in a search ranking dungeon, but it would have just pushed its users to visit worse sites, and that just doesn’t jive with Google’s mission “to organize the world’s information and make it universally accessible and useful.”
For more on the absurdity of Rap Genius, check out:
Google Destroys Rap Genius’ Search Rankings As Punishment For SEO Spam
Video Interview: Ben Horowitz And The Founders Explain Why A16Z Put $15M Into Rap Genius
Rap Genius Is Getting Into Breaking News Analysis With News Genius
Rap Genius Reveals One Of Its Business Models Will Be ‘Enterprise Genius’ Collaborative Tool
Comment