SocialCrawlytics: Providing Insight On How Your Content Is Being Shared

It’s funny how subjective using social media tools can be from person to person. Every day, I use at least a half dozen tools to post and track social activity online, primarily SocialFlow, Chartbeat, Google Analytics, Tweetdeck, Facebook Insights, Exact Target.

On occasion, that toolset grows: Google Webmaster Tools, Simply Measured, PageLever, Followerwonk, Raven Tools, Curalate, Twitter Analytics, bitly, Muckrack, Tweetreach, Google Insights, Statigram, IFTTT… and there are more if I count the tools I use on a personal basis. I haven’t used Union metrics for Tumblr yet (frankly I am sitting on my hands until something cheaper comes out), and just today our Founding Editor Danny Sullivan introduced me to another potential tool, Crowdbooster (see this week’s Link Week column from Julie Joyce for more about that).

Many of these tools listen, track performance, provide competitive data, even make recommendations. They are all unique – although they may be working with the same datasets, or use the same APIs, they each provide something useful. Yet despite all of this data and opportunity, aggregating and understanding how our content is being shared across the most popular social channels simply just seems daunting.

Fortunately, while during our end-of-the-year posts, I had the opportunity to dig into Social Crawlytics, a social analysis tool that crawls sites and generates a report that shows how often each URL was shared on different social platforms.

There’s no easy way to run a report like this in Google Analytics (that I know of anyway), and looking at individual URL metrics within social platforms is not efficient. I had the chance to talk to Yousaf Sekander from RocketMill, who created the tool, and asked him a few questions on how the tool works.

Getting Started

Social Crawlytics is free when you first start using it. Once you sign up, you’ll notice you’ll have 5,000 credits. Each credit is for one page on a domain, so you’ll be able to crawl and analyze 5,000 pages right away. You can donate to get more pages, or you may consider signing up for the enterprise level program, depending on what’s best for you.

Be warned: it can take anywhere between 1-6 hours to crawl 5,000 pages, so be prepared to build in some time to generate your reports.

Scheduling Reports

When you are setting up the domain you want to crawl, you have the option to schedule a report. You can choose to skip this step, but if you prefer to monitor a domain regularly, scheduling a report is a good option.

Yousef recommended that a sites like Search Engine Land or Marketing Land that generate a lot daily content and social activity should be scheduled at least twice a month in order to get a historical snapshot of shares for each URL. Of course, each time a you run a report for a site with thousands of pages you’ll use up your credits rather quickly, so use your best judgement.

Notifications

The last step in generating a report is simply filling out the notification form, this will let you know when the crawl is complete and the data is ready to be viewed. This is important: you don’t want to download data that’s incomplete.

Social Shares Reports

Under “Reports” in the left navigation you will see the domain for reports generated – the most recent report will be listed at the bottom. Yousef mentioned that he is still tweaking with the reports, and plans on adding dates to these soon. If you are regularly monitoring a domain and have a report scheduled, it creates a new report each time the crawler runs. All metrics are cumulative, so the most recent report will show the total shares generated at that time.

Below is the last report we ran for Marketing Land, you can see right away that our posts are monthly shared on Twitter, Followed by LinkedIn.

You can also see the most popular authors based on social stats alone. the colored graph are the Page Shares Per Network, where Social Crawlytics takes the top 50 shared pages, and brakes them down by network. Each color block represents a URL; for example, a recent post called Content Marketing for People Who Hate Writing took off on StumbleUpon. That same shade of yellow is represents the same URL in the rest of the graph.

One of the most useful aspects of the tool, however, is the .csv export. Although you can see page-level results within the dashboard, exporting the data as a spreadsheet allows you to customize results and sort by the number of shares that occurred on a specific social channel: Facebook, Twitter, LinkedIn, Digg, Delicious, StumbleUpon, Google+ and Pinterest.

You can also sort by the total number of shares a specific URL received. The tool also goes one step further by including the anchor text pointing to the URL, and the header status code for that page.

Final Thoughts

I asked Yousef his motivation behind building such a great tool:

“Creating compelling content is not cheap nor is it easy, it is imperative to carry out research to find out what type of content has worked for your competitors. I created Social Crawlytics because there was nothing out there that could allow me to carry out this type of research, particularly for large websites.”

Yousef also repeatedly mentioned that Social Crawlytics is a free service, and he does not have any commercial motives at this time. “It is a tool made out of love and passion,”  he said, “I am footing the server bills myself… I don’t ask for money but if a user thinks the service is worthy or useful they can choose to donate.”

Personally, with the countless tools I use regularly,  I can attest that this tool fulfills a professional need, not just for social media marketers and community managers, but for any business who markets online and relies on site traffic and visibility. And not just for internal business reasons, but for competitive analyses as well.

How will you use a tool like this? Share your ideas in the comments below.