​​Measure Your Mobile Rankings and Search Visibility in Moz Analytics

Posted by jon.white

We have launched a couple of new things in Moz Pro that we are excited to share with you all: Mobile Rankings and a Search Visibility score. If you want, you can jump right in by heading to a campaign and adding a mobile engine, or keep reading for more details!

Track your mobile vs. desktop rankings in Moz Analytics

Mobilegeddon came and went with slightly less fanfare than expected, somewhat due to the vast ‘Mobile Friendly’ updates we all did at super short notice (nice work everyone!). Nevertheless, mobile rankings visibility is now firmly on everyone’s radar, and will only become more important over time.

Now you can track your campaigns’ mobile rankings for all of the same keywords and locations you are tracking on desktop.

For this campaign my mobile visibility is almost 20% lower than my desktop visibility and falling;
I can drill down to find out why

Clicking on this will take you into a new Engines tab within your Keyword Rankings page where you can find a more detailed version of this chart as well as a tabular view by keyword for both desktop and mobile. Here you can also filter by label and location.

Here I can see Search Visibility across engines including mobile;
in this case, for my branded keywords.

We have given an extra engine to all campaigns

We’ve given customers an extra engine for each campaign, increasing the number from 3 to 4. Use the extra slot to add the mobile engine and unlock your mobile data!

We will begin to track mobile rankings within 24 hours of adding to a campaign. Once you are set up, you will notice a new chart on your dashboard showing visibility for Desktop vs. Mobile Search Visibility.

Measure your Search Visibility score vs. competitors

The overall Search Visibility for my campaign

Along with this change we have also added a Search Visibility score to your rankings data. Use your visibility score to track and report on your overall campaign ranking performance, compare to your competitors, and look for any large shifts that might indicate penalties or algorithm changes. For a deeper drill-down into your data you can also segment your visibility score by keyword labels or locations. Visit the rankings summary page on any campaign to get started.

How is Search Visibility calculated?

Good question!

The Search Visibility score is the percentage of clicks we estimate you receive based on your rankings positions, across all of your keywords.

We take each ranking position for each keyword, multiply by an estimated click-thru-rate, and then take the average of all of your keywords. You can think of it as the percentage of your SERPs that you own. The score is expressed as a percentage, though scores of 100% would be almost impossible unless you are tracking keywords using the “site:” modifier. It is probably more useful to measure yourself vs. your competitors rather than focus on the actual score, but, as a rule of thumb, mid-40s is probably the realistic maximum for non-branded keywords.

Jeremy, our Moz Analytics TPM, came up with this metaphor:

Think of the SERPs for your keywords as villages. Each position on the SERP is a plot of land in SERP-village. The Search Visibility score is the average amount of plots you own in each SERP-village. Prime real estate plots (i.e., better ranking positions, like #1) are worth more. A complete monopoly of real estate in SERP-village would equate to a score of 100%. The Search Visibility score equates to how much total land you own in all SERP-villages.

Some neat ways to use this feature

  • Label and group your keywords, particularly when you add them – As visibility score is an average of all of your keywords, when you add or remove keywords from your campaign you will likely see fluctuations in the score that are unrelated to performance. Solve this by getting in the habit of labeling keywords when you add them. Then segment your data by these labels to track performance of specific keyword groups over time.
  • See how location affects your mobile rankings – Using the Engines tab in Keyword Rankings, use the filters to select just local keywords. Look for big differences between Mobile and Desktop where Google might be assuming local intent for mobile searches but not for desktop. Check out how your competitors perform for these keywords. Can you use this data?

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 3 years ago from tracking.feedpress.it

Controlling Search Engine Crawlers for Better Indexation and Rankings – Whiteboard Friday

Posted by randfish

When should you disallow search engines in your robots.txt file, and when should you use meta robots tags in a page header? What about nofollowing links? In today’s Whiteboard Friday, Rand covers these tools and their appropriate use in four situations that SEOs commonly find themselves facing.

For reference, here’s a still of this week’s whiteboard. Click on it to open a high resolution image in a new tab!

Video transcription

Howdy Moz fans, and welcome to another edition of Whiteboard Friday. This week we’re going to talk about controlling search engine crawlers, blocking bots, sending bots where we want, restricting them from where we don’t want them to go. We’re going to talk a little bit about crawl budget and what you should and shouldn’t have indexed.

As a start, what I want to do is discuss the ways in which we can control robots. Those include the three primary ones: robots.txt, meta robots, and—well, the nofollow tag is a little bit less about controlling bots.

There are a few others that we’re going to discuss as well, including Webmaster Tools (Search Console) and URL status codes. But let’s dive into those first few first.

Robots.txt lives at yoursite.com/robots.txt, it tells crawlers what they should and shouldn’t access, it doesn’t always get respected by Google and Bing. So a lot of folks when you say, “hey, disallow this,” and then you suddenly see those URLs popping up and you’re wondering what’s going on, look—Google and Bing oftentimes think that they just know better. They think that maybe you’ve made a mistake, they think “hey, there’s a lot of links pointing to this content, there’s a lot of people who are visiting and caring about this content, maybe you didn’t intend for us to block it.” The more specific you get about an individual URL, the better they usually are about respecting it. The less specific, meaning the more you use wildcards or say “everything behind this entire big directory,” the worse they are about necessarily believing you.

Meta robots—a little different—that lives in the headers of individual pages, so you can only control a single page with a meta robots tag. That tells the engines whether or not they should keep a page in the index, and whether they should follow the links on that page, and it’s usually a lot more respected, because it’s at an individual-page level; Google and Bing tend to believe you about the meta robots tag.

And then the nofollow tag, that lives on an individual link on a page. It doesn’t tell engines where to crawl or not to crawl. All it’s saying is whether you editorially vouch for a page that is being linked to, and whether you want to pass the PageRank and link equity metrics to that page.

Interesting point about meta robots and robots.txt working together (or not working together so well)—many, many folks in the SEO world do this and then get frustrated.

What if, for example, we take a page like “blogtest.html” on our domain and we say “all user agents, you are not allowed to crawl blogtest.html. Okay—that’s a good way to keep that page away from being crawled, but just because something is not crawled doesn’t necessarily mean it won’t be in the search results.

So then we have our SEO folks go, “you know what, let’s make doubly sure that doesn’t show up in search results; we’ll put in the meta robots tag:”

<meta name="robots" content="noindex, follow">

So, “noindex, follow” tells the search engine crawler they can follow the links on the page, but they shouldn’t index this particular one.

Then, you go and run a search for “blog test” in this case, and everybody on the team’s like “What the heck!? WTF? Why am I seeing this page show up in search results?”

The answer is, you told the engines that they couldn’t crawl the page, so they didn’t. But they are still putting it in the results. They’re actually probably not going to include a meta description; they might have something like “we can’t include a meta description because of this site’s robots.txt file.” The reason it’s showing up is because they can’t see the noindex; all they see is the disallow.

So, if you want something truly removed, unable to be seen in search results, you can’t just disallow a crawler. You have to say meta “noindex” and you have to let them crawl it.

So this creates some complications. Robots.txt can be great if we’re trying to save crawl bandwidth, but it isn’t necessarily ideal for preventing a page from being shown in the search results. I would not recommend, by the way, that you do what we think Twitter recently tried to do, where they tried to canonicalize www and non-www by saying “Google, don’t crawl the www version of twitter.com.” What you should be doing is rel canonical-ing or using a 301.

Meta robots—that can allow crawling and link-following while disallowing indexation, which is great, but it requires crawl budget and you can still conserve indexing.

The nofollow tag, generally speaking, is not particularly useful for controlling bots or conserving indexation.

Webmaster Tools (now Google Search Console) has some special things that allow you to restrict access or remove a result from the search results. For example, if you have 404’d something or if you’ve told them not to crawl something but it’s still showing up in there, you can manually say “don’t do that.” There are a few other crawl protocol things that you can do.

And then URL status codes—these are a valid way to do things, but they’re going to obviously change what’s going on on your pages, too.

If you’re not having a lot of luck using a 404 to remove something, you can use a 410 to permanently remove something from the index. Just be aware that once you use a 410, it can take a long time if you want to get that page re-crawled or re-indexed, and you want to tell the search engines “it’s back!” 410 is permanent removal.

301—permanent redirect, we’ve talked about those here—and 302, temporary redirect.

Now let’s jump into a few specific use cases of “what kinds of content should and shouldn’t I allow engines to crawl and index” in this next version…

[Rand moves at superhuman speed to erase the board and draw part two of this Whiteboard Friday. Seriously, we showed Roger how fast it was, and even he was impressed.]

Four crawling/indexing problems to solve

So we’ve got these four big problems that I want to talk about as they relate to crawling and indexing.

1. Content that isn’t ready yet

The first one here is around, “If I have content of quality I’m still trying to improve—it’s not yet ready for primetime, it’s not ready for Google, maybe I have a bunch of products and I only have the descriptions from the manufacturer and I need people to be able to access them, so I’m rewriting the content and creating unique value on those pages… they’re just not ready yet—what should I do with those?”

My options around crawling and indexing? If I have a large quantity of those—maybe thousands, tens of thousands, hundreds of thousands—I would probably go the robots.txt route. I’d disallow those pages from being crawled, and then eventually as I get (folder by folder) those sets of URLs ready, I can then allow crawling and maybe even submit them to Google via an XML sitemap.

If I’m talking about a small quantity—a few dozen, a few hundred pages—well, I’d probably just use the meta robots noindex, and then I’d pull that noindex off of those pages as they are made ready for Google’s consumption. And then again, I would probably use the XML sitemap and start submitting those once they’re ready.

2. Dealing with duplicate or thin content

What about, “Should I noindex, nofollow, or potentially disallow crawling on largely duplicate URLs or thin content?” I’ve got an example. Let’s say I’m an ecommerce shop, I’m selling this nice Star Wars t-shirt which I think is kind of hilarious, so I’ve got starwarsshirt.html, and it links out to a larger version of an image, and that’s an individual HTML page. It links out to different colors, which change the URL of the page, so I have a gray, blue, and black version. Well, these four pages are really all part of this same one, so I wouldn’t recommend disallowing crawling on these, and I wouldn’t recommend noindexing them. What I would do there is a rel canonical.

Remember, rel canonical is one of those things that can be precluded by disallowing. So, if I were to disallow these from being crawled, Google couldn’t see the rel canonical back, so if someone linked to the blue version instead of the default version, now I potentially don’t get link credit for that. So what I really want to do is use the rel canonical, allow the indexing, and allow it to be crawled. If you really feel like it, you could also put a meta “noindex, follow” on these pages, but I don’t really think that’s necessary, and again that might interfere with the rel canonical.

3. Passing link equity without appearing in search results

Number three: “If I want to pass link equity (or at least crawling) through a set of pages without those pages actually appearing in search results—so maybe I have navigational stuff, ways that humans are going to navigate through my pages, but I don’t need those appearing in search results—what should I use then?”

What I would say here is, you can use the meta robots to say “don’t index the page, but do follow the links that are on that page.” That’s a pretty nice, handy use case for that.

Do NOT, however, disallow those in robots.txt—many, many folks make this mistake. What happens if you disallow crawling on those, Google can’t see the noindex. They don’t know that they can follow it. Granted, as we talked about before, sometimes Google doesn’t obey the robots.txt, but you can’t rely on that behavior. Trust that the disallow in robots.txt will prevent them from crawling. So I would say, the meta robots “noindex, follow” is the way to do this.

4. Search results-type pages

Finally, fourth, “What should I do with search results-type pages?” Google has said many times that they don’t like your search results from your own internal engine appearing in their search results, and so this can be a tricky use case.

Sometimes a search result page—a page that lists many types of results that might come from a database of types of content that you’ve got on your site—could actually be a very good result for a searcher who is looking for a wide variety of content, or who wants to see what you have on offer. Yelp does this: When you say, “I’m looking for restaurants in Seattle, WA,” they’ll give you what is essentially a list of search results, and Google does want those to appear because that page provides a great result. But you should be doing what Yelp does there, and make the most common or popular individual sets of those search results into category-style pages. A page that provides real, unique value, that’s not just a list of search results, that is more of a landing page than a search results page.

However, that being said, if you’ve got a long tail of these, or if you’d say “hey, our internal search engine, that’s really for internal visitors only—it’s not useful to have those pages show up in search results, and we don’t think we need to make the effort to make those into category landing pages.” Then you can use the disallow in robots.txt to prevent those.

Just be cautious here, because I have sometimes seen an over-swinging of the pendulum toward blocking all types of search results, and sometimes that can actually hurt your SEO and your traffic. Sometimes those pages can be really useful to people. So check your analytics, and make sure those aren’t valuable pages that should be served up and turned into landing pages. If you’re sure, then go ahead and disallow all your search results-style pages. You’ll see a lot of sites doing this in their robots.txt file.

That being said, I hope you have some great questions about crawling and indexing, controlling robots, blocking robots, allowing robots, and I’ll try and tackle those in the comments below.

We’ll look forward to seeing you again next week for another edition of Whiteboard Friday. Take care!

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 3 years ago from tracking.feedpress.it

5 Spreadsheet Tips for Manual Link Audits

Posted by MarieHaynes

Link auditing is the part of my job that I love the most. I have audited a LOT of links over the last few years. While there are some programs out there that can be quite helpful to the avid link auditor, I still prefer to create a spreadsheet of my links in Excel and then to audit those links one-by-one from within Google Spreadsheets. Over the years I have learned a few tricks and formulas that have helped me in this process. In this article, I will share several of these with you.

Please know that while I am quite comfortable being labelled a link auditing expert, I am not an Excel wizard. I am betting that some of the things that I am doing could be improved upon if you’re an advanced user. As such, if you have any suggestions or tips of your own I’d love to hear them in the comments section!

1. Extract the domain or subdomain from a URL

OK. You’ve downloaded links from as many sources as possible and now you want to manually visit and evaluate one link from every domain. But, holy moly, some of these domains can have THOUSANDS of links pointing to the site. So, let’s break these down so that you are just seeing one link from each domain. The first step is to extract the domain or subdomain from each url.

I am going to show you examples from a Google spreadsheet as I find that these display nicer for demonstration purposes. However, if you’ve got a fairly large site, you’ll find that the spreadsheets are easier to create in Excel. If you’re confused about any of these steps, check out the animated gif at the end of each step to see the process in action.

Here is how you extract a domain or subdomain from a url:

  • Create a new column to the left of your url column.
  • Use this formula:

    =LEFT(B1,FIND(“/”,B1,9)-1)

    What this will do is remove everything after the trailing slash following the domain name. http://www.example.com/article.html will now become http://www.example.com and http://www.subdomain.example.com/article.html will now become http://www.subdomain.example.com.

  • Copy our new column A and paste it right back where it was using the “paste as values” function. If you don’t do this, you won’t be able to use the Find and Replace feature.
  • Use Find and Replace to replace each of the following with a blank (i.e. nothing):
    http://
    https://
    www.

And BOOM! We are left with a column that contains just domain names and subdomain names. This animated gif shows each of the steps we just outlined:

2. Just show one link from each domain

The next step is to filter this list so that we are just seeing one link from each domain. If you are manually reviewing links, there’s usually no point in reviewing every single link from every domain. I will throw in a word of caution here though. Sometimes a domain can have both a good link and a bad link pointing to you. Or in some cases, you may find that links from one page are followed and from another page on the same site they are nofollowed. You can miss some of these by just looking at one link from each domain. Personally, I have some checks built in to my process where I use Scrapebox and some internal tools that I have created to make sure that I’m not missing the odd link by just looking at one link from each domain. For most link audits, however, you are not going to miss very much by assessing one link from each domain.

Here’s how we do it:

  • Highlight our domains column and sort the column in alphabetical order.
  • Create a column to the left of our domains, so that the domains are in column B.
  • Use this formula:

    =IF(B1=B2,”duplicate”,”unique”)

  • Copy that formula down the column.
  • Use the filter function so that you are just seeing the duplicates.
  • Delete those rows. Note: If you have tens of thousands of rows to delete, the spreadsheet may crash. A workaround here is to use “Clear Rows” instead of “Delete Rows” and then sort your domains column from A-Z once you are finished.

We’ve now got a list of one link from every domain linking to us.

Here’s the gif that shows each of these steps:

You may wonder why I didn’t use Excel’s dedupe function to simply deduplicate these entries. I have found that it doesn’t take much deduplication to crash Excel, which is why I do this step manually.

3. Finding patterns FTW!

Sometimes when you are auditing links, you’ll find that unnatural links have patterns. I LOVE when I see these, because sometimes I can quickly go through hundreds of links without having to check each one manually. Here is an example. Let’s say that your website has a bunch of spammy directory links. As you’re auditing you notice patterns such as one of these:

  • All of these directory links come from a url that contains …/computers/internet/item40682/
  • A whole bunch of spammy links that all come from a particular free subdomain like blogspot, wordpress, weebly, etc.
  • A lot of links that all contain a particular keyword for anchor text (this is assuming you’ve included anchor text in your spreadsheet when making it.)

You can quickly find all of these links and mark them as “disavow” or “keep” by doing the following:

  • Create a new column. In my example, I am going to create a new column in Column C and look for patterns in urls that are in Column B.
  • Use this formula:

    =FIND(“/item40682”,B1)
    (You would replace “item40682” with the phrase that you are looking for.)

  • Copy this formula down the column.
  • Filter your new column so that you are seeing any rows that have a number in this column. If the phrase doesn’t exist in that url, you’ll see “N/A”, and we can ignore those.
  • Now you can mark these all as disavow

4. Check your disavow file

This next tip is one that you can use to check your disavow file across your list of domains that you want to audit. The goal here is to see which links you have disavowed so that you don’t waste time reassessing them. This particular tip only works for checking links that you have disavowed on the domain level.

The first thing you’ll want to do is download your current disavow file from Google. For some strange reason, Google gives you the disavow file in CSV format. I have never understood this because they want you to upload the file in .txt. Still, I guess this is what works best for Google. All of your entries will be in column A of the CSV:

What we are going to do now is add these to a new sheet on our current spreadsheet and use a VLOOKUP function to mark which of our domains we have disavowed.

Here are the steps:

  • Create a new sheet on your current spreadsheet workbook.
  • Copy and paste column A from your disavow spreadsheet onto this new sheet. Or, alternatively, use the import function to import the entire CSV onto this sheet.
  • In B1, write “previously disavowed” and copy this down the entire column.
  • Remove the “domain:” from each of the entries by doing a Find and Replace to replace domain: with a blank.
  • Now go back to your link audit spreadsheet. If your domains are in column A and if you had, say, 1500 domains in your disavow file, your formula would look like this:

    =VLOOKUP(A1,Sheet2!$A$1:$B$1500,2,FALSE)

When you copy this formula down the spreadsheet, it will check each of your domains, and if it finds the domain in Sheet 2, it will write “previously disavowed” on our link audit spreadsheet.

Here is a gif that shows the process:

5. Make monthly or quarterly disavow work easier

That same formula described above is a great one to use if you are doing regular repeated link audits. In this case, your second sheet on your spreadsheet would contain domains that you have previously audited, and column B of this spreadsheet would say, “previously audited” rather than “previously disavowed“.

Your tips?

These are just a few of the formulas that you can use to help make link auditing work easier. But there are lots of other things you can do with Excel or Google Sheets to help speed up the process as well. If you have some tips to add, leave a comment below. Also, if you need clarification on any of these tips, I’m happy to answer questions in the comments section.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 3 years ago from tracking.feedpress.it

Stream Splitting- Video Optimization

4 min Chalk talk about using new WAN optimization technologies to remove the bandwidth cost for live and on-demand video on corporate WANs and Internet gateway connections.

Reblogged 3 years ago from www.youtube.com

​Inbound Lead Generation: eCommerce Marketing’s Missing Link

Posted by Everett

If eCommerce businesses hope to remain competitive with Amazon, eBay, big box brands, and other online retail juggernauts, they’ll need to learn how to conduct content marketing, lead generation, and contact nurturing as part of a comprehensive inbound marketing strategy.

First, I will discuss some of the ways most online retailers are approaching email from the bottom of the funnel upward, and why this needs to be turned around. Then we can explore how to go about doing this within the framework of “Inbound Marketing” for eCommerce businesses. Lastly, popular marketing automation and email marketing solutions are discussed in the context of inbound marketing for eCommerce.

Key differences between eCommerce and lead generation approaches to email

Different list growth strategies

Email acquisition sources differ greatly between lead gen. sites and online stores. The biggest driver of email acquisition for most eCommerce businesses are their shoppers, especially when the business doesn’t collect an email address for their contact database until the shopper provides it during the check-out process—possibly, not until the very end.

With most B2B/B2C lead gen. websites, the entire purpose of every landing page is to get visitors to submit a contact form or pick up the phone. Often, the price tag for their products or services is much higher than those of an eCommerce site or involves recurring payments. In other words, what they’re selling is more difficult to sell. People take longer to make those purchasing decisions. For this reason, leads—in the form of contact names and email addresses—are typically acquired and nurtured without having first become a customer.

Contacts vs. leads

Whether it is a B2B or B2C website, lead gen. contacts (called leads) are thought of as potential customers (clients, subscribers, patients) who need to be nurtured to the point of becoming “sales qualified,” meaning they’ll eventually get a sales call or email that attempts to convert them into a customer.

On the other hand, eCommerce contacts are often thought of primarily as existing customers to whom the marketing team can blast coupons and other offers by email.

Retail sites typically don’t capture leads at the top or middle of the funnel. Only once a shopper has checked out do they get added to the list. Historically, the buying cycle has been short enough that eCommerce sites could move many first-time visitors directly to customers in a single visit.
But this has changed.

Unless your brand is very strong—possibly a luxury brand or one with an offline retail presence—it is probably getting more difficult (i.e. expensive) to acquire new customers. At the same time, attrition rates are rising. Conversion optimization helps by converting more bottom of the funnel visitors. SEO helps drive more traffic into the site, but mostly for middle-of-funnel (category page) and bottom-of-funnel (product page) visitors who may not also be price/feature comparison shopping, or are unable to convert right away because of device or time limitations.

Even savvy retailers publishing content for shoppers higher up in the funnel, such as buyer guides and reviews, aren’t getting an email address and are missing a lot of opportunities because of it.

attract-convert-grow-funnel-inflow-2.jpg

Here’s a thought. If your eCommerce site has a 10 percent conversion rate, you’re doing pretty good by most standards. But what happened to the other 90 percent of those visitors? Will you have the opportunity to connect with them again? Even if you bump that up a few percentage points with retargeting, a lot of potential revenue has seeped out of your funnel without a trace.

I don’t mean to bash the eCommerce marketing community with generalizations. Most lead gen. sites aren’t doing anything spectacular either, and a lot of opportunity is missed all around.

There are many eCommerce brands doing great things marketing-wise. I’m a big fan of
Crutchfield for their educational resources targeting early-funnel traffic, and Neman Tools, Saddleback Leather and Feltraiger for the stories they tell. Amazon is hard to beat when it comes to scalability, product suggestions and user-generated reviews.

Sadly, most eCommerce sites (including many of the major household brands) still approach marketing in this way…

The ol’ bait n’ switch: promising value and delivering spam

Established eCommerce brands have gigantic mailing lists (compared with lead gen. counterparts), to whom they typically send out at least one email each week with “offers” like free shipping, $ off, buy-one-get-one, or % off their next purchase. The lists are minimally segmented, if at all. For example, there might be lists for repeat customers, best customers, unresponsive contacts, recent purchasers, shoppers with abandoned carts, purchases by category, etc.

The missing points of segmentation include which campaign resulted in the initial contact (sometimes referred to as a cohort) and—most importantly—the persona and buying cycle stage that best applies to each contact.

Online retailers often send frequent “blasts” to their entire list or to a few of the large segments mentioned above. Lack of segmentation means contacts aren’t receiving emails based on their interests, problems, or buying cycle stage, but instead, are receiving what they perceive as “generic” emails.

The result of these missing segments and the lack of overarching strategy looks something like this:

My, What a Big LIST You Have!

iStock_000017047747Medium.jpg

TIME reported in 2012 on stats from Responsys that the average online retailer sent out between five and six emails the week after Thanksgiving. Around the same time, the Wall Street Journal reported that the top 100 online retailers sent an average of 177 emails apiece to each of their contacts in 2011. Averaged out, that’s somewhere between three and four emails each week that the contact is receiving from these retailers.

The better to SPAM you with!

iStock_000016088853Medium.jpg

A 2014 whitepaper from SimpleRelevance titled
Email Fail: An In-Depth Evaluation of Top 20 Internet Retailer’s Email Personalization Capabilities (
PDF) found that, while 70 percent of marketing executives believed personalization was of “utmost importance” to their business…

“Only 17 percent of marketing leaders are going beyond basic transactional data to deliver personalized messages to consumers.”

Speaking of email overload, the same report found that some major online retailers sent ten or more emails per week!

simplerelevance-email-report-frequency.png

The result?

All too often, the eCommerce business will carry around big, dead lists of contacts who don’t even bother reading their emails anymore. They end up scrambling toward other channels to “drive more demand,” but because the real problems were never addressed, this ends up increasing new customer acquisition costs.

The cycle looks something like this:

  1. Spend a fortune driving in unqualified traffic from top-of-the-funnel channels
  2. Ignore the majority of those visitors who aren’t ready to purchase
  3. Capture email addresses only for the few visitors who made a purchase
  4. Spam the hell out of those people until they unsubscribe
  5. Spend a bunch more money trying to fill the top of the funnel with even more traffic

It’s like trying to fill your funnel with a bucket full of holes, some of them patched with band-aids.

The real problems

  1. Lack of a cohesive strategy across marketing channels
  2. Lack of a cohesive content strategy throughout all stages of the buying cycle
  3. Lack of persona, buying cycle stage, and cohort-based list segmentation to nurture contacts
  4. Lack of tracking across customer touchpoints and devices
  5. Lack of gated content that provides enough value to early-funnel visitors to get them to provide their email address

So, what’s the answer?

Inbound marketing allows online retailers to stop competing with Amazon and other “price focused” competitors with leaky funnels, and to instead focus on:

  1. Persona-based content marketing campaigns designed to acquire email addresses from high-quality leads (potential customers) by offering them the right content for each stage in their buyer’s journey
  2. A robust marketing automation system that makes true personalization scalable
  3. Automated contact nurturing emails triggered by certain events, such as viewing specific content, abandoning their shopping cart, adding items to their wish list or performing micro-conversions like downloading a look book
  4. Intelligent SMM campaigns that match visitors and customers with social accounts by email addresses, interests and demographics—as well as social monitoring
  5. Hyper-segmented email contact lists to support the marketing automation described above, as well as to provide highly-customized email and shopping experiences
  6. Cross-channel, closed loop reporting to provide a complete “omnichannel” view of online marketing efforts and how they assist offline conversions, if applicable

Each of these areas will be covered in more detail below. First, let’s take a quick step back and define what it is we’re talking about here.

Inbound marketing: a primer

A lot of people think “inbound marketing” is just a way some SEO agencies are re-cloaking themselves to avoid negative associations with search engine optimization. Others think it’s synonymous with “internet marketing.” I think it goes more like this:

Inbound marketing is to Internet marketing as SEO is to inbound marketing: One piece of a larger whole.

There are many ways to define inbound marketing. A cursory review of definitions from several trusted sources reveals some fundamental similarities :

Rand Fishkin

randfishkin.jpeg

“Inbound Marketing is the practice of earning traffic and attention for your business on the web rather than buying it or interrupting people to get it. Inbound channels include organic search, social media, community-building content, opt-in email, word of mouth, and many others. Inbound marketing is particularly powerful because it appeals to what people are looking for and what they want, rather than trying to get between them and what they’re trying to do with advertising. Inbound’s also powerful due to the flywheel-effect it creates. The more you invest in Inbound and the more success you have, the less effort required to earn additional benefit.”


Mike King

mikeking.jpeg

“Inbound Marketing is a collection of marketing activities that leverage remarkable content to penetrate earned media channels such as Organic Search, Social Media, Email, News and the Blogosphere with the goal of engaging prospects when they are specifically interested in what the brand has to offer.”

This quote is from 2012, and is still just as accurate today. It’s from an
Inbound.org comment thread where you can also see many other takes on it from the likes of Ian Lurie, Jonathon Colman, and Larry Kim.


Inflow

inflow-logo.jpeg

“Inbound Marketing is a multi-channel, buyer-centric approach to online marketing that involves attracting, engaging, nurturing and converting potential customers from wherever they are in the buying cycle.”

From Inflow’s
Inbound Services page.


Wikipedia

wikipedia.jpeg

“Inbound marketing refers to marketing activities that bring visitors in, rather than marketers having to go out to get prospects’ attention. Inbound marketing earns the attention of customers, makes the company easy to be found, and draws customers to the website by producing interesting content.”

From
Inbound Marketing – Wikipedia.


Larry-Kim.jpeg

Larry Kim

“Inbound marketing” refers to marketing activities that bring leads and customers in when they’re ready, rather than you having to go out and wave your arms to try to get people’s attention.”

Via
Marketing Land in 2013. You can also read more of Larry Kim’s interpretation, along with many others, on Inbound.org.


Hubspot

“Instead of the old outbound marketing methods of buying ads, buying email lists, and praying for leads, inbound marketing focuses on creating quality content that pulls people toward your company and product, where they naturally want to be.”

Via
Hubspot, a marketing automation platform for inbound marketing.

When everyone has their own definition of something, it helps to think about what they have in common, as opposed to how they differ. In the case of inbound, this includes concepts such as:

  • Pull (inbound) vs. push (interruption) marketing
  • “Earning” media coverage, search engine rankings, visitors and customers with outstanding content
  • Marketing across channels
  • Meeting potential customers where they are in their buyer’s journey

Running your first eCommerce inbound marketing campaign

Audience personas—priority no. 1

The magic happens when retailers begin to hyper-segment their list based on buyer personas and other relevant information (i.e. what they’ve downloaded, what they’ve purchased, if they abandoned their cart…). This all starts with audience research to develop personas. If you need more information on persona development, try these resources:

Once personas are developed, retailers should choose one on which to focus. A complete campaign strategy should be developed around this persona, with the aim of providing the “right value” to them at the “right time” in their buyer’s journey.

Ready to get started?

We’ve developed a quick-start guide in the form of a checklist for eCommerce marketers who want to get started with inbound marketing, which you can access below.

inbound ecommerce checklist

Hands-on experience running one campaign will teach you more about inbound marketing than a dozen articles. My advice: Just do one. You will make mistakes. Learn from them and get better each time.

Example inbound marketing campaign

Below is an example of how a hypothetical inbound marketing campaign might play out, assuming you have completed all of the steps in the checklist above. Imagine you handle marketing for an online retailer of high-end sporting goods.

AT Hiker Tommy campaign: From awareness to purchase

When segmenting visitors and customers for a “high-end sporting goods / camping retailer” based on the East Coast, you identified a segment of “Trail Hikers.” These are people with disposable income who care about high-quality gear, and will pay top dollar if they know it is tested and reliable. The top trail on their list of destinations is the
Appalachian Trail (AT).

Top of the Funnel: SEO & Strategic Content Marketing

at-tommy.jpg

Tommy’s first action is to do “top of the funnel” research from search engines (one reason why SEO is still so important to a complete inbound marketing strategy).

A search for “Hiking the Appalachian Trail” turns up your article titled “What NOT to Pack When Hiking the Appalachian Trail,” which lists common items that are bulky/heavy, and highlights slimmer, lighter alternatives from your online catalog.

It also highlights the difference between cheap gear and the kind that won’t let you down on your 2,181 mile journey through the wilderness of Appalachia, something you learned was important to Tommy when developing his persona. This allows you to get the company’s value proposition of “tested, high-end, quality gear only” in front of readers very early in their buyer’s journey—important if you want to differentiate your site from all of the retailers racing Amazon to the bottom of their profit margins.

So far you have yet to make “contact” with AT Hiker Tommy. The key to “acquiring” a contact before the potential customer is ready to make a purchase is to provide something of value to that specific type of person (i.e. their persona) at that specific point in time (i.e. their buying cycle stage).

In this case, we need to provide value to AT Hiker Tommy while he is getting started on his research about hiking the Appalachian Trail. He has an idea of what gear not to bring, as well as some lighter, higher-end options sold on your site. At this point, however, he is not ready to buy anything without researching the trail more. This is where retailers lose most of their potential customers. But not you. Not this time…

Middle of the funnel: Content offers, personalization, social & email nurturing

at-hiker-ebook.png

On the “What NOT to Pack When Hiking the Appalachian Trail” article (and probably several others), you have placed a call-to-action (CTA) in the form of a button that offers something like:

Download our Free 122-page Guide to Hiking the Appalachian Trail

This takes Tommy to a landing page showcasing some of the quotes from the book, and highlighting things like:

“We interviewed over 50 ‘thru-hikers’ who completed the AT and have curated and organized the best first-hand tips, along with our own significant research to develop a free eBook that should answer most of your questions about the trail.”

By entering their email address potential customers agree to allow you to send them the free PDF downloadable guide to hiking the AT, and other relevant information about hiking.

An automated email is sent with a link to the downloadable PDF guide, and several other useful content links, such as “The AT Hiker’s Guide to Gear for the Appalachian Trail”—content designed to move Tommy further toward the purchase of hiking gear.

If Tommy still has not made a purchase within the next two weeks, another automated email is sent asking for feedback about the PDF guide (providing the link again), and to again provide the link to the “AT Hiker’s Guide to Gear…” along with a compelling offer just for him, perhaps “Get 20% off your first hiking gear purchase, and a free wall map of the AT!”

Having Tommy’s email address also allows you to hyper-target him on social channels, while also leveraging his initial visit to initiate retargeting efforts.

Bottom of the funnel: Email nurturing & strategic, segmented offers

Eventually Tommy makes a purchase, and he may or may not receive further emails related to this campaign, such as post-purchase emails for reviews, up-sells and cross-sells.

Upon checkout, Tommy checked the box to opt-in to weekly promotional emails. He is now on multiple lists. Your marketing automation system will automatically update Tommy’s status from “Contact” or lead, to “Customer” and potentially remove or deactivate him from the marketing automation system database. This is accomplished either by default integration features, or with the help of integration tools like
Zapier and IFTTT.

You have now nurtured Tommy from his initial research on Google all the way to his first purchase without ever having sent a spammy newsletter email full of irrelevant coupons and other offers. However, now that he is a loyal customer, Tommy finds value in these bottom-of-funnel email offers.

And this is just the start

Every inbound marketing campaign will have its own mix of appropriate channels. This post has focused mostly on email because acquiring the initial permission to contact the person is what fuels most of the other features offered by marketing automation systems, including:

  • Personalization of offers and other content on the site.
  • Knowing exactly which visitors are interacting on social media
  • Knowing where visitors and social followers are in the buying cycle and which persona best represents them, among other things.
  • Smart forms that don’t require visitors to put in the same information twice and allow you to build out more detailed profiles of them over time.
  • Blogging platforms that tie into email and marketing automation systems
  • Analytics data that isn’t blocked by Google and is tied directly to real people.
  • Closed-loop reporting that integrates with call-tracking and Google’s Data Import tool
  • Up-sell, cross-sell, and abandoned cart reclamation features
Three more things…
  1. If you can figure out a way to get Tommy to “log in” when he comes to your site, the personalization possibilities are nearly limitless.
  2. The persona above is based on a real customer segment. I named it after my friend Tommy Bailey, who actually did write the eBook
    Guide to Hiking the Appalachian Trail, featured in the image above.
  3. This Moz post is part of an inbound marketing campaign targeting eCommerce marketers, a segment Inflow identified while building out our own personas. Our hope, and the whole point of inbound marketing, is that it provides value to you.

Current state of the inbound marketing industry

Inbound has, for the the most part, been applied to businesses in which the website objective is to generate leads for a sales team to follow-up with and close the deal. An examination of various marketing automation platforms—a key component of scalable inbound marketing programs—highlights this issue.

Popular marketing automation systems

Most of the major marketing automation systems can be be used very effectively as the backbone of an inbound marketing program for eCommerce businesses. However, only one of them (Silverpop) has made significant efforts to court the eCommerce market with content and out-of-box features. The next closest thing is Hubspot, so let’s start with those two:

Silverpop – an IBMⓇ Company

silver-pop.jpeg

Unlike the other platforms below, right out of the box Silverpop allows marketers to tap into very specific behaviors, including the items purchased or left in the cart.

You can easily segment based on metrics like the Recency, Frequency and Monetary Value (RFM) of purchases:

silverpop triggered campaigns

You can automate personalized shopping cart abandonment recovery emails:

silverpop cart abandonment recovery

You can integrate with many leading brands offering complementary services, including: couponing, CRM, analytics, email deliverability enhancement, social and most major eCommerce platforms.

What you can’t do with Silverpop is blog, find pricing info on their website, get a free trial on their website or have a modern-looking user experience. Sounds like an IBMⓇ company, doesn’t it?

HubSpot

Out of all the marketing automation platforms on this list, HubSpot is the most capable of handling “inbound marketing” campaigns from start to finish. This should come as no surprise, given the phrase is credited to
Brian Halligan, HubSpot’s co-founder and CEO.

While they don’t specifically cater to eCommerce marketing needs with the same gusto they give to lead gen. marketing, HubSpot does have
an eCommerce landing page and a demo landing page for eCommerce leads, which suggests that their own personas include eCommerce marketers. Additionally, there is some good content on their blog written specifically for eCommerce.

HubSpot has allowed some key partners to develop plug-ins that integrate with leading eCommerce platforms. This approach works well with curation, and is not dissimilar to how Google handles Android or Apple handles their approved apps.

magento and hubspot

The
Magento Connector for HubSpot, which costs $80 per month, was developed by EYEMAGiNE, a creative design firm for eCommerce websites. A similar HubSpot-approved third-party integration is on the way for Bigcommerce.

Another eCommerce integration for Hubspot is a Shopify plug-in called
HubShoply, which was developed by Groove Commerce and costs $100 per month.

You can also use HubSpot’s native integration capabilities with
Zapier to sync data between HubSpot and most major eCommerce SaaS vendors, including the ones above, as well as WooCommerce, Shopify, PayPal, Infusionsoft and more. However, the same could be said of some of the other marketing automation platforms, and using these third-party solutions can sometimes feel like fitting a square peg into a round hole.

HubSpot can and does handle inbound marketing for eCommerce websites. All of the features are there, or easy enough to integrate. But let’s put some pressure on them to up their eCommerce game even more. The least they can do is put an eCommerce link in the footer:

hubspot menus

Despite the lack of clear navigation to their eCommerce content, HubSpot seems to be paying more attention to the needs of eCommerce businesses than the rest of the platforms below.

Marketo

Nothing about Marketo’s in-house marketing strategy suggests “Ecommerce Director Bob” might be one of their personas. The description for each of
their marketing automation packages (from Spark to Enterprise) mentions that it is “for B2B” websites.

marketo screenshot

Driving Sales could apply to a retail business so I clicked on the link. Nope. Clearly, this is for lead generation.

marketo marketing automation

Passing “purchase-ready leads” over to your “sales reps” is a good example of the type of language used throughout the site.

Make no mistake, Marketo is a top-notch marketing automation platform. Powerful and clean, it’s a shame they don’t launch a full-scale eCommerce version of their core product. In the meantime, there’s the
Magento Integration for Marketo Plug-in developed by an agency out of Australia called Hoosh Marketing.

magento marketo integration

I’ve never used this integration, but it’s part of Marketo’s
LaunchPoint directory, which I imagine is vetted, and Hoosh seems like a reputable agency.

Their
pricing page is blurred and gated, which is annoying, but perhaps they’ll come on here and tell everyone how much they charge.

marketo pricing page

As with all others except Silverpop, the Marketo navigation provides no easy paths to landing pages that would appeal to “Ecommerce Director Bob.”

Pardot

This option is a
SalesForce product, so—though I’ve never had the opportunity to use it—I can imagine Pardot is heavy on B2B/Sales and very light on B2C marketing for retail sites.

The hero image on their homepage says as much.

pardot tagline

pardot marketing automationAgain, no mention of eCommerce or retail, but clear navigation to lead gen and sales.

Eloqua / OMC

eloqua-logo.jpeg

Eloqua, now part of the Oracle Marketing Cloud (OMC), has a landing page
for the retail industry, on which they proclaim:

“Retail marketers know that the path to lifelong loyalty and increased revenue goes through building and growing deep client relationships.”

Since when did retail marketers start calling customers clients?

eloqua integration

The Integration tab on OMC’s “…Retail.html” page helpfully informs eCommerce marketers that their sales teams can continue using CRM systems like SalesForce and Microsoft Dynamics but doesn’t mention anything about eCommerce platforms and other SaaS solutions for eCommerce businesses.

Others

There are many other players in this arena. Though I haven’t used them yet, three I would love to try out are
SharpSpring, Hatchbuck and Act-On. But none of them appear to be any better suited to handle the concerns of eCommerce websites.

Where there’s a gap, there’s opportunity

The purpose of the section above wasn’t to highlight deficiencies in the tools themselves, but to illustrate a gap in who they are being marketed to and developed for.

So far, most of your eCommerce competitors probably aren’t using tools like these because they are not marketed to by the platforms, and don’t know how to apply the technology to online retail in a way that would justify the expense.

The thing is, a tool is just a tool

The
key concepts behind inbound marketing apply just as much to online retail as they do to lead generation.

In order to “do inbound marketing,” a marketing automation system isn’t even strictly necessary (in theory). They just help make the activities scalable for most businesses.

They also bring a lot of different marketing activities under one roof, which saves time and allows data to be moved and utilized between channels and systems. For example, what a customer is doing on social could influence the emails they receive, or content they see on your site. Here are some potential uses for most of the platforms above:

Automated marketing uses

  • Personalized abandoned cart emails
  • Post-purchase nurturing/reorder marketing
  • Welcome campaigns for the newsletter (other free offer) signups
  • Winback campaigns
  • Lead-nurturing email campaigns for cohorts and persona-based segments

Content marketing uses

  • Optimized, strategic blogging platforms, and frameworks
  • Landing pages for pre-transactional/educational offers or contests
  • Social media reporting, monitoring, and publishing
  • Personalization of content and user experience

Reporting uses

  • Revenue reporting (by segment or marketing action)
  • Attribution reporting (by campaign or content)

Assuming you don’t have the budget for a marketing automation system, but already have a good email marketing platform, you can still get started with inbound marketing. Eventually, however, you may want to graduate to a dedicated marketing automation solution to reap the full benefits.

Email marketing platforms

Most of the marketing automation systems claim to replace your email marketing platform, while many email marketing platforms claim to be marketing automation systems. Neither statement is completely accurate.

Marketing automation systems, especially those created specifically for the type of “inbound” campaigns described above, provide a powerful suite of tools all in one place. On the other hand, dedicated email platforms tend to offer “email marketing” features that are better, and more robust, than those offered by marketing automation systems. Some of them are also considerably cheaper—such as
MailChimp—but those are often light on even the email-specific features for eCommerce.

A different type of campaign

Email “blasts” in the form of B.O.G.O., $10 off or free shipping offers can still be very successful in generating incremental revenue boosts — especially for existing customers and seasonal campaigns.

The conversion rate on a 20% off coupon sent to existing customers, for instance, would likely pulverize the conversion rate of an email going out to middle-of-funnel contacts with a link to content (at least with how CR is currently being calculated by email platforms).

Inbound marketing campaigns can also offer quick wins, but they tend to focus mostly on non-customers after the first segmentation campaign (a campaign for the purpose of segmenting your list, such as an incentivised survey). This means lower initial conversion rates, but long-term success with the growth of new customers.

Here’s a good bet if works with your budget: Rely on a marketing automation system for inbound marketing to drive new customer acquisition from initial visit to first purchase, while using a good email marketing platform to run your “promotional email” campaigns to existing customers.

If you have to choose one or the other, I’d go with a robust marketing automation system.

Some of the most popular email platforms used by eCommerce businesses, with a focus on how they handle various Inbound Marketing activities, include:

Bronto

bronto.jpeg

This platform builds in features like abandoned cart recovery, advanced email list segmentation and automated email workflows that nurture contacts over time.

They also offer a host of eCommerce-related
features that you just don’t get with marketing automation systems like Hubspot and Marketo. This includes easy integration with a variety of eCommerce platforms like ATG, Demandware, Magento, Miva Merchant, Mozu and MarketLive, not to mention apps for coupons, product recommendations, social shopping and more. Integration with enterprise eCommerce platforms is one reason why Bronto is seen over and over again when browsing the Internet Retailer Top 500 reports.

On the other hand, Bronto—like the rest of these email platforms—doesn’t have many of the features that assist with content marketing outside of emails. As an “inbound” marketing automation system, it is incomplete because it focuses almost solely on one channel: email.

Vertical Response

verticalresponse.jpeg

Another juggernaut in eCommerce email marketing platforms, Vertical Response, has even fewer inbound-related features than Bronto, though it is a good email platform with a free version that includes up to 1,000 contacts and 4,000 emails per month (i.e. 4 emails to a full list of 1,000).

Oracle Marketing Cloud (OMC)

Responsys (the email platform), like Eloqua (the marketing automation system) was gobbled up by Oracle and is now part of their “Marketing Cloud.”

It has been my experience that when a big technology firm like IBM or Oracle buys a great product, it isn’t “great” for the users. Time will tell.

Listrak

listrak.jpeg

Out of the established email platforms for eCommerce, Listrak may do the best job at positioning themselves as a full inbound marketing platform.

Listrak’s value proposition is that they’re an “Omnichannel” solution. Everything is all in one “Single, Integrated Digital Marketing Platform for Retailers.” The homepage image promises solutions for Email, Mobile, Social, Web and In-Store channels.

I haven’t had the opportunity to work with Listrak yet, but would love to hear feedback in the comments on whether they could handle the kind of persona-based content marketing and automated email nurturing campaigns described in the example campaign above.

Key takeaways

Congratulations for making this far! Here are a few things I hope you’ll take away from this post:

  • There is a lot of opportunity right now for eCommerce sites to take advantage of marketing automation systems and robust email marketing platforms as the infrastructure to run comprehensive inbound marketing campaigns.
  • There is a lot of opportunity right now for marketing automation systems to develop content and build in eCommerce-specific features to lure eCommerce marketers.
  • Inbound marketing isn’t email marketing, although email is an important piece to inbound because it allows you to begin forming lasting relationships with potential customers much earlier in the buying cycle.
  • To see the full benefits of inbound marketing, you should focus on getting the right content to the right person at the right time in their shopping journey. This necessarily involves several different channels, including search, social and email. One of the many benefits of marketing automation systems is their ability to track your efforts here across marketing channels, devices and touch-points.

Tools, resources, and further reading

There is a lot of great content on the topic of Inbound marketing, some of which has greatly informed my own understanding and approach. Here are a few resources you may find useful as well.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 3 years ago from tracking.feedpress.it

The Incredible Shrinking SERP – 2015 Edition

Posted by Dr-Pete

In the beginning, there were 10 results, and it was good. Then, came expanded site-links and Google’s 
7-result SERP. Around the middle of 2014, we started to hear reports of SERPs with odd numbers of organic results – 9, 8, 6, 5, and even 4 page-1 results. At first, these were sporadic and hard to replicate, but they quietly expanded. This is a recent 4-result SERP for “autism speaks”:

By some counts, there are as many as 16 non-paid links on this page (not counting images), but by traditional SEO standards, there are only 4 true organic positions for which you can compete. So, what’s going on here? Is it just random, or is there a method to Google’s madness?

It’s all in the news

For a couple of months, I just assumed these strange result counts were some kind of glitch. Then I noticed an unusual pattern. Last October, Google rolled out the 
“In The News” Update. This update expanded news results to many new sources, but it also seemed to change the pattern of when news results appear. This is 28 days of data from MozCast’s Feature Graph (10K queries):

The presence of News results seemed to be cyclical, dipping early in the week and peaking later in the week. I don’t follow News results closely, so it was just a curiosity at first, until I saw another bit of data. This is the average page-1 result count for that same period:

While the scale of the change was much smaller (please note that both graphs have a restricted Y-axis to make the effect more visible), the opposing shapes of the curves seemed like more than a coincidence. As News results increased, the average page-1 organic result count decreased.

It’s a vertical, vertical world

Spot-checking various SERPs, I was able to confirm this effect. If page 1 had a News box, then the organic result count would be decreased by one (to either 9 results or 6, depending on the starting point). Here’s a sample SERP (I’ve removed snippets to simplify the image) for “samsung galaxy tab”:

This is a basic 10-result SERP, but when a News box comes into play, we’re only left with 9 organic results. This raised the question – were other verticals having a similar impact? Digging deeper, I found that, in addition to News results, Image results and In-depth Articles also occupied one organic position. Remember the example at the top of the post? It’s a brand query, resulting in a 7-result SERP, but it also has News results, Image results, and In-depth Articles. If we do the math: 7 – 1 – 1 – 1 = 4 results. It’s not random at all.

In the interest of being more methodical, what if we looked at the average page-1 organic result across every combination of verticals in our data set? We’ll stick with a starting point of 10 results, to keep the data clean. Here’s a table with the average counts by vertical combination:

I’ve taken the average out to two decimal places just to be more transparent, but what we’re seeing here is little more than a tiny bit of measurement error. Generally speaking, each instance of a vertical result type (as a whole, not individual links within these verticals) costs a 10-result SERP one organic ranking position. It’s worth nothing that SERPs with all 3 verticals are pretty rare, but when they occur, each of those 3 verticals costs one position and one opportunity for you to rank on page 1.

It’s always something

So, do the same rules apply to 7-result SERPs? Well, Google isn’t a big fan of making my life easy, so it turns out this gets a bit more complicated. When 7-result SERPs originally launched, our data showed that they almost always came with expanded sitelinks in the #1 organic position. By “expanded sitelinks”, I mean something like the following:

Sitelinks usually appear for queries that either have a strong brand connotation or at least a dominant interpretation. While we typically use 6-packs of expanded sitelinks as an example, actual counts can vary from 1 to 6. Originally, the presence of any sitelinks yielded a 7-result SERP. Now, it’s gotten a bit more complicated, as shown by the table below:

Since each row of sitelinks can contain up to 2 links, the general logic seems to be that 1 row of sitelinks equates to 1 additional organic result. If you have 3 rows of sitelinks, then Google will remove 3 organic results from page 1.

Google’s logic here seems to revolve around the actual display of information and length of the page. As they add some elements, they’re going to subtract others. Since the physical display length of of most elements can vary quite a bit, the rules right now are pretty simplistic, but the core logic seems to be based on constraining the total number of results displayed on page 1.

It’s time to rethink organic

All of this raises a difficult question – what is an organic result? As SEOs, we typically don’t think of vertical results as “organic” by our fairly narrow definition, but they’re much more organic than paid results or even Knowledge Graph. What’s more, Google is starting to blur the lines with verticals.

For example, in the past couple of weeks, Google has redesigned the look of In-depth Articles twice. You might think “So what? It’s just a design change,” but take a closer look. At the end of March, Googled removed the “In-depth articles” header. Here’s an example of the new design (for the query “jobs”):

While the thumbnail images and horizontal dividers still set these results apart somewhat, Google’s intent seems to be to make them appear more organic. Keep in mind, too, that other, organic results use thumbnails as well (including videos and recipes).

Then, just a couple of weeks later (our systems detected this on the morning of April 8th), Google went much farther, removing the thumbnails and even the byline. Here’s part of a screenshot for “Putin”:

Can you spot the true organic results here? They’re the first two – the rest of this screenshot is In-depth Articles. The only real clue, beside the count and source-code markers, is the horizontal divider on either end of the 3-pack. On mobile, even the dividers are gone, as every result is treated like a “card” (see below).

As an SEO, I’m still inclined to call these results “vertical” for two reasons: (1) historical precedent, and (2) these results play by different ranking rules. I think reason #2 is the more important one – In-depth Articles are currently dominated by a core set of big publishers, and the algorithm differs quite a bit from regular, organic results.

It’s only the beginning…

You wanna get really crazy? Let’s look at an entire SERP for “polar” on an Android device (Moto G). This result also includes In-depth Articles (warning: scrolling ahead):

Let’s do the math. For starters, it’s a branded result with expanded sitelinks, so we should have a 7-result page. Remember that those last 3 results are In-depth Articles, so we’ll subtract 1, leaving us with what should be 6 results. See the “app pack” in the middle? That’s an Android-specific vertical, and instead of counting the pack as just 1 result, Google is counting each link as a result. So, we’re only left with 3 traditional organic results on this SERP, despite it being packed with information.

I strongly suspect this trend will continue, and it will probably expand. The definition of “organic” is blurring, and I think that all of these vertical results represent SEO opportunities that can’t be ignored. If we’re stuck in the mindset of only one “true” organic, then our opportunities are going to keep shrinking every day.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 3 years ago from tracking.feedpress.it

Spam Score: Moz’s New Metric to Measure Penalization Risk

Posted by randfish

Today, I’m very excited to announce that Moz’s Spam Score, an R&D project we’ve worked on for nearly a year, is finally going live. In this post, you can learn more about how we’re calculating spam score, what it means, and how you can potentially use it in your SEO work.

How does Spam Score work?

Over the last year, our data science team, led by 
Dr. Matt Peters, examined a great number of potential factors that predicted that a site might be penalized or banned by Google. We found strong correlations with 17 unique factors we call “spam flags,” and turned them into a score.

Almost every subdomain in 
Mozscape (our web index) now has a Spam Score attached to it, and this score is viewable inside Open Site Explorer (and soon, the MozBar and other tools). The score is simple; it just records the quantity of spam flags the subdomain triggers. Our correlations showed that no particular flag was more likely than others to mean a domain was penalized/banned in Google, but firing many flags had a very strong correlation (you can see the math below).

Spam Score currently operates only on the subdomain level—we don’t have it for pages or root domains. It’s been my experience and the experience of many other SEOs in the field that a great deal of link spam is tied to the subdomain-level. There are plenty of exceptions—manipulative links can and do live on plenty of high-quality sites—but as we’ve tested, we found that subdomain-level Spam Score was the best solution we could create at web scale. It does a solid job with the most obvious, nastiest spam, and a decent job highlighting risk in other areas, too.

How to access Spam Score

Right now, you can find Spam Score inside 
Open Site Explorer, both in the top metrics (just below domain/page authority) and in its own tab labeled “Spam Analysis.” Spam Score is only available for Pro subscribers right now, though in the future, we may make the score in the metrics section available to everyone (if you’re not a subscriber, you can check it out with a free trial). 

The current Spam Analysis page includes a list of subdomains or pages linking to your site. You can toggle the target to look at all links to a given subdomain on your site, given pages, or the entire root domain. You can further toggle source tier to look at the Spam Score for incoming linking pages or subdomains (but in the case of pages, we’re still showing the Spam Score for the subdomain on which that page is hosted).

You can click on any Spam Score row and see the details about which flags were triggered. We’ll bring you to a page like this:

Back on the original Spam Analysis page, at the very bottom of the rows, you’ll find an option to export a disavow file, which is compatible with Google Webmaster Tools. You can choose to filter the file to contain only those sites with a given spam flag count or higher:

Disavow exports usually take less than 3 hours to finish. We can send you an email when it’s ready, too.

WARNING: Please do not export this file and simply upload it to Google! You can really, really hurt your site’s ranking and there may be no way to recover. Instead, carefully sort through the links therein and make sure you really do want to disavow what’s in there. You can easily remove/edit the file to take out links you feel are not spam. When Moz’s Cyrus Shepard disavowed every link to his own site, it took more than a year for his rankings to return!

We’ve actually made the file not-wholly-ready for upload to Google in order to be sure folks aren’t too cavalier with this particular step. You’ll need to open it up and make some edits (specifically to lines at the top of the file) in order to ready it for Webmaster Tools

In the near future, we hope to have Spam Score in the Mozbar as well, which might look like this: 

Sweet, right? 🙂

Potential use cases for Spam Analysis

This list probably isn’t exhaustive, but these are a few of the ways we’ve been playing around with the data:

  1. Checking for spammy links to your own site: Almost every site has at least a few bad links pointing to it, but it’s been hard to know how much or how many potentially harmful links you might have until now. Run a quick spam analysis and see if there’s enough there to cause concern.
  2. Evaluating potential links: This is a big one where we think Spam Score can be helpful. It’s not going to catch every potentially bad link, and you should certainly still use your brain for evaluation too, but as you’re scanning a list of link opportunities or surfing to various sites, having the ability to see if they fire a lot of flags is a great warning sign.
  3. Link cleanup: Link cleanup projects can be messy, involved, precarious, and massively tedious. Spam Score might not catch everything, but sorting links by it can be hugely helpful in identifying potentially nasty stuff, and filtering out the more probably clean links.
  4. Disavow Files: Again, because Spam Score won’t perfectly catch everything, you will likely need to do some additional work here (especially if the site you’re working on has done some link buying on more generally trustworthy domains), but it can save you a heap of time evaluating and listing the worst and most obvious junk.

Over time, we’re also excited about using Spam Score to help improve the PA and DA calculations (it’s not currently in there), as well as adding it to other tools and data sources. We’d love your feedback and insight about where you’d most want to see Spam Score get involved.

Details about Spam Score’s calculation

This section comes courtesy of Moz’s head of data science, Dr. Matt Peters, who created the metric and deserves (at least in my humble opinion) a big round of applause. – Rand

Definition of “spam”

Before diving into the details of the individual spam flags and their calculation, it’s important to first describe our data gathering process and “spam” definition.

For our purposes, we followed Google’s definition of spam and gathered labels for a large number of sites as follows.

  • First, we randomly selected a large number of subdomains from the Mozscape index stratified by mozRank.
  • Then we crawled the subdomains and threw out any that didn’t return a “200 OK” (redirects, errors, etc).
  • Finally, we collected the top 10 de-personalized, geo-agnostic Google-US search results using the full subdomain name as the keyword and checked whether any of those results matched the original keyword. If they did not, we called the subdomain “spam,” otherwise we called it “ham.”

We performed the most recent data collection in November 2014 (after the Penguin 3.0 update) for about 500,000 subdomains.

Relationship between number of flags and spam

The overall Spam Score is currently an aggregate of 17 different “flags.” You can think of each flag a potential “warning sign” that signals that a site may be spammy. The overall likelihood of spam increases as a site accumulates more and more flags, so that the total number of flags is a strong predictor of spam. Accordingly, the flags are designed to be used together—no single flag, or even a few flags, is cause for concern (and indeed most sites will trigger at least a few flags).

The following table shows the relationship between the number of flags and percent of sites with those flags that we found Google had penalized or banned:

ABOVE: The overall probability of spam vs. the number of spam flags. Data collected in Nov. 2014 for approximately 500K subdomains. The table also highlights the three overall danger levels: low/green (< 10%) moderate/yellow (10-50%) and high/red (>50%)

The overall spam percent averaged across a large number of sites increases in lock step with the number of flags; however there are outliers in every category. For example, there are a small number of sites with very few flags that are tagged as spam by Google and conversely a small number of sites with many flags that are not spam.

Spam flag details

The individual spam flags capture a wide range of spam signals link profiles, anchor text, on page signals and properties of the domain name. At a high level the process to determine the spam flags for each subdomain is:

  • Collect link metrics from Mozscape (mozRank, mozTrust, number of linking domains, etc).
  • Collect anchor text metrics from Mozscape (top anchor text phrases sorted by number of links)
  • Collect the top five pages by Page Authority on the subdomain from Mozscape
  • Crawl the top five pages plus the home page and process to extract on page signals
  • Provide the output for Mozscape to include in the next index release cycle

Since the spam flags are incorporated into in the Mozscape index, fresh data is released with each new index. Right now, we crawl and process the spam flags for each subdomains every two – three months although this may change in the future.

Link flags

The following table lists the link and anchor text related flags with the the odds ratio for each flag. For each flag, we can compute two percents: the percent of sites with that flag that are penalized by Google and the percent of sites with that flag that were not penalized. The odds ratio is the ratio of these percents and gives the increase in likelihood that a site is spam if it has the flag. For example, the first row says that a site with this flag is 12.4 times more likely to be spam than one without the flag.

ABOVE: Description and odds ratio of link and anchor text related spam flags. In addition to a description, it lists the odds ratio for each flag which gives the overall increase in spam likelihood if the flag is present).

Working down the table, the flags are:

  • Low mozTrust to mozRank ratio: Sites with low mozTrust compared to mozRank are likely to be spam.
  • Large site with few links: Large sites with many pages tend to also have many links and large sites without a corresponding large number of links are likely to be spam.
  • Site link diversity is low: If a large percentage of links to a site are from a few domains it is likely to be spam.
  • Ratio of followed to nofollowed subdomains/domains (two separate flags): Sites with a large number of followed links relative to nofollowed are likely to be spam.
  • Small proportion of branded links (anchor text): Organically occurring links tend to contain a disproportionate amount of banded keywords. If a site does not have a lot of branded anchor text, it’s a signal the links are not organic.

On-page flags

Similar to the link flags, the following table lists the on page and domain name related flags:

ABOVE: Description and odds ratio of on page and domain name related spam flags. In addition to a description, it lists the odds ratio for each flag which gives the overall increase in spam likelihood if the flag is present).

  • Thin content: If a site has a relatively small ratio of content to navigation chrome it’s likely to be spam.
  • Site mark-up is abnormally small: Non-spam sites tend to invest in rich user experiences with CSS, Javascript and extensive mark-up. Accordingly, a large ratio of text to mark-up is a spam signal.
  • Large number of external links: A site with a large number of external links may look spammy.
  • Low number of internal links: Real sites tend to link heavily to themselves via internal navigation and a relative lack of internal links is a spam signal.
  • Anchor text-heavy page: Sites with a lot of anchor text are more likely to be spam then those with more content and less links.
  • External links in navigation: Spam sites may hide external links in the sidebar or footer.
  • No contact info: Real sites prominently display their social and other contact information.
  • Low number of pages found: A site with only one or a few pages is more likely to be spam than one with many pages.
  • TLD correlated with spam domains: Certain TLDs are more spammy than others (e.g. pw).
  • Domain name length: A long subdomain name like “bycheapviagra.freeshipping.onlinepharmacy.com” may indicate keyword stuffing.
  • Domain name contains numerals: domain names with numerals may be automatically generated and therefore spam.

If you’d like some more details on the technical aspects of the spam score, check out the 
video of Matt’s 2012 MozCon talk about Algorithmic Spam Detection or the slides (many of the details have evolved, but the overall ideas are the same):

We’d love your feedback

As with all metrics, Spam Score won’t be perfect. We’d love to hear your feedback and ideas for improving the score as well as what you’d like to see from it’s in-product application in the future. Feel free to leave comments on this post, or to email Matt (matt at moz dot com) and me (rand at moz dot com) privately with any suggestions.

Good luck cleaning up and preventing link spam!



Not a Pro Subscriber? No problem!



Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 3 years ago from tracking.feedpress.it

The Most Important Link Penalty Removal Tool: Your Mindset

Posted by Eric Enge

Let’s face it. Getting slapped by a manual link penalty, or by the Penguin algorithm, really stinks. Once this has happened to you, your business is in a world of hurt. Worse still is the fact that you can’t get clear information from Google on which of your links are the bad ones. In today’s post, I am going to focus on the number one reason why people fail to get out from under these types of problems, and how to improve your chances of success.

The mindset

Success begins, continues, and ends with the right mindset. A large percentage of people I see who go through a link cleanup process are not aggressive enough about cleaning up their links. They worry about preserving some of that hard-won link juice they obtained over the years.

You have to start by understanding what a link cleanup process looks like, and just how long it can take. Some of the people I have spoken with have gone through a process like this one:

link removal timeline

In this fictitious timeline example, we see someone who spends four months working on trying to recover, and at the end of it all, they have not been successful.
A lot of time and money have been spent, and they have nothing to show for it. Then, the people at Google get frustrated and send them a message that basically tells them they are not getting it. At this point, they have no idea when they will be able to recover. The result is that the complete process might end up taking six months or more.

In contrast, imagine someone who is far more aggressive in removing and disavowing links. They are so aggressive that 20 percent of the links they cut out are actually ones that Google has not currently judged as being bad. They also start on March 9, and by April 30, the penalty has been lifted on their site.

Now they can begin rebuilding their business, five or months sooner than the person who does not take as aggressive an approach. Yes, they cut out some links that Google was not currently penalizing, but this is a small price to pay for getting your penalty cleared five months sooner. In addition, using our mindset-based approach, the 20 percent of links we cut out were probably not links that were helping much anyway, and that Google might also take action on them in the future.

Now that you understand the approach, it’s time to make the commitment. You have to make the decision that you are going to do whatever it takes to get this done, and that getting it done means cutting hard and deep, because that’s what will get you through it the fastest. Once you’ve got your head on straight about what it will take and have summoned the courage to go through with it, then and only then, you’re ready to do the work. Now let’s look at what that work entails.

Obtaining link data

We use four sources of data for links:

  1. Google Webmaster Tools
  2. Open Site Explorer
  3. Majestic SEO
  4. ahrefs

You will want to pull in data from all four of these sources, get them into one list, and then dedupe them to create a master list. Focus only on followed links as well, as nofollowed links are not an issue. The overall process is shown here:

pulling a link set

One other simplification is also possible at this stage. Once you have obtained a list of the followed links, there is another thing you can do to dramatically simplify your life.
You don’t need to look at every single link.

You do need to look at a small sampling of links from every domain that links to you. Chances are that this is a significantly smaller quantity of links to look at than all links. If a domain has 12 links to you, and you look at three of them, and any of those are bad, you will need to disavow the entire domain anyway.

I take the time to emphasize this because I’ve seen people with more than 1 million inbound links from 10,000 linking domains. Evaluating 1 million individual links could take a lifetime. Looking at 10,000 domains is not small, but it’s 100 times smaller than 1 million. But here is where the mindset comes in.
Do examine every domain.

This may be a grinding and brutal process, but there is no shortcut available here. What you don’t look at will hurt you. The sooner you start on the entire list, the sooner you will get the job done.

How to evaluate links

Now that you have a list, you can get to work. This is a key part where having the right mindset is critical. The first part of the process is really quite simple. You need to eliminate each and every one of these types of links:

  1. Article directory links
  2. Links in forum comments, or their related profiles
  3. Links in blog comments, or their related profiles
  4. Links from countries where you don’t operate/sell your products
  5. Links from link sharing schemes such as Link Wheels
  6. Any links you know were paid for

Here is an example of a foreign language link that looks somewhat out of place:

foreign language link

For the most part, you should also remove any links you have from web directories. Sure, if you have a link from DMOZ, Business.com, or BestofTheWeb.com, and the most important one or two directories dedicated to your market space, you can probably keep those.

For a decade I have offered people a rule for these types of directories, which is “no more than seven links from directories.” Even the good ones carry little to no value, and the bad ones can definitely hurt you. So there is absolutely no win to be had running around getting links from a bunch of directories, and there is no win in trying to keep them during a link cleanup process.

Note that I am NOT talking about local business directories such as Yelp, CityPages, YellowPages, SuperPages, etc. Those are a different class of directory that you don’t need to worry about. But general purpose web directories are, generally speaking, a poison.

Rich anchor text

Rich anchor text has been the downfall of many a publisher. Here is one of my favorite examples ever of rich anchor text:

The author wanted the link to say “buy cars,” but was too lazy to fit the two words into the same sentence! Of course, you may have many guest posts that you have written that are not nearly as obvious as this one. One great way to deal with that is to take your list of links that you built and sort them by URL and look at the overall mix of anchor text. You know it’s a problem if it looks anything like this:

overly optimized anchor text

The problem with the distribution in the above image is that the percentage of links that are non “rich” in nature is way too small. In the real world, most people don’t conveniently link to you using one of your key money phrases. Some do, but it’s normally a small percentage.

Other types of bad links

There is no way for me to cover every type of bad link in this post, but here are other types of links, or link scenarios, to be concerned about:

  1. If a large percentage of your links are coming from over on the right rail of sites, or in the footers of sites
  2. If there are sites that give you a site-wide link, or a very large number of links from one domain
  3. Links that come from sites whose IP address is identical in the A block, B block, and C block (read more about what these are here)
  4. Links from crappy sites

The definition of a crappy site may seem subjective, but if a site has not been updated in a while, or its information is of poor quality, or it just seems to have no one who cares about it, you can probably consider it a crappy site. Remember our discussion on mindset. Your objective is to be harsh in cleaning up your links.

In fact, the most important principle in evaluating links is this:
If you can argue that it’s a good link, it’s NOT. You don’t have to argue for good quality links. To put it another way, if they are not obviously good, then out they go!

Quick case study anecdote: I know of someone who really took a major knife to their backlinks. They removed and/or disavowed every link they had that was below a Moz Domain Authority of 70. They did not even try to justify or keep any links with lower DA than that. It worked like a champ. The penalty was lifted. If you are willing to try a hyper-aggressive approach like this one, you can avoid all the work evaluating links I just outlined above. Just get the Domain Authority data for all the links pointing to your site and bring out the hatchet.

No doubt that they ended up cutting out a large number of links that were perfectly fine, but their approach was way faster than doing the complete domain by domain analysis.

Requesting link removals

Why is it that we request link removals? Can’t we just build a
disavow file and submit that to Google? In my experience, for manual link penalties, the answer to this question is no, you can’t. (Note: if you have been hit by Penguin, and not a manual link penalty, you may not need to request link removals.)

Yes, disavowing a link is supposed to tell Google that you don’t want to receive any PageRank, or benefit, from it. However, there is a human element at play here.
Google likes to see that you put some effort into cleaning up the bad links that you have gotten that led to your penalty. The more bad links you have, the more important this becomes.

This does make the process a lot more expensive to get through, but if you approach this with the “whatever it takes” mindset, you dive into the requesting link removal process and go ahead and get it done.

I usually have people go through three rounds of requests asking people to remove links. This can be a very annoying process for those receiving your request, so you need to be aware of that. Don’t start your email with a line like “Your site is causing mine to be penalized …”, as that’s just plain offensive.

I’d be honest, and tell them “Hey, we’ve been hit by a penalty, and as part of our effort to recover we are trying to get many of the links we have gotten to our site removed. We don’t know which sites are causing the problem, but we’d appreciate your help …”

Note that some people will come back to you and ask for money to remove the link. Just ignore them, and put their domains in your disavow file.

Once you are done with the overall removal requests, and had whatever success you have had, take the rest of the domains and disavow them. There is a complete guide to
creating a disavow file here. The one incremental tip I would add is that you should nearly always disavow entire domains, not just the individual links you see.

This is important because even with the four tools we used to get information on as many links as we could, we still only have a subset of the total links. For example, the tools may have only seen one link from a domain, but in fact you have five. If you disavow only the one link, you still have four problem links, and that will torpedo your reconsideration request.

Disavowing the domain is a better-safe-than-sorry step you should take almost every time. As I illustrated at the beginning of this post, adding extra cleanup/reconsideration request loops is very expensive for your business.

The overall process

When all is said and done, the process looks something like this:

link removal process

If you run this process efficiently, and you don’t try to cut corners, you might be able to get out from your penalty in a single pass through the process. If so, congratulations!

What about tools?

There are some fairly well-known tools that are designed to help you with the link cleanup process. These include
Link Detox and Remove’em. In addition, at STC we have developed our own internal tool that we use with our clients.

These tools can be useful in flagging some of your links, but they are not comprehensive—they will help identify some really obvious offenders, but the great majority of links you need to deal with and remove/disavow are not identified. Plan on investing substantial manual time and effort to do the heavy lifting of a comprehensive review of all your links. Remember the “mindset.”

Summary

As I write this post, I have this sense of being heartless because I outline an approach that is often grueling to execute. But consider it tough love. Recovering from link penalties is indeed brutal.
In my experience, the winners are the ones who come with meat cleaver in hand, don’t try to cut corners, and take on the full task from the very start, no matter how extensive an effort it may be.

Does this type of process succeed? You bet. Here is an example of a traffic chart from a successful recovery:

manual penalty recovery graph

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 3 years ago from tracking.feedpress.it