Has Google Gone Too Far with the Bias Toward Its Own Content?

Posted by ajfried

Since the beginning of SEO time, practitioners have been trying to crack the Google algorithm. Every once in a while, the industry gets a glimpse into how the search giant works and we have opportunity to deconstruct it. We don’t get many of these opportunities, but when we do—assuming we spot them in time—we try to take advantage of them so we can “fix the Internet.”

On Feb. 16, 2015, news started to circulate that NBC would start removing images and references of Brian Williams from its website.

This was it!

A golden opportunity.

This was our chance to learn more about the Knowledge Graph.

Expectation vs. reality

Often it’s difficult to predict what Google is truly going to do. We expect something to happen, but in reality it’s nothing like we imagined.

Expectation

What we expected to see was that Google would change the source of the image. Typically, if you hover over the image in the Knowledge Graph, it reveals the location of the image.

Keanu-Reeves-Image-Location.gif

This would mean that if the image disappeared from its original source, then the image displayed in the Knowledge Graph would likely change or even disappear entirely.

Reality (February 2015)

The only problem was, there was no official source (this changed, as you will soon see) and identifying where the image was coming from proved extremely challenging. In fact, when you clicked on the image, it took you to an image search result that didn’t even include the image.

Could it be? Had Google started its own database of owned or licensed images and was giving it priority over any other sources?

In order to find the source, we tried taking the image from the Knowledge Graph and “search by image” in images.google.com to find others like it. For the NBC Nightly News image, Google failed to even locate a match to the image it was actually using anywhere on the Internet. For other television programs, it was successful. Here is an example of what happened for Morning Joe:

Morning_Joe_image_search.png

So we found the potential source. In fact, we found three potential sources. Seemed kind of strange, but this seemed to be the discovery we were looking for.

This looks like Google is using someone else’s content and not referencing it. These images have a source, but Google is choosing not to show it.

Then Google pulled the ol’ switcheroo.

New reality (March 2015)

Now things changed and Google decided to put a source to their images. Unfortunately, I mistakenly assumed that hovering over an image showed the same thing as the file path at the bottom, but I was wrong. The URL you see when you hover over an image in the Knowledge Graph is actually nothing more than the title. The source is different.

Morning_Joe_Source.png

Luckily, I still had two screenshots I took when I first saw this saved on my desktop. Success. One screen capture was from NBC Nightly News, and the other from the news show Morning Joe (see above) showing that the source was changed.

NBC-nightly-news-crop.png

(NBC Nightly News screenshot.)

The source is a Google-owned property: gstatic.com. You can clearly see the difference in the source change. What started as a hypothesis in now a fact. Google is certainly creating a database of images.

If this is the direction Google is moving, then it is creating all kinds of potential risks for brands and individuals. The implications are a loss of control for any brand that is looking to optimize its Knowledge Graph results. As well, it seems this poses a conflict of interest to Google, whose mission is to organize the world’s information, not license and prioritize it.

How do we think Google is supposed to work?

Google is an information-retrieval system tasked with sourcing information from across the web and supplying the most relevant results to users’ searches. In recent months, the search giant has taken a more direct approach by answering questions and assumed questions in the Answer Box, some of which come from un-credited sources. Google has clearly demonstrated that it is building a knowledge base of facts that it uses as the basis for its Answer Boxes. When it sources information from that knowledge base, it doesn’t necessarily reference or credit any source.

However, I would argue there is a difference between an un-credited Answer Box and an un-credited image. An un-credited Answer Box provides a fact that is indisputable, part of the public domain, unlikely to change (e.g., what year was Abraham Lincoln shot? How long is the George Washington Bridge?) Answer Boxes that offer more than just a basic fact (or an opinion, instructions, etc.) always credit their sources.

There are four possibilities when it comes to Google referencing content:

  • Option 1: It credits the content because someone else owns the rights to it
  • Option 2: It doesn’t credit the content because it’s part of the public domain, as seen in some Answer Box results
  • Option 3: It doesn’t reference it because it owns or has licensed the content. If you search for “Chicken Pox” or other diseases, Google appears to be using images from licensed medical illustrators. The same goes for song lyrics, which Eric Enge discusses here: Google providing credit for content. This adds to the speculation that Google is giving preference to its own content by displaying it over everything else.
  • Option 4: It doesn’t credit the content, but neither does it necessarily own the rights to the content. This is a very gray area, and is where Google seemed to be back in February. If this were the case, it would imply that Google is “stealing” content—which I find hard to believe, but felt was necessary to include in this post for the sake of completeness.

Is this an isolated incident?

At Five Blocks, whenever we see these anomalies in search results, we try to compare the term in question against others like it. This is a categorization concept we use to bucket individuals or companies into similar groups. When we do this, we uncover some incredible trends that help us determine what a search result “should” look like for a given group. For example, when looking at searches for a group of people or companies in an industry, this grouping gives us a sense of how much social media presence the group has on average or how much media coverage it typically gets.

Upon further investigation of terms similar to NBC Nightly News (other news shows), we noticed the un-credited image scenario appeared to be a trend in February, but now all of the images are being hosted on gstatic.com. When we broadened the categories further to TV shows and movies, the trend persisted. Rather than show an image in the Knowledge Graph and from the actual source, Google tends to show an image and reference the source from Google’s own database of stored images.

And just to ensure this wasn’t a case of tunnel vision, we researched other categories, including sports teams, actors and video games, in addition to spot-checking other genres.

Unlike terms for specific TV shows and movies, terms in each of these other groups all link to the actual source in the Knowledge Graph.

Immediate implications

It’s easy to ignore this and say “Well, it’s Google. They are always doing something.” However, there are some serious implications to these actions:

  1. The TV shows/movies aren’t receiving their due credit because, from within the Knowledge Graph, there is no actual reference to the show’s official site
  2. The more Google moves toward licensing and then retrieving their own information, the more biased they become, preferring their own content over the equivalent—or possibly even superior—content from another source
  3. If feels wrong and misleading to get a Google Image Search result rather than an actual site because:
    • The search doesn’t include the original image
    • Considering how poor Image Search results are normally, it feels like a poor experience
  4. If Google is moving toward licensing as much content as possible, then it could make the Knowledge Graph infinitely more complicated when there is a “mistake” or something unflattering. How could one go about changing what Google shows about them?

Google is objectively becoming subjective

It is clear that Google is attempting to create databases of information, including lyrics stored in Google Play, photos, and, previously, facts in Freebase (which is now Wikidata and not owned by Google).

I am not normally one to point my finger and accuse Google of wrongdoing. But this really strikes me as an odd move, one bordering on a clear bias to direct users to stay within the search engine. The fact is, we trust Google with a heck of a lot of information with our searches. In return, I believe we should expect Google to return an array of relevant information for searchers to decide what they like best. The example cited above seems harmless, but what about determining which is the right religion? Or even who the prettiest girl in the world is?

Religion-and-beauty-queries.png

Questions such as these, which Google is returning credited answers for, could return results that are perceived as facts.

Should we next expect Google to decide who is objectively the best service provider (e.g., pizza chain, painter, or accountant), then feature them in an un-credited answer box? The direction Google is moving right now, it feels like we should be calling into question their objectivity.

But that’s only my (subjective) opinion.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 4 years ago from tracking.feedpress.it

Support 4.0: Using Snapchat for all of Moz’s Support

Posted by Nick_Sayers

Innovation. Mobile. Community. Social. All words that come to mind when I think of Snapchat. Well, now a new word is creeping in… a word so disruptive to the Snapchat ecosphere that I’m going to bold it, then repeat it.
Support. Yes, support.

april fools placeholder

Moz has always been a customer-centric company. We innovate, and you enjoy. Moz is ready to take it further than ever.
Support+Snapchat is going to change how you talk to us and learn about the Moz products. Now read the following emotionally driven marketing copy to get a better sense of our new (industry-changing) means of support

Move the needle on the go. Using Moz on the go with a desktop-based browser and have a question about Local Rankings? Just hold your phone up to your other screen and send us a snap of your issue. Make sure to shout loud enough. We love to hear you.

Why boil the ocean? This is easy. Sleek. And, dare we say, innovative. It’s like chat, but it completely disappears. You just need your phone and a crippling support issue.

A team of unicorns. We’ve “transitioned” the zebras and horses to unemployment. We now only have unicorns. They will be blowing you away while helping with your support needs. Get ready to puke rainbows, folks.

Game-changing privacy. NSA. FBI. CIA. NYPD. Google. Illuminati. They’re all watching. Feel secure that your in-depth support explanations will disappear soon after you receive them. You won’t have to worry about anyone knowing that you couldn’t find an export button without our help.

Don’t open the kimono. Keep it clean. Unicorns are sensitive. Think of Moz’s Snapchat as your sweet old grandmother’s mailbox. The one those old Scholastic books she ordered for you always arrived in. Don’t tell her you didn’t read them.

Now reach out. Feel the disruption in the
Support Force. Send a Snapchat to moz_help. And welcome to Support 4.0.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 4 years ago from tracking.feedpress.it

When Is a Blog the Right Form of Content Marketing?

Posted by Isla_McKetta

You’ve heard the wisdom: 

“Your business should have a blog.” 

“Blogging helps your SEO.” 

“Why aren’t you blogging yet?” 

According to the experts, a blog will solve all your Internet woes. Blogging will increase your traffic, expand your audience, improve your engagement, position you as an authority, and allow you to shape the message in your space

In fact, blogging is so hyped as a panacea, you’d think that simply adding a blog to your site would also help you find the perfect spouse, cure the common cold, and even turn lead into gold. 

While I won’t deny the power of a good blog on the right site (seriously, as a writer, I’m pro-blog in general) to do all of those good things and more, you should always question anything that’s touted as the right answer for everyone (and everything). So should you blog?

When a blog is NOT necessarily the right form of content marketing

Now that you’re asking whether all that time and energy you’re putting (or planning to put) into your blog is really the right investment, let’s look at a few examples of when blogging is a bad idea (or is simply unnecessary).

1. You own your market

Johnson & Johnson. Amazon. Target. Google. These companies have already captured the hearts and minds of so many consumers that their names are nearly synonymous with their products. Here’s why blogging would only offer each of them a marginal benefit.

Traffic

Does Johnson & Johnson really care about traffic to its site when you already have Band-Aids (and all their other name brand products) in your medicine cabinet? Sure, they produce infographics, but there’s no real blog, and you were going to buy their products anyway, right?

Audience reach

Ordering anything from books to pet-waste bags online? You didn’t need a blog to discover Amazon, it’s so ingrained in your Internet history that you probably went straight there and those products will be on your doorstep in two days or less.

Engagement

Target mastered engagement when Oprah and Tyra started referring to the store as Tarzhay and shoppers only got more loyal as they added designer labels at discount prices. It didn’t matter that most of their products weren’t even available on their website, let alone that they didn’t have a blog. Their site has gotten a lot better in the past decade, but they still don’t need a blog to get customers in the door.

Authority

And Google… Sure they have a blog, but Google is such an authority for search queries that most of the consumers of their search results have no interest in, or need for, the blog.
So if you have little or no competition or your business is (and you expect it to remain) the top-of-mind brand in your market, you can skip blogging.

2. You have a better way of getting customers into the top of your funnel

A blog is only one way to attract new customers. For example, I live less than a mile from the nearest grocery store, and I can get there and back with a spare stick of butter before my oven even warms up. If the next nearest store had the most amazing blog ever, I’m still not going to go there when I’m missing an ingredient. But if they send me a coupon in the mail, I might just try them out when it’s less of an emergency.

The point is that different types of businesses require different types of tactics to get customers to notice them. 

My mom, a small-town accountant who knows all of her clients by name, doesn’t blog. She’s much more likely to get recommended by a neighbor than to be found on the Internet. If paid search brings you $50k in conversions every month and your blog contributes to $10k, it’s easy (and fair) to prioritize paid search. If you find that readers of white papers are the hottest leads for your SaaS company, offering a 50:1 ROI over blog readers, write those white papers. And if your customers are sharing your deals across email and/or social at a rate that your blog has never seen, give them more of what they want.

None of that means you’ll never have to create a blog. Instead, a blog might be something to reassess when your rate of growth slows in any of those channels, but if you’ve crunched your numbers and a blog just doesn’t pan out for now, use the tactics your customers are already responding to.

3. The most interesting things about your business are strictly confidential (or highly complicated)

Sure the CIA has a blog, but with posts like “CIA Unveils Portrait of Former Director Leon E. Panetta” and “CIA Reaches Deep to Feed Local Families” it reads more like a failed humanizing effort than anything you’d actually want to subscribe to (or worse, read). If you’re in a business where you can’t talk about what you do, a blog might not be for you. 

For example, while a CPA who handles individual tax returns might have success blogging about tips to avoid a big tax bill at year end, a big four accounting firm that specializes in corporate audits might want to think twice about that blog. Do you really have someone on hand who has something new and interesting to say about Sarbanes Oxley and has the time to write? 

The difference is engagement. So if you’re in a hush-hush or highly technical field, think about what you can reasonably write about and whether anyone is going to want (or legally be able) to publicly comment on or share what you’re writing. 

Instead, you might want to take the example of Deloitte which thinks beyond the concept of your typical blog to create all kinds of interesting evergreen content. The result is a host of interesting case studies and podcasts that could have been last updated three years ago for all it matters. This puts content on your site, but it also allows you to carefully craft and vet that content before it goes live, without building any expectation associated with an editorial calendar.

4. You think “thought leadership” means rehashing the news

There is a big difference between curating information and regurgitating it. True life confession: As much as I hate the term “thought leader,” I used it many a time in my agency days as a way to encourage clients to find the best in themselves. But the truth is, most people don’t have the time, energy, or vision to really commit to becoming a thought leader. 

A blog can be a huge opportunity to showcase your company’s mastery and understanding of your industry. But if you can’t find someone to write blog posts that expand on (or rethink) the existing knowledge base, save your ink. 

Some people curate and compile information in order to create “top 10” type posts. That kind of content can be helpful for readers who don’t have time to source content on their own, but I wouldn’t suggest it as the core content strategy for a company’s blog. If that’s all you have time for, focus on social media instead.

5. Your site is all timely content

A blog can help you shape the message around your industry and your brand, but what if your brand is built entirely around messaging? The BBC doesn’t need a blog because any reader would expect what they’re reading to be timely content and to adhere to the BBC’s standard voice. If readers want to engage with the content by commenting on the articles, they can. 

If you can explain the value that blogs.foxnews.com adds to the Fox News site, you’ve got a keener eye for content strategy than I do. My guess, from the empty blog bubbles here, is that this is a failed (or abandoned) experiment and will soon disappear.

6. Your business is truly offline

There’s one final reason that blogging might not fit your business model, and that’s if you have chosen not to enter the digital realm. I had lunch with a high-end jeweler in India recently where he was debating whether to go online (he was worried that his designs might get stolen) or continue to do business in person the way his family had done for at least three generations. 

If you are successful at selling your products offline, especially if your product has as much variation as a gemstone, an argument can be made for staying offline entirely.

When you should be blogging

Now that we’ve looked at some times it’s okay not to have a blog, let’s take a quick, expanded look at five reasons you might want to blog as part of your content marketing strategy (just in case you thought you’d gotten off scot-free by almost fitting into one of the boxes above).

1. You want traffic to your website

Conventional wisdom goes that the more pages you build, the more chances you have to rank. Heck, the more (good) content you create on your blog, the more collateral you have to showcase on your social channels, in email, and anywhere else you want to.

2. You want to expand your audience

If the content you’re creating is truly awesome, people will share it and find it and love it. Some of those people will be potential customers who haven’t even heard of you before. Keep up the excellence and you might just keep them interested.

3. You want to connect with customers

That blog is a fantastic place to answer FAQs, play with new ideas, and show off the humanity of all those fantastic individuals you have working for you. All of those things help customers get to know you, plus they can engage with you directly via the comments. You might just find ideas for new campaigns and even new products just by creating that venue for conversation.

4. You have something to add to the discussion

Do you really have a fresh perspective on what’s going on in your industry? Help others out by sharing your interesting stories and thoughtful commentary. You’re building your authority and the authority of your company at the same time.

5. You’re ready to invest in your future

Content is a long game, so the payoffs from blogging may be farther down the road than you might hope. But if a blog is right for your company, you’re giving yourself the chance to start shaping the message about your industry and your company the day you publish your first post. Keep at it and you might find that you start attracting customers from amongst your followers.

The gist

Don’t blog just because someone told you to. A blog is a huge investment and sustaining that blog can take a lot of work. But there are a lot of good reasons to dig in and blog like you mean it. 

What’s your decision? Do you have a good reason that you’ve decided to abstain from blogging? Or have you decided that a blog is the right thing for your business? Help others carefully consider their investment in blogging by sharing your story in the comments.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 5 years ago from feedproxy.google.com

Link Echoes (a.k.a. Link Ghosts): Why Rankings Remain Even After Links Disappear – Whiteboard Friday

Posted by randfish

One of the more interesting phenomena illustrated by Rand’s IMEC Lab project is that of “link echoes,” sometimes referred to as “link ghosts.” The idea is that if we move a page up in rankings by pointing links to it, and then remove those links, the bump in rankings often remains.

In today’s Whiteboard Friday, Rand explains what’s going on.

One quick note: Rand mentions a bit.ly link in this video that isn’t quite accurate; here’s the correct one. =)

For reference, here’s a still of this week’s whiteboard!

Video Transcription

Howdy Moz fans and welcome to another edition of Whiteboard Friday. This week I’m going to talk a little bit about link echoes. This is the reverberation of the effect of a link across Google’s link graph and across the rankings, that has an impact even after a link has been removed. In the past, we have also referred to these as link ghosts, but I think link echoes is actually a much better name. I appreciate some folks pointing that out for me.

Let me show you exactly what I’m talking about. So, as you might know, I’ve been running a number of tests, and those tests have been happening through a project I call IMEC Lab. If you go to http://bit.ly/imeclab, you will find this project ongoing.

We’ve been performing a number of tests over the last six months. I started with a smaller group. The group has gotten bigger. So we’ve been able to test some really fascinating things. A number of those have been around tests related to links. I’m going to share one of those tests, because it helps really highlight what’s going on with link echoes.

So we had a page point ranking number 31 for a key phrase, a not very competitive keyword search phrase, and the only reason I’m not transparently sharing these, at least not yet, is because we prefer that Google didn’t know all of the websites and pages that we’re pointing links from. Otherwise, they could potentially mess with the test. We like to keep the test results as clean as possible, and so we’re not disclosing these for right now.

Another page, page B ranking number 11 for the same query. So page ranking for query A, that’s page A ranking number 31, page B ranking number 11. Of course, our first step . . . well, this was one of the steps in our test was we pointed 22 links from 22 different websites, all the same pages of those sites to both A and B. We were actually trying to test anchor text. So we pointed anchor text exact match links at A, non-match at B. We wanted to see which one would boost it up. Some of the links we put first, some of the links we put second. We tried to control a bunch of variables.

We ran tests like these many times. I think this particular one we repeated four or five different times. In this case, we saw A, the one that was ranking number 31, it moved up to position one. Just 22 links were able to move it, bam. Anchor text links able to move it up to position one. Anchor text links obviously still pretty darn powerful. We could see that in each of our tests.

B we pointed those same 22 links at, that moved up 6 positions. Remember it didn’t have the exact match anchor text, so it moved up to position five, still quite impressive.

Then we did something else. We took those links away. We removed all the links, and this is pretty natural. We want to run more tests. We’re going to use some of these same sites and pages, so we removed all the links, no longer exist. The next week, they’d all been indexed. What happened?

Well, gosh, page A, that was ranking number 31 and moved up to 1, even after all those pages that were linking to it had been indexed with no link there anymore by Google, didn’t move. It stayed in position number one. That’s pretty weird. Almost the same thing happened with result B. It moved down one position. It’s ranking number six.

Even weirder, this happened over four and a half months ago. We’re now in the middle end of July. This was in mid-April, early April. That’s a very long time, right? Google’s indexed these pages that we’re linking many times, never seen the links to them. As far as we can tell, there are no new links pointing to either of those pages. At least we haven’t seen them, and none of the link tools out there have seen them. So it’s possible, maybe some new links.

Here’s where it gets weird. This effect of these link tests, remaining in place long after the link had been removed, happened in every single link test we ran, of which I counted eight where I feel highly confident that there were no confounding variables, feeling really good that we followed a process kind of just like this. The links pointed, the ranking rose. The links disappeared, the ranking stayed high. Eight different consecutive tests every single time. In fact, there wasn’t one test where, when we removed the links, the rankings fell back to their original position. Some of them like this one fell a position or two. Almost everything that we moved from page two or three stayed on page one after we linked to it, even after removing the links.

This argues strongly in favor of a phenomenon that some SEOs have speculated about for a good amount of time. I believe one of them is Martin Panayotov — I might not be pronouncing his name correctly — and, of course, Moz contributor Michael King, iPullRank. Both of them had commented on a post years ago saying link ghosts, aka link echoes, are real. You guys should look into them. Sorry it took us so long to look into this, but this is fascinating.

Now, there could be a number of explanations behind this link echo phenomenon, the continuing reverberation of a link’s effect on a ranking. It could be that maybe that page ends up performing well in Google’s analysis of its user and usage data. It ranks well for this relatively unpopular query. It’s ranking number one. And you know what? Google’s finding that the click-throughs are still pretty high. There’s not a lot of pogo sticking back to the results. Maybe they’re going, “Hey, this page looks legit. Let’s leave it here,” even after the links disappear.

It could be that the site or page was bolstered by other factors, other ranking factors that we may not know about. It could be that every one of these eight times when we moved it up, maybe by moving it up through links we inadvertently did something else to it. Maybe that helped it rank higher for other pages, and those other pages generated links each of these times. That’s fairly unlikely when you repeat the test this many times, but not impossible.

Or it could be that Google actually has something in their algorithm around link echoes, where they say, “Hey, you know what? After a link has disappeared, that doesn’t necessarily mean we should take away the value of that link as a vote forever and ever.” In fact, maybe we should, for a long time, perhaps depending on how many links the page has or how uncompetitive the search results are, or something that they say, “You know what? Let’s leave some remnant, some echo, a ghost of that link’s value in the ranking equation for the site or page.” These things are all possible.

What’s fascinating about practice to me is that it means that, for a lot of us who worry tremendously about link reclamation, about losing links on sites or pages that may produce things freshly, but then remove them on blogs that don’t always stay consistent across time, that we may be getting more value than we think from a link that disappears in the future. Of course, learning more about how Google works, about their operations is just fascinating to me. Google says their mission is to organize the world’s information and to make it universally accessible and useful. Well, I think part of Moz’s mission and my mission is to organize information about how Google works and make it universally accessible and useful. That’s what I hope we’re doing with some of these tests, particularly around link ghosts.

So I’m looking forward to some great comments. I’m sure many of you are going to have things that you’ve observed as well. If you’d like to follow along with this and other tests, I’d suggest checking out . . . you can go to bit.ly/mozmadscience and see the full presentation from my MozCon talk, in which I talk about link ghosts and a number of other tests we’ve been performing. I’ll be sharing a few of those individually here on Whiteboard Friday as well. But link echoes is such a fascinating one, I thought we should bring that out right away.

Thanks everyone. Take care. We’ll see you again next week for another edition of Whiteboard Friday.

Video transcription by Speechpad.com

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 5 years ago from feedproxy.google.com

Experiment: We Removed a Major Website from Google Search, for Science!

Posted by Cyrus-Shepard

The folks at Groupon surprised us earlier this summer when they reported the
results of an experiment that showed that up to 60% of direct traffic is organic.

In order to accomplish this, Groupon de-indexed their site, effectively removing themselves from Google search results. That’s crazy talk!

Of course, we knew we had to try this ourselves.

We rolled up our sleeves and chose to de-index
Followerwonk, both for its consistent Google traffic and its good analytics setup—that way we could properly measure everything. We were also confident we could quickly bring the site back into Google’s results, which minimized the business risks.

(We discussed de-indexing our main site moz.com, but… no soup for you!)

We wanted to measure and test several things:

  1. How quickly will Google remove a site from its index?
  2. How much of our organic traffic is actually attributed as direct traffic?
  3. How quickly can you bring a site back into search results using the URL removal tool?

Here’s what happened.

How to completely remove a site from Google

The fastest, simplest, and most direct method to completely remove an entire site from Google search results is by using the
URL removal tool

We also understood, via statements form Google engineers, that using this method gave us the biggest chance of bringing the site back, with little risk. Other methods of de-indexing, such as using meta robots NOINDEX, might have taken weeks and caused recovery to take months.

CAUTION: Removing any URLs from a search index is potentially very dangerous, and should be taken very seriously. Do not try this at home; you will not pass go, and will not collect $200!

CAUTION: Removing any URLs from a search index is potentially very dangerous, and should be taken very seriously. Do not try this at home; you will not pass go, and will not collect $200!

After submitting the request, Followerwonk URLs started
disappearing from Google search results in 2-3 hours

The information needs to propagate across different data centers across the globe, so the effect can be delayed in areas. In fact, for the entire duration of the test, organic Google traffic continued to trickle in and never dropped to zero.

The effect on direct vs. organic traffic

In the Groupon experiment, they found that when they lost organic traffic, they
actually lost a bunch of direct traffic as well. The Groupon conclusion was that a large amount of their direct traffic was actually organic—up to 60% on “long URLs”.

At first glance, the overall amount of direct traffic to Followerwonk didn’t change significantly, even when organic traffic dropped.

In fact, we could find no discrepancy in direct traffic outside the expected range.

I ran this by our contacts at Groupon, who said this wasn’t totally unexpected. You see, in their experiment they saw the biggest drop in direct traffic on
long URLs, defined as a URL that is at least as long enough to be in a subfolder, like https://followerwonk.com/bio/?q=content+marketer.

For Followerwonk, the vast majority of traffic goes to the homepage and a handful of other URLs. This means we didn’t have a statistically significant sample size of long URLs to judge the effect. For the long URLs we were able to measure, the results were nebulous. 

Conclusion: While we can’t confirm the Groupon results with our outcome, we can’t discount them either.

It’s quite likely that a portion of your organic traffic is attributed as direct. This is because of different browsers, operating systems and user privacy settings can potentially block referral information from reaching your website.

Bringing your site back from death

After waiting 2 hours,
we deleted the request. Within a few hours all traffic returned to normal. Whew!

Does Google need to recrawl the pages?

If the time period is short enough, and you used the URL removal tool, apparently not.

In the case of Followerwonk, Google removed over
300,000 URLs from its search results, and made them all reappear in mere hours. This suggests that the domain wasn’t completely removed from Google’s index, but only “masked” from appearing for a short period of time.

What about longer periods of de-indexation?

In both the Groupon and Followerwonk experiments, the sites were only de-indexed for a short period of time, and bounced back quickly.

We wanted to find out what would happen if you de-indexed a site for a longer period, like
two and a half days?

I couldn’t convince the team to remove any of our sites from Google search results for a few days, so I choose a smaller personal site that I often subject to merciless SEO experiments.

In this case, I de-indexed the site and didn’t remove the request until three days later. Even with this longer period, all URLs returned within just
a few hours of cancelling the URL removal request.

In the chart below, we revoked the URL removal request on Friday the 25th. The next two days were Saturday and Sunday, both lower traffic days.

Test #2: De-index a personal site for 3 days

Likely, the URLs were still in Google’s index, so we didn’t have to wait for them to be recrawled. 

Here’s another shot of organic traffic before and after the second experiment.

For longer removal periods, a few weeks for example, I speculate Google might drop these semi-permanently from the index and re-inclusion would comprise a much longer time period.

What we learned

  1. While a portion of your organic traffic may be attributed as direct (due to browsers, privacy settings, etc) in our case the effect on direct traffic was negligible.
  2. If you accidentally de-index your site using Google Webmaster Tools, in most cases you can quickly bring it back to life by deleting the request.
  3. Reinclusion happens quickly even after we removed a site for over 2 days. Longer than this, the result is unknown, and you could have problems getting all the pages of your site indexed again.

Further reading

Moz community member Adina Toma wrote an excellent YouMoz post on the re-inclusion process using the same technique, with some excellent tips for other, more extreme situations.

Big thanks to
Peter Bray for volunteering Followerwonk for testing. You are a brave man!

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 5 years ago from feedproxy.google.com

What Happened after Google Pulled Author and Video Snippets: A Moz Case Study

Posted by Cyrus-Shepard

In the past 2 months Google made
big changes to its search results

Webmasters saw disappearing 
Google authorship photos, reduced video snippets, changes to local packs and in-depth articles, and more.

Here at Moz, we’ve closely monitored our own URLs to measure the effect of these changes on our actual traffic.
The results surprised us.

Authorship traffic—surprising results

In the early days of authorship, many webmasters worked hard to get their photo in Google search results. I confess, I doubt anyone worked harder at author snippets
than me

Search results soon became crowded with smiling faces staring back at us. Authors hired professional photographers. Publishers worked to correctly follow Google’s guidelines to set up authorship for thousands of authors.

The race for more clicks was on.

Then on June 28th,
Google cleared the page. No more author photos. 

To gauge the effect on traffic, we examined eight weeks’ worth of data from Google Analytics and Webmaster Tools, before and after the change. We then examined our top 15 authorship URLs (where author photos were known to show consistently) compared to non-authorship URLs. 

The results broke down like this:

Change in Google organic traffic to Moz

  • Total Site:  -1.76%
  • Top 15 Non-Authorship URLs:  -5.96%
  • Top 15 Authorship URLs:  -2.86%

Surprisingly,
authorship URLs performed as well as non-authorship URLs in terms of traffic. Even though Moz was highly optimized for authors, traffic didn’t significantly change.

On an individual level, things looked much different. We actually observed big changes in traffic with authorship URLs increasing or decreasing in traffic by as much as 45%. There is no clear pattern: Some went up, some went down—exactly like any URL would over an extended time.

Authorship photos don’t exist in a vacuum; each photo on the page competed for attention with all the other photos on the page.
Each search result is as unique as a fingerprint. What worked for one result didn’t work for another.

Consider what happens visually when multiple author photos exist in the same search result:

One hypothesis speculates that more photos has the effect of drawing eyes down the page. In the absence of rich snippets, search click-through rates might follow more closely studied models, which dictate that
results closer to the top earn more clicks.

In the absence of author photos, it’s likely click-through rate expectations have once again become more standardized.

Video snippets: a complex tale

Shortly after Google removed author photos, they took aim at video snippets as well. On July 17th,
MozCast reported a sharp decline in video thumbnails.

Most sites, Moz included, lost
100% of their video results. Other sites appeared to be “white-listed” as reported by former Mozzer Casey Henry at Wistia. 

A few of the sites Casey found where Google continues to show video thumbnails:

  • youtube.com
  • vimeo.com
  • vevo.com
  • ted.com
  • today.com
  • discovery.com

Aside from these “giants,” most webmasters, even very large publishers at the top of the industry, saw their video snippets vanish in search results.

How did this loss affect traffic for our URLs with embedded videos? Fortunately, here at Moz we have a large collection of ready-made video URLs we could easily study: our
Whiteboard Friday videos, which we produce every, well, Friday. 

To our surprise, most URLs actually saw more traffic.

On average, our Whiteboard Friday videos saw a
10% jump in organic traffic after losing video snippets.

A few other with video saw
dramatic increases:

The last example, the Learn SEO page, didn’t have an actual video on it, but a bug with Google caused them to display an older video thumbnail. (Several folks we’ve talked to speculate that Google removed video snippets simply to clean up their bugs in the system)

We witnessed a significant increase in traffic after losing video snippets. How did this happen? 

Did Google change the way they rank and show video pages?

It turns out that many of our URLs that contained videos also saw a significant change in the number of search
impressions at the exact same time.

According to Google, impressions for the majority of our video URLs shot up dramatically around July 14th.

Impressions for Whiteboard Friday URLs also rose 20% during this time. For Moz, most of the video URLs saw many more impressions, but for others, it appears rankings dropped.

While Moz saw video impressions rise,
other publishers saw the opposite effect.

Casey Henry, our friend at video hosting company
Wistia, reports seeing rankings drop for many video URLs that had thin or little content.

“…it’s only pages hosting video with thin content… the pages that only had video and a little bit of text went down.”


Casey Henry

For a broader perspective, we talked to
Marshall Simmonds, founder of Define Media Group, who monitors traffic to millions of daily video pageviews for large publishers. 

Marshall found that despite the fact that
most of the sites they monitor lost video snippets, they observed no visible change in either traffic or pageviews across hundreds of millions of visits.

Define Media Group also recently released its
2014 Mid-Year Digital Traffic Report which sheds fascinating light on current web traffic trends.

What does it all mean?

While we have anecdotal evidence of ranking and impression changes for video URLs on individual sites, on the grand scale across all Google search results these differences aren’t visible.

If you have video content, the evidence suggests it’s now worth more than ever to follow
video SEO best practices: (taken from video SEO expert Phil Nottingham)

  • Use a crawlable player (all the major video hosting platforms use these today)
  • Surround the video with supporting information (caption files and transcripts work great)
  • Include schema.org video markup

SEO finds a way

For the past several years web marketers competed for image and video snippets, and it’s with a sense of sadness that they’ve been taken away.

The smart strategy follows the data, which suggest that more traditional click-through rate optimization techniques and strategies could now be more effective. This means strong titles, meta descriptions, rich snippets (those that remain), brand building and traditional ranking signals.

What happened to your site when Google removed author photos and video snippets? Let us know in the comments below.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 5 years ago from feedproxy.google.com

The Month Google Shook the SERPs

Posted by Dr-Pete

As a group, we SEOs still tend to focus most of our attention on just one place – traditional, organic results. In the past two years, I’ve spent a lot of time studying these results and how they change over time. The more I experience the reality of SERPs in the wild, though, the more I’ve become interested in situations like this one (a search for “diabetes symptoms”)…

See the single blue link and half-snippet on the bottom-left? That’s the only thing about this above-the-fold page that most SEOs in 2014 would call “organic”. Of course, it’s easy to find fringe cases, but the deeper I dig into the feature landscape that surrounds and fundamentally alters SERPs, the more I find that the exceptions are inching gradually closer to the rule.

Monday, July 28th was my 44th birthday, and I think Google must have decided to celebrate by giving me extra work (hooray for job security?). In the month between June 28th and July 28th, there were four major shake-ups to the SERPs, all of them happening beyond traditional, organic results. This post is a recap of our data on each of those shake-ups.

Authorship photos disappear (June 28)

On June 25th, Google’s John Mueller made a surprise announcement via Google+:

We had seen 
authorship shake-ups in the past, but the largest recent drop had measured around 15%. It was clear that Google was rethinking the prevalence of author photos and their impact on perceived quality, but most of us assumed this would be a process of small tweaks. Given Google’s push toward Google+ and its inherent tie-in with authorship, not a single SEO I know had predicted a complete loss of authorship photos.

Yet, over the next few days, culminating on the morning of June 28th, a 
total loss of authorship photos is exactly what happened:

While some authorship photos still appeared in personalized results, the profile photos completely disappeared from general results, after previously being present on about 21% of the SERPs that MozCast tracks. It’s important to note that the concept of authorship remains, and author bylines are still being shown (we track that at about 24%, as of this writing), but the overall visual impact was dramatic for many SERPs.

In-depth gets deeper (July 2nd)

Most SEOs still don’t pay much attention to Google’s “In-depth Articles,” but they’ve been slowly gain SERP share. When we first started tracking them, they popped up on about 3.5% of the searches MozCast covers. This data seems to only get updated periodically, and the number had grown to roughly 6.0% by the end of June 2014. On the morning of July 2nd, I (and, seemingly, everyone else), missed a major change:

Overnight, the presence of in-depth articles jumped from 6.0% to 12.7%, more than doubling (a +112% increase, to be precise). Some examples of queries that gained in-depth articles include:

  • xbox 360
  • hotels
  • raspberry pi
  • samsung galaxy tab
  • job search
  • pilates
  • payday loans
  • apartments
  • car sales
  • web design

Here’s an example set of in-depth for a term SEOs know all too well, “payday loans”:

The motivation for this change is unclear, and it comes even as Google continues to test designs with pared down in-depth results (almost all of their tests seem to take up less space than the current design). Doubling this feature hardly indicates a lack of confidence, though, and many competitive terms are now showing in-depth results.

Video looks more like radio (July 16th)

Just a couple of weeks after the authorship drop, we saw a smaller but still significant shake-up in video results, with about 28% of results MozCast tracks losing video thumbnails:

As you can see, the presence of thumbnails does vary day-to-day, but the two plateaus, before and after June 16th, are clear here. At this point, the new number seems to be holding.

Since our data doesn’t connect the video thumbnails to specific results, it’s tough to say if this change indicates a removal of thumbnails or a drop in rankings for video results overall. Considering how smaller drops in authorship signaled a much larger change down the road, I think this shift deserves more attention. It could be that Google is generally questioning the value and prevalence of rich snippets, especially when quality concerns come into play.

I originally hypothesized that this might not be a true loss, but could be a sign that some video snippets were switching to the new “mega-video” format (or video answer box, if you prefer). This does not appear to be the case, as the larger video format is still fairly uncommon, and the numbers don’t match up.

For reference, here’s a mega-video format (for the query “bartender”):

Mega-videos are appearing on such seemingly generic queries as “partition”, “headlights”, and “california king bed”. If you have the budget and really want to dominate the SERPs, try writing a pop song.

Pigeons attack local results (July 24th)

By now, many of you have heard of 
Google’s “Pigeon” update. The Pigeon update hit local SERPs hard and seems to have dramatically changed how Google determines and uses a searcher’s location. Local search is more than an algorithmic layer, though – it’s also a feature set. When Pigeon hit, we saw a sharp decline in local “pack” results (the groups of 2-7 pinned local results):

We initially reported that pack results dropped more than 60% after the Pigeon update. We now are convinced that this was a mistake (indicated by the “?” zone) – essentially, Pigeon changed localization so much that it broke the method we were using. We’ve found a new method that seems to match manually setting your location, and the numbers for July 29-30 are, to the best of my knowledge, accurate.

According to these new numbers, local pack results have fallen 23.4% (in our data set) after the Pigeon update. This is the exact same number 
Darren Shaw of WhiteSpark found, using a completely different data set and methodology. The perfect match between those two numbers is probably a bit of luck, but they suggest that we’re at least on the right track. While I over-reported the initial drop, and I apologize for any confusion that may have caused, the corrected reality still shows a substantial change in pack results.

It’s important to note that this 23.4% drop is a net change – among queries, there were both losers and winners. Here are 10 searches that lost pack results (and have been manually verified):

  • jobs
  • cars for sale
  • apartments
  • cruises
  • train tickets
  • sofa
  • wheels
  • liposuction
  • social security card
  • motorcycle helmets

A couple of important notes – first, some searches that lost packs only lost packs in certain regions. Second, Pigeon is a very recent update and may still be rolling out or being tweaked. This is only the state of the data as we know it today.

Here are 10 searches that gained pack results (in our data set):

  • skechers
  • mortgage
  • apartments for rent
  • web designer
  • long john silvers
  • lamps
  • mystic
  • make a wish foundation
  • va hospital
  • internet service

The search for “mystic” is an interesting example – no matter what your location (if you’re in the US), Google is showing a pack result for Mystic, CT. This pattern seems to be popping up across the Pigeon update. For example, a search for “California Pizza Kitchen” automatically targets California, regardless of your location (h/t 
Tony Verre), and a search for “Buffalo Wild Wings” sends you to Buffalo, NY (h/t Andrew Mitschke).

Of course, local search is complex, and it seems like Google is trying to do a lot in one update. The simple fact that a search for “apartments” lost pack results in our data, while “apartments for rent” gained them, shows that the Pigeon update isn’t based on a few simplistic rules.

Some local SEOs have commented that Pigeon seemed to increase the number of smaller packs (2-3 results). Looking at the data for pack size before and after Pigeon, this is what we’re seeing:

Both before and after Pigeon, there are no 1-packs, and 4-, 5-, and 6-packs are relatively rare. After Pigeon, the distribution of 2-packs is similar, but there is a notable jump in 3-packs and a corresponding decrease in 7-packs. The total number of 3-packs actually increased after the Pigeon update. While our data set (once we restrict it to just searches with pack results) is fairly small, this data does seem to match the observations of local SEOs.

Sleep with one eye open

Ok, maybe that’s a bit melodramatic. All of the changes do go to show, though, that, if you’re laser-focused on ranking alone, you may be missing a lot. We as SEOs not only need to look beyond our own tunnel vision, we need to start paying more attention to post-ranking data, like CTR and search traffic. SERPs are getting richer and more dynamic, and Google can change the rules overnight.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 5 years ago from feedproxy.google.com