How to Use Server Log Analysis for Technical SEO

Posted by SamuelScott

It’s ten o’clock. Do you know where your logs are?

I’m introducing this guide with a pun on a common public-service announcement that has run on late-night TV news broadcasts in the United States because log analysis is something that is extremely newsworthy and important.

If your technical and on-page SEO is poor, then nothing else that you do will matter. Technical SEO is the key to helping search engines to crawl, parse, and index websites, and thereby rank them appropriately long before any marketing work begins.

The important thing to remember: Your log files contain the only data that is 100% accurate in terms of how search engines are crawling your website. By helping Google to do its job, you will set the stage for your future SEO work and make your job easier. Log analysis is one facet of technical SEO, and correcting the problems found in your logs will help to lead to higher rankings, more traffic, and more conversions and sales.

Here are just a few reasons why:

  • Too many response code errors may cause Google to reduce its crawling of your website and perhaps even your rankings.
  • You want to make sure that search engines are crawling everything, new and old, that you want to appear and rank in the SERPs (and nothing else).
  • It’s crucial to ensure that all URL redirections will pass along any incoming “link juice.”

However, log analysis is something that is unfortunately discussed all too rarely in SEO circles. So, here, I wanted to give the Moz community an introductory guide to log analytics that I hope will help. If you have any questions, feel free to ask in the comments!

What is a log file?

Computer servers, operating systems, network devices, and computer applications automatically generate something called a log entry whenever they perform an action. In a SEO and digital marketing context, one type of action is whenever a page is requested by a visiting bot or human.

Server log entries are specifically programmed to be output in the Common Log Format of the W3C consortium. Here is one example from Wikipedia with my accompanying explanations:

127.0.0.1 user-identifier frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326
  • 127.0.0.1 — The remote hostname. An IP address is shown, like in this example, whenever the DNS hostname is not available or DNSLookup is turned off.
  • user-identifier — The remote logname / RFC 1413 identity of the user. (It’s not that important.)
  • frank — The user ID of the person requesting the page. Based on what I see in my Moz profile, Moz’s log entries would probably show either “SamuelScott” or “392388” whenever I visit a page after having logged in.
  • [10/Oct/2000:13:55:36 -0700] — The date, time, and timezone of the action in question in strftime format.
  • GET /apache_pb.gif HTTP/1.0 — “GET” is one of the two commands (the other is “POST”) that can be performed. “GET” fetches a URL while “POST” is submitting something (such as a forum comment). The second part is the URL that is being accessed, and the last part is the version of HTTP that is being accessed.
  • 200 — The status code of the document that was returned.
  • 2326 — The size, in bytes, of the document that was returned.

Note: A hyphen is shown in a field when that information is unavailable.

Every single time that you — or the Googlebot — visit a page on a website, a line with this information is output, recorded, and stored by the server.

Log entries are generated continuously and anywhere from several to thousands can be created every second — depending on the level of a given server, network, or application’s activity. A collection of log entries is called a log file (or often in slang, “the log” or “the logs”), and it is displayed with the most-recent log entry at the bottom. Individual log files often contain a calendar day’s worth of log entries.

Accessing your log files

Different types of servers store and manage their log files differently. Here are the general guides to finding and managing log data on three of the most-popular types of servers:

What is log analysis?

Log analysis (or log analytics) is the process of going through log files to learn something from the data. Some common reasons include:

  • Development and quality assurance (QA) — Creating a program or application and checking for problematic bugs to make sure that it functions properly
  • Network troubleshooting — Responding to and fixing system errors in a network
  • Customer service — Determining what happened when a customer had a problem with a technical product
  • Security issues — Investigating incidents of hacking and other intrusions
  • Compliance matters — Gathering information in response to corporate or government policies
  • Technical SEO — This is my favorite! More on that in a bit.

Log analysis is rarely performed regularly. Usually, people go into log files only in response to something — a bug, a hack, a subpoena, an error, or a malfunction. It’s not something that anyone wants to do on an ongoing basis.

Why? This is a screenshot of ours of just a very small part of an original (unstructured) log file:

Ouch. If a website gets 10,000 visitors who each go to ten pages per day, then the server will create a log file every day that will consist of 100,000 log entries. No one has the time to go through all of that manually.

How to do log analysis

There are three general ways to make log analysis easier in SEO or any other context:

  • Do-it-yourself in Excel
  • Proprietary software such as Splunk or Sumo-logic
  • The ELK Stack open-source software

Tim Resnik’s Moz essay from a few years ago walks you through the process of exporting a batch of log files into Excel. This is a (relatively) quick and easy way to do simple log analysis, but the downside is that one will see only a snapshot in time and not any overall trends. To obtain the best data, it’s crucial to use either proprietary tools or the ELK Stack.

Splunk and Sumo-Logic are proprietary log analysis tools that are primarily used by enterprise companies. The ELK Stack is a free and open-source batch of three platforms (Elasticsearch, Logstash, and Kibana) that is owned by Elastic and used more often by smaller businesses. (Disclosure: We at Logz.io use the ELK Stack to monitor our own internal systems as well as for the basis of our own log management software.)

For those who are interested in using this process to do technical SEO analysis, monitor system or application performance, or for any other reason, our CEO, Tomer Levy, has written a guide to deploying the ELK Stack.

Technical SEO insights in log data

However you choose to access and understand your log data, there are many important technical SEO issues to address as needed. I’ve included screenshots of our technical SEO dashboard with our own website’s data to demonstrate what to examine in your logs.

Bot crawl volume

It’s important to know the number of requests made by Baidu, BingBot, GoogleBot, Yahoo, Yandex, and others over a given period time. If, for example, you want to get found in search in Russia but Yandex is not crawling your website, that is a problem. (You’d want to consult Yandex Webmaster and see this article on Search Engine Land.)

Response code errors

Moz has a great primer on the meanings of the different status codes. I have an alert system setup that tells me about 4XX and 5XX errors immediately because those are very significant.

Temporary redirects

Temporary 302 redirects do not pass along the “link juice” of external links from the old URL to the new one. Almost all of the time, they should be changed to permanent 301 redirects.

Crawl budget waste

Google assigns a crawl budget to each website based on numerous factors. If your crawl budget is, say, 100 pages per day (or the equivalent amount of data), then you want to be sure that all 100 are things that you want to appear in the SERPs. No matter what you write in your robots.txt file and meta-robots tags, you might still be wasting your crawl budget on advertising landing pages, internal scripts, and more. The logs will tell you — I’ve outlined two script-based examples in red above.

If you hit your crawl limit but still have new content that should be indexed to appear in search results, Google may abandon your site before finding it.

Duplicate URL crawling

The addition of URL parameters — typically used in tracking for marketing purposes — often results in search engines wasting crawl budgets by crawling different URLs with the same content. To learn how to address this issue, I recommend reading the resources on Google and Search Engine Land here, here, here, and here.

Crawl priority

Google might be ignoring (and not crawling or indexing) a crucial page or section of your website. The logs will reveal what URLs and/or directories are getting the most and least attention. If, for example, you have published an e-book that attempts to rank for targeted search queries but it sits in a directory that Google only visits once every six months, then you won’t get any organic search traffic from the e-book for up to six months.

If a part of your website is not being crawled very often — and it is updated often enough that it should be — then you might need to check your internal-linking structure and the crawl-priority settings in your XML sitemap.

Last crawl date

Have you uploaded something that you hope will be indexed quickly? The log files will tell you when Google has crawled it.

Crawl budget

One thing I personally like to check and see is Googlebot’s real-time activity on our site because the crawl budget that the search engine assigns to a website is a rough indicator — a very rough one — of how much it “likes” your site. Google ideally does not want to waste valuable crawling time on a bad website. Here, I had seen that Googlebot had made 154 requests of our new startup’s website over the prior twenty-four hours. Hopefully, that number will go up!

As I hope you can see, log analysis is critically important in technical SEO. It’s eleven o’clock — do you know where your logs are now?

Additional resources

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

[ccw-atrib-link]

Know What Your Audience Wants Before Investing in Content Creation and Marketing – Whiteboard Friday

Posted by randfish

Content marketing is an iterative process: We learn and improve by analyzing the success of the things we produce. That doesn’t mean, though, that we shouldn’t set ourselves up for that success in the first place, and the best way to do that is by knowing what our audiences want before we actually go through the effort to create it. In today’s Whiteboard Friday, Rand (along with his stick-figure friends Rainy Bill and Hailstorm Hal) explains how we can stack our own decks in our favor with that knowledge.

Know What Your Audience Wants – Whiteboard Friday_1

For reference, here’s a still of this week’s whiteboard!

Video transcription

Howdy, Moz fans, and welcome to another edition of Whiteboard Friday. It’s 2015. It’s going to be a year where, again, many, many marketers engage in a ton of content investments and content marketing for a wide variety of purposes from SEO to driving traffic to growing their email newsletters and lists to earning links and attention and growing their social channels. Unfortunately, there’s a content marketing problem that we see over and over and over again, and that is that folks are making investments in content without knowing whether their audience is going to know and love and appreciate what they’re doing beforehand.

That kind of sucks because it adds a lot of risk to a process that is already risk intensive. You’re going to put a lot of work into the content that you’re creating. Well, hopefully you are. If you’re not, I don’t know how well it’s going to do. All of that work can be for naught.

Let me show you two examples. Over here I have Rainy Bill from WhatTheWeather.com, and here’s Hailstorm Hal from KingOfClimate.com. We’ll start with Rainy Bill’s story.

So Rainy Bill, he’s thinking to himself, “You know, I want to invest in some content marketing for WhatTheWeather.com.” He has an idea. He’s like, “You know, maybe I could make a chart of the T-shirts that meteorologists wear by season. I’ll look at all the TV meteorologists, all the Internet meteorologists, and I’ll look at the T-shirts that they wear. They all wear T-shirts, and I’ll make a big chart of them.”

You might think this is a ridiculous idea. I have seen worse. But Rainy Bill is thinking to himself, “Well, if I do this, it’s kind of ego bait. I get all the meteorologists involved. I’ll feature all their T-shirts, and, of course, all of them will see it and they’ll all link to me, talk about me, share it on their social media channels, email their friends with it. Oh check it out. Put it on their Facebook.”

He makes it. He’s got this beautiful chart showing different kinds of T-shirts that meteorologists are wearing over the seasons, and Bill’s just as happy as a clam. He can’t believe how beautiful that is until he tries to launch and promote it. Then it’s just sadness. He’s just crying tears.

What happened here is that no one actually cared what Bill had to say. No one cared about T-shirt patterns that are worn by meteorologists, and Bill didn’t actually realize this until he had already made the investment and started trying to do the promotion.

This might be a slightly ridiculous example, but I can’t tell you how many times I’ve seen exactly this story play out by marketer after marketer of content investments. They put something together that they hope will achieve their goal of reaching a new audience, of getting promoted, but it falls flat mostly because they had the idea before they talked to anyone else. Before they realized whether anyone else was interested, they went and built it.

That’s actually kind of a terrible idea. Unless you have your finger on the pulse of an industry, a field so incredibly well that you don’t need that process, I’m going to say that is the 1% of the 1% who can do this without going out and first talking to their audience and understanding.

Hailstorm Hal, from KingOfClimate, instead of having a great idea for a piece of content, Hailstorm Hal is going to start with the idea from which all content marketing springs, which is, “I want to make something people will really want and something they’ll really love.” Okay. They want it, and they’re going to love it when they see it and when they get it.

So Hailstorm Hal is going to go out and say, “Well, what are the weather watchers talking about? People who are active in this community, in this industry, the people who do the sharing and the amplification, who influence what the rest of us see, what are they talking about?”

So he goes onto this weather forum and hears someone complaining, “The weather in Cincinnati is totally unpredictable.” The reply, “Yeah, but it’s way more predictable than Seattle is.” “Nuh-uh, you liar.” From this, eureka, Hailstorm Hal has a great idea. “Wait a minute. What if I were to actually go and take all of this online commentary and turn it into something useful where these two commenters could prove to each other who’s correct and people would know for certain how much . . .”

It’s not just helpful to them. This is helpful to a huge, broad swath of society. How accurate are your meteorologists, on average, city by city? I don’t actually know, but I would be fascinated to know whether when I go to San Diego — I was there for the holidays to see my wife’s family — maybe the weather reports in San Diego are much more or much less accurate than what I’m used to here at home in Seattle.

So Hal’s going to put together this great map that’s got an illustration of different regions of the United States, and you can see that in the Midwest actually weather is more predictable than it is on the coast or less predictable than it is on the coast. That’s awesome. That’s terrific. This is going to work far, far better than anything that Hal could have come up with on his own without first understanding the industry.

Now the process and tips that I’m going to recommend here are not exhaustive. There are a lot more things in this. But if you follow these five, at least, I think you’re going to do much better with your content investment.

First off, even before you do this process, get to know the industry, the niche, or the community that you’re operating in. If Hal didn’t know where to find weather watchers, he might just search weather forum, click on the first link in Google, and be at some place that doesn’t really have a very serious investment from the community of people he’s trying to reach. Without understanding all of the sites and pages, without understanding who are the big influencers in the community on social media, without understanding what are the popular websites, what gets a lot of interaction and engagement and doesn’t, that’s going to be really tough for him to figure out.

So that’s why I would say you need to go out and learn about your industry before you make something for it. Incidentally, this is why it’s really tough to do this as a consultant and why if you are paying consultants to go and do this, you’re going to actually be paying quite a bit of money for this research time. This is going to be dozens of hours of research to understand the niche before you can effectively create content for it. That’s something where it isn’t just an on demand kind of thing.

Then from there you want to use the discussion forums, Q&A sites, social media, and blog comments to find topics and discussions that inspire questions, curiosity, and need. Some of that is going to be very blatant. Some of it is going to be much more latent, and you’re going to be drawing from both of those. Your job is to have insight and empathy, and that’s what a great marketer should be able to do when they’re researching these communities.

Number three, you want to validate that if you created something, (a) it would be unique, no one else has made it before, and (b) others would actually share it. You can do this very directly by reaching out and talking to people.

So Hal can go and say, “Hey, who’s this commenter right here? Let’s have a quick conversation. Would you like this?” If the answer is, “Yeah, not only would I like that, I would help share that. I would spread that. I would love to know the answer to this question.” Or no reply, or “Sounds interesting, let me know when you get it up.” There’s going to be a different variation.

You can go and use Twitter, Google+, and email to reach out directly to these people. Most of the time, if you’re finding commentary on these forums and in these places, there will be a way to reach them. I also have two tools I’m going to recommend, both for email. One is Conspire and the other is VoilaNorbert. VoilaNorbert.com is an email finding tool. I think it’s the best one out there right now, and Conspire is a great tool for seeing who you’re connected to that’s connected to people you might want to reach. When you’re trying to reach someone, those can be very helpful.

Number four, it tends to be the case that visual and/or interactive content is going to perform a lot better than text. So if Hal’s list had simply been a list of data — here are all the major U.S. regions and here’s how predictable and unpredictable their weather is — well, that might work okay. But this map, this visual is probably going to sail around the weather world much faster, much better, be picked up by news sources, be written about, be embedded in social media graphics, all that kind of stuff, far better than a mere chart would be.

Number five, remember that as you’re doing the creation, you need to align the audience goals with your business goals. So if KingOfClimate’s goal is to get people signing up for a weather tracking service on an email list, well great, you should have this and then say, “We can send you variability reports. We can tell you if things are getting more or less accurate,” and have an email call to action to get people to sign up to the newsletter. But you want to tie those business goals together.

The one thing I’d be careful of and this is a mistake that many, many folks who invest in content marketing make is that a lot of those benefits are going to be indirect and long term, meaning if the goal is that KingOfClimate.com is trying to sell professional meteorologists on a software subscription service, well, you know what? You’re probably not going to sell a whole lot with this. But you are going to get a lot more professional meteorologists who remember the name, KingOfClimate, and that brand memory is going to influence future purchase decisions, likely nudging conversation rates up a little bit.

It’s probably going to help with links. Links will lead to rankings. Rankings will lead to being higher up in search engines when professional meteorologists search for precisely, “I’m looking for weather tracking software or weather notification software.” So these kings of things are long term and indirect. You have to make sure you’re tying together all of the benefits of content marketing with your business goals that you might achieve.

I hope to see some phenomenal content here in 2015. I’m sure you guys are already working on some great stuff. Applying this can mean that you don’t have to be psychic. You just have to put in a little bit of elbow grease, and you can make things that will perform far better for your customers, for your community, and for your business.

All right, everyone. Look forward to the discussion, and we will see you again next week for another edition of Whiteboard Friday. Take care.

Video transcription by Speechpad.com

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

[ccw-atrib-link]

International SEO Study: How Searchers Perceive Country Code Top-Level Domains

Posted by 5le

The decision to focus your site on an international audience is a big step and one fraught with complexities. There are, of course, issues to deal with around language and user experience, but in addition there are some big technical choices to make including what domains to use.

Any authoritative
international SEO guide will elaborate on the differences between the options of subdirectory, subdomain, and country-code top level domain (CCTLD). One of the most common suggestions is for a site to opt to use a ccTLD (e.g. domain.co.uk) as the domain extension. The reasoning behind this is the theory that the ccTLD extension will “hint” to search engines and users exactly who your target audience should be versus the other, less explicit options. For example, a search engine and human user would know, even without clicking into a site, that a site that ends with .co.uk is targeting a user looking for UK content. 

We have solid data from
Google that a ccTLD does indicate country targeting; however, when it comes to users there is only an assumption that users even notice and make choices based on the ccTLD. However, this is a fairly broad assumption that doesn’t address whether a ccTLD is more important than a brand name in the domain or the quality of a website’s content. To test this theory, we ran a survey to discover what users really thought.

User knowledge of TLDs

Even before trying to understand how users related to ccTLDs it is essential to validate the assumption that users even know that general TLDs exist. To establish this fact, we asked respondents to pick which TLD might be the one in use by a non-profit. Close to
100% of respondents correctly identified a TLD ending with .org as the one most likely to be used by a non-profit. Interestingly, only 4% of people in the US stated that they were unsure of the correct TLD compared to 13% of Australians. Predictably, nearly all marketers (98%) chose the .org answer.

Another popular TLD is the .edu in use by educational assumptions, and we wanted to understand if users thought that content coming from a .edu domain might be more trustworthy. We asked users if they received an unsolicited email about water quality in their town whether they would place more trust in a sender’s email address that ended with .edu or .com.
89% of respondents in the US chose the .edu as more trustworthy, while only 79% said the same in Australia. Quite interestingly, the marketer responses (from the survey posted on Inbound.org were exactly the same as the Australians with 79% declaring the .edu to be more trustworthy.

.org cctld survey australia

If users can identify a .org as the correct TLD for a non-profit, and a .edu as a TLD that might be more trustworthy, it is likely that users are familiar with the existence of TLDs and how they might be used. The next question to answer is if users are aware of the connection between TLDs and locations.

Country relationship awareness

Next, we asked respondents to identify the location of a local business using a .ca TLD extension. The majority of respondents across all three surveys correctly chose Canada; and nearly all marketers (92%) got this correct. Oddly, more Australians (67%) correctly identified Canada than Americans (62%). We would have thought Americans should have been more familiar with the TLD of a neighboring country. Additionally, more Americans (23%) fell for the trick answer of California than Australians (15%). Regardless, we were able to conclude that most Internet users are aware of TLDs and that they are tied to a specific country.

canada cctld survey

To really gauge how much users know about TLDs and countries, we asked users to pick the right domain extension for a website in another country. In the US survey, we asked users to pick the correct TLD for an Australian company, and in the Australian survey we used a British company. In each of the questions we gave one correct answer possibility, one almost correct, and two entire wrong choices.For example, we gave .co.uk and .uk as answer choices to Australians.

In both the US and Australia, the majority of respondents chose the correct TLD, although Americans seem to have been confused by whether Australia’s TLD was .AU (35%) or .com.AU (24%).

There is a common practice of using country-code domain extensions as a vanity URL for content that is not geotargeted. For example, .ly is the domain extension for Libya, but it is frequently used on domains that have a word that ends with “ly.” Additionally, .me is the domain extension for Montenegro; however, the TLD is used for many purposes other than Montenegro content.

We wanted to understand if users noticed this type of TLD usage or if they thought the content might still be related to another country. We asked respondents what might be on a website that ended with .TV which is the TLD for the island nation of Tuvalu and is also a popular TLD for TV show websites. 51% of US respondents thought it might be a TV show and 42% chose the “it could be anything” answer. In Australia, 43% thought the site would be a TV show, and 44% said “it could be anything”.

tuvalu cctld survey

One of the answer options was that it could be a website in Tuvalu and interestingly twice as many Australian (9%) chose this option vs US respondents (4.5%). This question was one of the areas where marketers’ answers were very different from those in the US and Australia. 77% of marketers chose the TV show option and only 19% said it could be anything.

Based on the these three results, it is apparent that
users recognize TLDs, know that they are from other countries, and appear to make some judgments around the content based on the TLD.

Decision making using TLDs

Since users know that TLDs are an important part of a URL that is tied to a country of origin, it is important to understand how the TLD factors into their decision-making processes about whether or not they visit certain websites.

We asked users whether they thought medical content on a foreign TLD would be as reliable as similar content found on their local TLD. In the US, only 24% thought the content on the non-local TLD (.co.uk) was less reliable than content on a .com. In Australia, the results were nearly identical to what we saw in the US with only 28% answering that the non-local TLD (.co.uk) was less reliable than the content on a .com.au. Even 24% of marketers answered that the content was less reliable. The remaining respondents chose either that the content equally reliable or they just didn’t know. Based on these results, the TLD (at least as long as it was a reputable one)
does not seem to impact user trust.

UK cctld survey

Digging into the idea of trust and TLD a bit further, we asked the same reliability question about results on Google.com vs Google.de. In the US, 56% of respondents said that the results on Google.de are equally reliable to those on Google.com, and in Australia, 51% said the same thing when compared to Google.com.au. In the marketer survey, 66% of respondents said the results were equally reliable. The fact that the majority of respondents stated that results are equally reliable should mean that users are more focused on the brand portion of a domain rather than its country extension.

CcTLD’s impact on ecommerce

Making the decision to use a ccTLD on a website can be costly, so it is important to justify this cost with an actual revenue benefit. Therefore the real test of TLD choice is how it impacts revenue. This type of answer is of course hard to gauge in a survey where customers are not actually buying products, but we did want to try to see if there might be a way to measure purchasing decisions.

To achieve this result, we compared two different online retailers and asked respondents to choose the establishment that they thought would have the most reliable express shipping. In the US survey, we compared Amazon.co.jp to BestBuy.com. In the Australian survey, we compared Bigw.com.au (a well known online retailer) to Target.com. (Interesting fact: there is a Target in Australia that is not affiliated with Target in the US and their website is target.com.au) The intent of the question was to see if users zeroed in on the recognizable brand name or the domain extension.

cctld trust survey

In the US, while 39% said that both websites would offer reliable shipping, 42% still said that Best Buy would be the better option. Australians may have been confused by the incorrect Target website, since 61% said both websites would have reliable shipping, but 34% chose Big W. Even marketers didn’t seem oblivious to domain names with only 34% choosing the equally reliable option, and 49% choosing Best Buy. The data in this question is a bit inconclusive, but we can definitively say that while a large portion of users are blind to domain names, however, when selling online it would be best to use a familiar domain extension.

cctld trust survey australia

New TLDs

Late last year, ICANN (the Internet governing body) announced that they would be releasing dozens of new
GTLDs, which opened up a new domain name land grab harkening back to the early days of the Internet. Many of these domain names can be quite expensive, and we wanted to discover whether they even mattered to users.

gtld survey

We asked users if, based solely on the domain name, they were more likely to trust an insurance quote from a website ending in .insurance.
62% of Americans, 53% of Australians, and 67% of marketers said they were unlikely to trust the quote based on the domain alone. Based on this result, if you’re looking to invest in a new TLD simply to drive more conversions, you should probably do more research first. 

A new gTLD is probably not a silver bullet.

Methodology

For this survey, I collaborated with
Sam Mallikarjunan at HubSpot and we decided that the two assumptions we absolutely needed to validate where 1) whether users even notice ccTLDs and 2) if so do they really prefer the TLD of their country. While we received 101 responses from a version of the survey targeted at marketers on an Inbound.org discussion, we primarily used SurveyMonkey Audience, which allowed us to get answers from a statistically significant random selection of people in both the United States and Australia.

We created two nearly identical surveys with one targeted to a US-only audience and the other targeted to an Australian-only audience. A proper sample set is essential when conducting any survey that attempts to draw conclusions about people’s general behavior and preferences. And in this case, the minimum number of respondents we needed in order to capture a representative example was 350 for the U.S. and 300 for Australia.

Additionally, in order for a sample to be valid, the respondents have to be chosen completely at random. SurveyMonkey Audience recruits its 4-million+ members from SurveyMonkey’s 40 million annual unique visitors, and members are not paid for their participation. Instead, they are rewarded for taking surveys with charitable donations, made on their behalf by SurveyMonkey.

When tested against much larger research projects, Audience data has been exactly in line with larger sample sizes. For example, an Audience survey with just 400 respondents about a new Lay’s potato chip flavor had the same results as a wider contest that had 3 million participants.

SurveyMonkey’s survey research team was also able to use SurveyMonkey Audience to accurately predict election results in both 2012 and 2013. With a US sample size of 458 respondents and an Australian one of 312 all drawn at random, our ccTLD user preferences should reliably mirror the actual reality.

Summary

There will be many reasons that you may or may not want to use ccTLDs for your website, and a survey alone can never answer whether a ccTLD is the right strategy for any particular site. If you are thinking about making any big decisions about TLDs on your site, you should absolutely conduct some testing or surveying of your own before relying on just the recommendations of those who advise a TLD as the best strategy or the others that tell you it doesn’t matter at all.

Launching a PPC campaign with a landing page on a ccTLD and measuring CTRs against a control is far cheaper than replicating your entire site on a new TLD.

Based on our survey results, here’s what you should keep in mind when it comes to whether or not investing your time and money in a ccTLD is worth it:

  1. Users are absolutely aware of the TLDs and how they might relate to the contents of a website
  2. Users are aware of the connection between TLDs and countries
  3. Users do make decisions about websites based on the TLD; however there are no absolutes. Brand and content absolutely matter.

As to whether a ccTLD will work for you on your own site, give it a try and report back!

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

[ccw-atrib-link]

Location is Everything: Local Rankings in Moz Analytics

Posted by MatthewBrown

Today we are thrilled to launch 
local rankings as a feature in Moz Analytics, which gives our customers the ability to assign geo-locations to their tracked keywords. If you’re a Moz Analytics customer and are ready to jump right in, here’s where you an find the new feature within the application:

Not a Moz Analytics customer? You can take the new features for a free spin…

One of the biggest SEO developments of the last several years is how frequently Google is returning localized organics across a rapidly increasing number of search queries. It’s not just happening for “best pizza in Portland” (the answer to that is
Apizza Scholls, by the way). Searches like “financial planning” and “election guide” now trigger Google’s localization algorithm:

local search results election guide

This type of query underscores the need to track rankings on a local level. I’m searching for a non-localized keyword (“election guide”), but Google recognizes I’m searching from Portland, Oregon so they add the localization layer to the result.

Local tends to get lost in the shuffle of zoo animal updates we’ve seen from Google in the last couple of years, but search marketers are coming around to realize the 2012 Venice update was one of the most important changes Google made to the search landscape. It certainly didn’t seem like a huge deal when it launched; here’s how Google described Venice as part of the late lamented
monthly search product updates they used to provide:

  • Improvements to ranking for local search results. [launch codename “Venice”] This improvement improves the triggering of Local Universal results by relying more on the ranking of our main search results as a signal.

Seems innocent enough, right? What the Venice update actually kicked off was a long-term relationship between local search results (what we see in Google local packs and map results) and the organic search results that, once upon a time, existed on their own. “Localized organics,” as they are known, have been increasingly altering the organic search landscape for keywords that normally triggered “generic” or national rankings. If you haven’t already read it, Mike Ramsey’s article on
how to adjust for the Venice update remains one of the best strategic looks at the algorithm update.

This jump in localized organic results has prompted both marketers and business owners to track rankings at the local level. An increasing number of Moz customers have been requesting the ability to add locations to their keywords since the 2012 Venice update, and this is likely due to Google expanding the queries which trigger a localized result. You asked for it, and today we’re delivering. Our new local rankings feature allows our customers to track keywords for any city, state, or ZIP/postal code.

Geo-located searches

We can now return rankings based on a location you specify, just like I set my search to Portland in the example above. This is critical for monitoring the health of your local search campaigns, as Google continues to fold the location layer into the organic results. Here’s how it looks in Moz Analytics:

tracking local keyword ranking

A keyword with a location specified counts against your keyword limit in Moz Analytics just like any other keyword.

The location being tracked will also be displayed in your rankings reports as well as on the keyword analysis page:

local keyword difficulty

The local rankings feature allows you to enter your desired tracking location by city, state, neighborhood, and zip or postal code. We provide neighborhood-level granularity via dropdown for the United States, United Kingdom, Canada and Australia. The dropdown will also provide city-level listings for other countries. It’s also possible to enter a location of your choice not on the list in the text box. Fair warning: We cannot guarantee the accuracy of rankings in mythical locations like Westeros or Twin Peaks, or mythical spellings like Pordland or Los Andules.

An easy way to get started with the new feature is to look at keywords you are already tracking, and find the ones that have an obvious local intent for searchers. Then add the neighborhood or city you are targeting for the most qualified searchers.

What’s next?

We will be launching local rankings functionality within the Moz Local application in the first part of 2015, which will provide needed visibility to folks who are mainly concerned with Local SEO. We’re also working on functionality to allow users to easily add geo-modifiers to their tracked keywords, so we can provide rankings for “health club Des Moines” alongside tracking rankings for “health clubs” in the 50301 zip code.

Right now this feature works with all Google engines (we’ll be adding Bing and Yahoo! later). We’ll also be keeping tabs on Google’s advancements on the local front so we can provide our customers with the best data on their local visibility.

Please let us know what you think in the comments below! Customer feedback, suggestions, and comments were instrumental into both the design and prioritization of this feature.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

[ccw-atrib-link]

How To Tap Into Social Norms to Build a Strong Brand

Posted by bridget.randolph

In recent years there has been a necessary shift in the way businesses advertise themselves to consumers, thanks to the increasingly common information overload experienced by the average person.

In 1945, just after WWII, the
annual total ad spend in the United States was about $2.8 billion (that’s around $36.8 million before the adjustment for inflation). In 2013, it was around $140 billion.

Don’t forget that this is just paid media advertising; it doesn’t include the many types of earned coverage like search, social, email, supermarket displays, direct mail and so on. Alongside the growth in media spends is a growth in the sheer volume of products available, which is made possible by increasingly sophisticated technologies for sales, inventory, delivery and so on.

What does this mean? Well, simply that the strategy of ‘just buy some ads and sell the benefits’ isn’t enough anymore: you’ll be lost in the noise. How can a brand retain customers and create loyalty in an atmosphere where everyone else has a better offer? Through tapping into the psychology of social relationships.


Imagine that you are at home for Thanksgiving, and your mother has pulled out all the stops to lovingly craft the most delicious, intricate dinner ever known to man. You and your family have enjoyed a wonderful afternoon of socializing and snacking on leftovers and watching football, and now it’s time to leave. As you hug your parents goodbye, you take out your wallet. “How much do I owe you for all the love and time you put into this wonderful afternoon?” you ask. “$100 for the food? here, have $50 more as a thank you for the great hospitality!” How would your mother respond to such an offer? I don’t know about your mother, but my mom would be deeply offended.

New scenario: You’ve gone to a restaurant for Thanksgiving dinner. It’s the most delicious dinner you’ve ever had, the atmosphere is great with the football playing in the background, and best of all, your server is attentive, warm, and maternal. You feel right at home. At the end of the meal, you give her a hug and thank her for the delicious meal before leaving. She calls the cops and has you arrested for a dine-and-dash.

And herein lies the difference between social norms and market norms.

Social norms vs. market norms

The Thanksgiving dinner example is one which I’ve borrowed from a book by Dan Ariely,
Predictably Irrational: The Hidden Forces that Shape Our Decisions. Ariely discusses two ways in which humans interact: social norms and market norms.


Social norms
, as Ariely explains, “are wrapped up in our social nature and our need for community. They are usually warm and fuzzy. Instant paybacks are not required.” Examples would be: helping a friend move house, babysitting your grandchild, having your parents over for dinner. There is an implied reciprocity on some level but it is not instantaneous nor is it expected that the action will be repaid on a financial level. These are the sort of relationships and interactions we expect to have with friends and family.


Market norms
, on the other hand, are about the exchange of resources and in particular, money. Examples of this type of interaction would be any type of business transaction where goods or services are exchanged for money: wages, prices, rents, interest, and cost-and-benefit. These are the sort of relationships and interactions we expect to have with businesses.

I’ve drawn you a very rough illustration – it may not be the most aesthetically pleasing visual, but it gets the point across:

Market norms come into play any time money enters into the equation, sometimes counter-intuitively! Ariely gives the example of a group of lawyers who were approached by the AARP and asked whether they would provide legal services to needy retirees at a drastically discounted rate of $30/hour. The lawyers said no. From a market norms perspective, the exchange didn’t make sense. Later the same lawyers were asked whether they would consider donating their time free of charge to needy retirees. The vast majority of the lawyers said yes. The difference is that, when no money changes hands, the exchange shifts from a poor-value market exchange to an altruistic and therefore high-value social exchange. It is a strange psychological quirk that ‘once market norms enter our considerations, the social norms depart.’

Mixed signals: when social and market norms collide

In a book called
Positioning: The Battle for Your Mind by Al Ries and Jack Trout (originally published in 1981), the authors describe the 1950s as the ‘product era’ of advertising, when ‘advertising people focused their attention on product features and customer benefits.’ It was all about the unique selling proposition (USP).


In this case, the USP is mildness: “not one single case of throat irritation!” (image source)

However, as the sheer volume of products on the market increased, it became more difficult to sell a product simply by pointing out the benefits. As Ries and Trout put it, ‘Your “better mousetrap” was quickly followed by two more just like it. Both claiming to be better than the first one.’

They describe the next phase of advertising (which hit its peak in the 1960s and 70s and which we can probably all relate to if we watch Mad Men) as the ‘image era’, pioneered by David Ogilvy. In this period, successful campaigns sold the reputation, or ‘image’ of a brand and a product rather than its features. Ries and Trout quote Ogilvy as saying that ‘Every advertisement is a long-term investment in the image of a brand’. Examples include Hathaway shirts and Rolls-Royce.

Rather than the product benefits, this ad focuses on the ‘image’ of the man who smokes Viceroys: “Viceroy has a thinking man’s filter and a smoking man’s taste. (image source)

But yet again, as more and more brands imitate the strategy of these successful campaigns, the space gets more crowded and the consumer becomes more jaded and these techniques become less effective.

According to Ries and Trout, this brought the world of advertising into the ‘positioning era’ of the 80s, which is where they positioned (hehe) themselves. As they described this, “To succeed in our overcommunicated society, a company must create a position in the prospect’s mind, a position that takes into consideration not only a company’s own strengths and weaknesses, but those of its competitors as well.”

This one’s all about positioning Winston’s in opposition to competitors: as the brand with real taste, as opposed to other brands which ‘promise taste’ but fail to deliver. (image source)

And yet, despite this evolution of advertising strategy over the course of the 20th century, all of these different approaches are ultimately based on market norms. The ‘product era’ sells you features and benefits in exchange for money; the ‘image era’ sells you on an image and a lifestyle in exchange for money, and the ‘positioning era’ sells you on why a particular company is the right one to supply your needs in exchange for money.

Social norms and loyalty


When does cheap not win?
When it comes to social norms. Social norms are about relationships, community and loyalty. If your sister is getting married, you don’t do a cost benefit analysis to decide whether or not you should go to her wedding or whether the food will be better and the travel cheaper if you go to your next door neighbor’s BBQ instead. If anything, it’s the opposite: some people take it to such an extreme that they will go into massive debt to attend friends’ weddings and bring lavish gifts. That is certainly not a decision based on monetary considerations.

Therefore, if the average brand wants to get out of the vicious cycle of undercutting competitors in order to gain business, they need to start focusing on relationships and community building instead of ‘SUPER CHEAP BEST LOW LOW PRICES!!®’ and sneaky upsells at the point of sale. This is something my colleague
Tim Allen spoke about in a presentation called “Make Me Love Your Brand, Not Just Tolerate It”. And this is what a large number of recent ‘advertising success stories’ are based on and it’s the whole premise behind many of the more recent trends in marketing: email marketing, personalization, SMS marketing, good social media marketing, and so on.

Some of the most popular brands are the ones which are able to find the perfect balance between:

  • a friendly, warm relationship with customers and potential customers, which also often includes a fun, personal tone of voice (the ‘brand personality’) – in these interactions there is often an offering of something to the customer without an expectation of instant payback, and
  • a strong product which they offer at a good price with good ‘market’ benefits like free returns and so on.

One example of this is John Lewis, who have good customer service policies around returns etc but also offer free perks to their shoppers, like the maternity room where breastfeeding mothers can relax. One of my colleagues mentioned that, as a new mother, his girlfriend always prefers to shop at John Lewis over other competitor stores for that very reason. Now if this is purely a convenience factor for her, and after her child is older she stops shopping at John Lewis in favor of a cheaper option, you could argue that this is less of a social interaction and more market influenced (in some sense it serves as a service differentiator between JL and their customers). However, if after she no longer requires the service, she continues to shop there because she wants to reciprocate their past support of her as a breastfeeding mother, that pushes it more firmly into the realm of the social.

Another thing John Lewis do for their fans is the annual Christmas ad, which (much like the 
Coca-Cola Santa truck in the UK) has become something which people look forward to each year because it’s a heartwarming little story more than just an ad for a home and garden store. Their 2012 ad was my favorite (and a lot of other people’s too, with over 4.5 million Youtube views).

But usually anytime a brand ‘do something nice’ for no immediate monetary benefit, it counts as a ‘social’ interaction – a classic example is
Sainsbury’s response to the little girl who wrote to them about ‘tiger bread’.

Some of my other favorite examples of social norm interactions by brands are:

The catch is, you have to be careful and keep the ‘mix’ of social and market norms consistent.

Ariely uses the example of a bank when describing the danger of bringing social norms into a business relationship:

“What happens if a customer’s check bounces? If the relationship is based on market norms, the bank charges a fee, and the customer shakes it off. Business is business. While the fee is annoying, it’s nonetheless acceptable. In a social relationship, however, a hefty late fee–rather than a friendly call from the manager or an automatic fee waiver–is not only a relationship-killer; it’s a stab in the back. Consumers will take personal offense. They’ll leave the bank angry and spend hours complaining to their friends about this awful bank.”

Richard Fergie also summed this issue up nicely in this G+ post about the recent outrage over Facebook manipulating users’ emotions; in this case, the back-stab effect was due to the fact that the implicit agreement between the users and the company about what was being ‘sold’ and therefore ‘valued’ in the exchange changed without warning.


The basic rule of thumb is that whether you choose to emphasize market norms or social norms, you can’t arbitrarily change the rules.

A side note about social media and brands: Act like a normal person

In a time when
the average American aged 18-64 spends 2-3 hours a day on social media, it is only logical that we would start to see brands and the advertising industry follow suit. But if this is your only strategy for building relationships and interacting with your customers socially, it’s not good enough. Instead, in this new ‘relationship era’ of advertising (as I’ve just pretentiously dubbed it, in true Ries-and-Trout fashion), the brands who will successfully merge market and social norms in their advertising will be the brands which are able to develop the sort of reciprocal relationships that we see with our friends and family. I wrote a post over on the Distilled blog about what social media marketers can learn from weddings. That was just one example, but the TL;DR is: as a brand, you still need to use social media the way that normal people do. Otherwise you risk becoming a Condescending Corporate Brand on Facebook. On Twitter too.

Social norms and authenticity: Why you actually do need to care

Another way in which brands tap into social norms are through their brand values. My colleague
Hannah Smith talked about this in her post on The Future of Marketing. Moz themselves are a great example of a brand with strong values: for them it’s TAGFEE. Hannah also gives the examples of Innocent Drinks (sustainability), Patagonia (environmentalism) and Nike (whose strapline ‘Find Your Greatness’ is about their brand values of everyone being able to ‘achieve their own defining moment of greatness’).

Havas Media have been doing some interesting work around trying to ‘measure’ brand sentiment with something call the
‘Meaningful Brands Index’ (MBi), based on how much a brand is perceived as making a meaningful difference in people’s lives, both for personal wellbeing and collective wellbeing. Whether or not you like their approach, they have some interesting stats: apparently only 20% of brands worldwide are seen to ‘meaningfully positively impact peoples’ lives’, but the brands that rank high on the MBi also tend to outperform other brands significantly (120%).

Now there may be a ‘correlation vs causation’ argument here, and I don’t have space to explore it. But regardless of whether you like the MBi as a metric or not, countless case studies demonstrate that it’s valuable for a brand to have strong brand values.

There are two basic rules of thumb when it comes to choosing brand values:

1) I
t has to be relevant to what you do. If a bingo site is running an environmentalism campaign, it might seem a bit weird and it won’t resonate well with your audience. You also need to watch out for accidental irony. For example, McDonalds and Coca-Cola came in for some flak when they sponsored the Olympics, due to their reputation as purveyors of unhealthy food/drink products.

Nike’s #FindYourGreatness campaign, on the other hand, is a great example of how to tie in your values with your product. Another example is one of our clients at Distilled, SimplyBusiness, a business insurance company whose brand values include being ‘the small business champion’. This has informed their content strategy, leading them to develop in-depth resources for small businesses, and it has served them very well.

2) I
t can’t be so closely connected to what you do that it comes across as self-serving. For example, NatWest’s NatYes campaign claims to be about enabling people to become homeowners, but ultimately (in no small part thanks to the scary legal compliance small print about foreclosure) the authenticity of the message is undermined.

The most important thing when it comes to brand values: it’s very easy for people to be cynical about brands and whether they ‘care’. Havas did a survey that found that
only 32% of people feel that brands communicate honestly about commitments and promises. So choose values that you do feel strongly about and follow through even if it means potentially alienating some people. The recent OKCupid vs Mozilla Firefox episode is an illustration of standing up for brand values (regardless of where you stand on this particular example, it got them a lot of positive publicity).

Key takeaways

So what can we take away from these basic principles of social norms and market norms? If you want to build a brand based on social relationships, here’s 3 things to remember.

1)
Your brand needs to provide something besides just a low price. In order to have a social relationship with your customers, your brand needs a personality, a tone of voice, and you need to do nice things for your customers without the expectation of immediate payback.

2)
You need to keep your mix of social and market norms consistent at every stage of the customer lifecycle. Don’t pull the rug out from under your loyal fans by hitting them with surprise costs after they checkout or other tricks. And don’t give new customers significantly better benefits. What you gain in the short term you will lose in the long term resentment they will feel about having been fooled. Instead, treat them with transparency and fairness and be responsive to customer service issues.

3)
You need brand values that make sense for your brand and that you (personally and as a company) really believe in. Don’t have values that don’t relate to your core business. Don’t have values which are obviously self-serving. Don’t be accidentally ironic like McDonalds.

Have you seen examples of brands building customer relationships based on social norms? Did it work? Do you do this type of relationship-building for your brand?

I’d love to hear your thoughts in the comments.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

[ccw-atrib-link]