Stop Ghost Spam in Google Analytics with One Filter

Posted by CarloSeo

The spam in Google Analytics (GA) is becoming a serious issue. Due to a deluge of referral spam from social buttons, adult sites, and many, many other sources, people are starting to become overwhelmed by all the filters they are setting up to manage the useless data they are receiving.

The good news is, there is no need to panic. In this post, I’m going to focus on the most common mistakes people make when fighting spam in GA, and explain an efficient way to prevent it.

But first, let’s make sure we understand how spam works. A couple of months ago, Jared Gardner wrote an excellent article explaining what referral spam is, including its intended purpose. He also pointed out some great examples of referral spam.

Types of spam

The spam in Google Analytics can be categorized by two types: ghosts and crawlers.

Ghosts

The vast majority of spam is this type. They are called ghosts because they never access your site. It is important to keep this in mind, as it’s key to creating a more efficient solution for managing spam.

As unusual as it sounds, this type of spam doesn’t have any interaction with your site at all. You may wonder how that is possible since one of the main purposes of GA is to track visits to our sites.

They do it by using the Measurement Protocol, which allows people to send data directly to Google Analytics’ servers. Using this method, and probably randomly generated tracking codes (UA-XXXXX-1) as well, the spammers leave a “visit” with fake data, without even knowing who they are hitting.

Crawlers

This type of spam, the opposite to ghost spam, does access your site. As the name implies, these spam bots crawl your pages, ignoring rules like those found in robots.txt that are supposed to stop them from reading your site. When they exit your site, they leave a record on your reports that appears similar to a legitimate visit.

Crawlers are harder to identify because they know their targets and use real data. But it is also true that new ones seldom appear. So if you detect a referral in your analytics that looks suspicious, researching it on Google or checking it against this list might help you answer the question of whether or not it is spammy.

Most common mistakes made when dealing with spam in GA

I’ve been following this issue closely for the last few months. According to the comments people have made on my articles and conversations I’ve found in discussion forums, there are primarily three mistakes people make when dealing with spam in Google Analytics.

Mistake #1. Blocking ghost spam from the .htaccess file

One of the biggest mistakes people make is trying to block Ghost Spam from the .htaccess file.

For those who are not familiar with this file, one of its main functions is to allow/block access to your site. Now we know that ghosts never reach your site, so adding them here won’t have any effect and will only add useless lines to your .htaccess file.

Ghost spam usually shows up for a few days and then disappears. As a result, sometimes people think that they successfully blocked it from here when really it’s just a coincidence of timing.

Then when the spammers later return, they get worried because the solution is not working anymore, and they think the spammer somehow bypassed the barriers they set up.

The truth is, the .htaccess file can only effectively block crawlers such as buttons-for-website.com and a few others since these access your site. Most of the spam can’t be blocked using this method, so there is no other option than using filters to exclude them.

Mistake #2. Using the referral exclusion list to stop spam

Another error is trying to use the referral exclusion list to stop the spam. The name may confuse you, but this list is not intended to exclude referrals in the way we want to for the spam. It has other purposes.

For example, when a customer buys something, sometimes they get redirected to a third-party page for payment. After making a payment, they’re redirected back to you website, and GA records that as a new referral. It is appropriate to use referral exclusion list to prevent this from happening.

If you try to use the referral exclusion list to manage spam, however, the referral part will be stripped since there is no preexisting record. As a result, a direct visit will be recorded, and you will have a bigger problem than the one you started with since. You will still have spam, and direct visits are harder to track.

Mistake #3. Worrying that bounce rate changes will affect rankings

When people see that the bounce rate changes drastically because of the spam, they start worrying about the impact that it will have on their rankings in the SERPs.

bounce.png

This is another mistake commonly made. With or without spam, Google doesn’t take into consideration Google Analytics metrics as a ranking factor. Here is an explanation about this from Matt Cutts, the former head of Google’s web spam team.

And if you think about it, Cutts’ explanation makes sense; because although many people have GA, not everyone uses it.

Assuming your site has been hacked

Another common concern when people see strange landing pages coming from spam on their reports is that they have been hacked.

landing page

The page that the spam shows on the reports doesn’t exist, and if you try to open it, you will get a 404 page. Your site hasn’t been compromised.

But you have to make sure the page doesn’t exist. Because there are cases (not spam) where some sites have a security breach and get injected with pages full of bad keywords to defame the website.

What should you worry about?

Now that we’ve discarded security issues and their effects on rankings, the only thing left to worry about is your data. The fake trail that the spam leaves behind pollutes your reports.

It might have greater or lesser impact depending on your site traffic, but everyone is susceptible to the spam.

Small and midsize sites are the most easily impacted – not only because a big part of their traffic can be spam, but also because usually these sites are self-managed and sometimes don’t have the support of an analyst or a webmaster.

Big sites with a lot of traffic can also be impacted by spam, and although the impact can be insignificant, invalid traffic means inaccurate reports no matter the size of the website. As an analyst, you should be able to explain what’s going on in even in the most granular reports.

You only need one filter to deal with ghost spam

Usually it is recommended to add the referral to an exclusion filter after it is spotted. Although this is useful for a quick action against the spam, it has three big disadvantages.

  • Making filters every week for every new spam detected is tedious and time-consuming, especially if you manage many sites. Plus, by the time you apply the filter, and it starts working, you already have some affected data.
  • Some of the spammers use direct visits along with the referrals.
  • These direct hits won’t be stopped by the filter so even if you are excluding the referral you will sill be receiving invalid traffic, which explains why some people have seen an unusual spike in direct traffic.

Luckily, there is a good way to prevent all these problems. Most of the spam (ghost) works by hitting GA’s random tracking-IDs, meaning the offender doesn’t really know who is the target, and for that reason either the hostname is not set or it uses a fake one. (See report below)

Ghost-Spam.png

You can see that they use some weird names or don’t even bother to set one. Although there are some known names in the list, these can be easily added by the spammer.

On the other hand, valid traffic will always use a real hostname. In most of the cases, this will be the domain. But it also can also result from paid services, translation services, or any other place where you’ve inserted GA tracking code.

Valid-Referral.png

Based on this, we can make a filter that will include only hits that use real hostnames. This will automatically exclude all hits from ghost spam, whether it shows up as a referral, keyword, or pageview; or even as a direct visit.

To create this filter, you will need to find the report of hostnames. Here’s how:

  1. Go to the Reporting tab in GA
  2. Click on Audience in the lefthand panel
  3. Expand Technology and select Network
  4. At the top of the report, click on Hostname

Valid-list

You will see a list of all hostnames, including the ones that the spam uses. Make a list of all the valid hostnames you find, as follows:

  • yourmaindomain.com
  • blog.yourmaindomain.com
  • es.yourmaindomain.com
  • payingservice.com
  • translatetool.com
  • anotheruseddomain.com

For small to medium sites, this list of hostnames will likely consist of the main domain and a couple of subdomains. After you are sure you got all of them, create a regular expression similar to this one:

yourmaindomain\.com|anotheruseddomain\.com|payingservice\.com|translatetool\.com

You don’t need to put all of your subdomains in the regular expression. The main domain will match all of them. If you don’t have a view set up without filters, create one now.

Then create a Custom Filter.

Make sure you select INCLUDE, then select “Hostname” on the filter field, and copy your expression into the Filter Pattern box.

filter

You might want to verify the filter before saving to check that everything is okay. Once you’re ready, set it to save, and apply the filter to all the views you want (except the view without filters).

This single filter will get rid of future occurrences of ghost spam that use invalid hostnames, and it doesn’t require much maintenance. But it’s important that every time you add your tracking code to any service, you add it to the end of the filter.

Now you should only need to take care of the crawler spam. Since crawlers access your site, you can block them by adding these lines to the .htaccess file:

## STOP REFERRER SPAM 
RewriteCond %{HTTP_REFERER} semalt\.com [NC,OR] 
RewriteCond %{HTTP_REFERER} buttons-for-website\.com [NC] 
RewriteRule .* - [F]

It is important to note that this file is very sensitive, and misplacing a single character it it can bring down your entire site. Therefore, make sure you create a backup copy of your .htaccess file prior to editing it.

If you don’t feel comfortable messing around with your .htaccess file, you can alternatively make an expression with all the crawlers, then and add it to an exclude filter by Campaign Source.

Implement these combined solutions, and you will worry much less about spam contaminating your analytics data. This will have the added benefit of freeing up more time for you to spend actually analyze your valid data.

After stopping spam, you can also get clean reports from the historical data by using the same expressions in an Advance Segment to exclude all the spam.

Bonus resources to help you manage spam

If you still need more information to help you understand and deal with the spam on your GA reports, you can read my main article on the subject here: http://www.ohow.co/what-is-referrer-spam-how-stop-it-guide/.

Additional information on how to stop spam can be found at these URLs:

In closing, I am eager to hear your ideas on this serious issue. Please share them in the comments below.

(Editor’s Note: All images featured in this post were created by the author.)

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 3 years ago from tracking.feedpress.it

How to Have a Successful Local SEO Campaign in 2015

Posted by Casey_Meraz

Another year in search has passed. It’s now 2015 and we have seen some major changes in local ranking factors since 2014, which I also expect to change greatly throughout 2015. For some a new year means a fresh starting point and yet for others it’s a time of reflection to analyze how successful your campaign has been. Whatever boat you’re in, make sure to sit down and read this guide. 

In this guide we will cover how you can have a successful local SEO campaign in 2015 starting with the basics and getting down to five action items you should focus on now. This is not limited to Google My Business and also includes localized organic results. 

Now the question is where do you start?

Since Pigeon has now rolled out to the US, UK, Australia, and Canada it’s important to make sure your strategies are in line with this no matter what part of the world you’re in. A successful local SEO Campaign in 2015 will be much more successful if you put more work into it. Don’t be fooled though. More work by itself isn’t going to get you where you need to be. You need to work smarter towards the goals which are going to fuel your conversions.

For some industries that might mean more localized content, for others it may mean more social interaction in your local area. Whatever it ends up being, the root of it should be the same for most. You need to get more conversions for your website or your client’s website. So with this in mind let’s make sure we’re on the same page as far as our goals are concerned.

Things you need to know first

Focus on the right goals

Recently I had a conversation with a client who wanted to really nail in the point that
he was not interested in traffic. He was interested in the conversions he could track. He was also interested to see how all of these content resource pieces I recommended would help. He was tired of the silly graphs from other agencies that showed great rankings on a variety of keywords when he was more interested to see which efforts brought him the most value. Instead, he wanted to see how his campaign was bringing him conversions or meaningful traffic. I really appreciated this statement and I felt like he really got it.

Still, however, far too often I have to talk to potential clients and explain to them why their sexy looking traffic reports aren’t actually helping them. You can have all of the traffic in the world but if it doesn’t meet one of your goals of conversions or education then it’s probably not helping. Even if you make the client happy with your snazzy reports for a few months, eventually they’re going to want to know their return on investment (ROI).

It’s 2015. If your clients aren’t tracking conversions properly, give them the help they need. Record their contacts in a CRM and track the source of each of these contacts. Track them all the way through the sales funnel. 

That’s a simple and basic marketing example but as SEOs
your role has transformed. If you can show this type of actual value and develop a plan accordingly, you will be unstoppable.

Second, don’t get tunnel vision

You may wonder why I started a little more basic than normal in this post. The fact is that in this industry there is not a full formal training program that covers all aspects of what we do. 

We all come from different walks of life and experience which makes it easy for us to get tunnel vision. You probably opened this article with the idea of “How Can I Dominate My Google Local Rankings?” While we cover some actionable tips you should be using, you need to think outside of the box as well. Your website is not the only online property you need to be concerned about.

Mike Ramsey from Nifty Marketing put out a great study on 
measuring the click-through rates from the new local stack. In this study he measured click-through rates of users conducting several different searches like “Salt Lake City Hotel” in the example below. With so many different options look where the users are clicking:

They’re really clicking all over the place! While it’s cool to be number one, it’s much better if you get clicks from your paid ad, organic result, local result, and barnacle SEO efforts (which we’ll talk about a little later). 

If you combine your conversion marketing data with your biggest priorities, you can put together a plan to tackle the most important areas for your industry. Don’t assume it’s a one-size-fits-all approach. 

Third, some spam still works. Don’t do it and rise above it.

There’s no doubt that some spammy tactics are still working. Google gets better everyday but you still see crap
like this example below show up in the SERPs.

While it sucks to see that kind of stuff, remember that in time it disappears (just as it did before this article was published). If you take shortcuts, you’re going to get caught and it’s not worth it for the client or the heartache on your site. Maintain the course and do things the right way. 

Now let’s get tactical and prepare for 2015

Now it’s time for some practical and tactical takeaways you can use to dominate your local search campaign in 2015.

Practical tip 1: start with an audit

Over the years, one of the best lessons I have learned is it’s OK to say “I don’t know” when you don’t have the answer. Consulting with industry experts or people with more experience than you is not a bad thing and will likely only lead to you to enhance your knowledge and get a different perspective. It can be humbling but the experience is amazing. It can open your mind.

Last year, I had the opportunity to work with over ten of the industry’s best minds and retained them for site audits on different matters. 

The perspective this gives is absolutely incredible and I believe it’s a great way to learn. Everyone in this industry has come from a different background and seen different things over the years. Combining that knowledge is invaluable to the success of your clients’ projects. Don’t be afraid to do it and learn from it. This is also a good idea if you feel like your project has reached a stalemate. Getting more advice, identifying potential problems, and having a fresh perspective will do wonders for your success.

As many of the experts have confirmed, ever since the Pigeon update, organic and local ranking factors have been more tied together than ever. Since they started going this direction in a big way, I would not expect it to stop. 

This means that you really do need to worry about things like site speed, content, penalties, mobile compatibility, site structure, and more. On a side note, guess what will happen to your organic results if you keep this as a top priority? They will flourish and you will thank me.

If you don’t have the budget or resources to get a third party opinion, you can also conduct an independent audit. 

Do it yourself local SEO audit resources:

Do it yourself organic SEO audit resources:

Alternatively if you’re more in the organic boat you should also check out this guide by Steve Webb on
How To Perform The World’s Greatest SEO Audit

Whatever your situation is, it’s worth the time to have this perspective yearly or even a couple times a year if possible.

Practical tip 2: consider behavioral signals and optimize accordingly

I remember having a conversation with Darren Shaw, the founder of 
Whitespark, at MozCon 2013 about his thoughts on user behavior affecting local results. At the time I didn’t do too much testing around it. However just this year, Darren had a mind-blowing presentation at the Dallas State of Search where he threw in the behavioral signals curve ball. Phil Rozek also spoke about behavioral signals and provided a great slide deck with actionable items (included below). 

We have always speculated on behavioral signals but between his tests and some of Rand’s IMEC Lab tests, I became more of a believer last year. Now, before we go too deep on this remember that your local campaign is NOT only focused on just your local pack results. If user behavior can have an impact on search results, we should definitely be optimizing for our users.


You can view Phil Rozek’s presentation below: 

Don’t just optimize for the engines, optimize for the humans. One day when Skynet is around this may not be an issue, but for now you need to do it.

So how can you optimize for behavioral signals?

There is a dark side and a light side path to this question. If you ask me I will always say follow the light side as it will be effective and you don’t have to worry about being penalized. That’s a serious issue and it’s unethical for you to put your clients in that position.

Local SEO: how to optimize for behavioral signals

Do you remember the click-through study we looked at a bit earlier from Nifty Marketing? Do you remember where the users clicked? If you look again or just analyze user and shopper behavior, you might notice that many of the results with the most reviews got clicks. We know that reviews are hard to get so here are two quick ways that I use and recommend to my clients:


1. Solicit your Gmail clients for reviews

If you have a list of happy Gmail clients you can simply send them an email with a direct link to your Google My Business Page. Just get the URL of your local page by pulling up your URL and copying and pasting it. A URL will look like the one below:

Once you have this URL, simply remove the /posts and replace it with: 

 /?hl=en&review=1


It will look like this:

If your clients click on this link via their logged-in Gmail, they will automatically be taken to the review page which will open up the box to leave a review which looks like the example below. It doesn’t get much more simple than that. 

2. Check out a service like Mike Blumenthal’s Get Five Stars for reviews

I recently used this with a client and got a lot of great feedback and several reviews.

Remember that these reviews will also help on third-party sites and can help your Google My Business ranking positions as well as click-through rates. You can
check out Get Five Stars Here.

Another way outside of getting reviews is to optimize the appearance of your Google My Business Page. 


3. Optimize your local photos

Your Google My Business page includes photos. Don’t use generic photos. Use high quality photos so when the users hover over your listing they get an accurate representation of what they’re looking for. Doing this will increase your click-through rate. 

Organic SEO: Optimize for Behavioral Signals

The optimization for click-through rates on organic results typically focus on three areas. While you’re likely very familiar with the first two, you should not ignore them.


1. Title tags: optimize them for the user and engine

Optimize your meta title tags to increase click-through rates. Each page should have a unique title tag and should help the viewer with their query. The example below (although fabricated) is a good example of what NOT to do. 


2. Meta descriptions: optimize them for the user

Optimize your meta description to get the user to click on the search result. If you’re not doing this just because Google may or may not pull it, you’re missing opportunities and clicks. 


3. Review Schema markup: add this to appropriate pages

Reviewing
Schema markup is still a very overlooked opportunity. Like we talked about above in the local section, if you don’t have reviews coded in Schema, you could be missing out on getting the orange stars in organic results. 

Practical tip 3: don’t ignore barnacle SEO

I firmly believe that most people are not taking advantage of barnacle SEO still to this day and I’m a big fan. When I first heard Will Scott introduce this term at Pubcon I thought it was spot on. According to Will Scott’s website Search Influence, barnacle SEO is “attaching oneself to a large fixed object and waiting for the customers to float by in the current.” In a nutshell, we know that if you’re trying to rank on page one of Google you will find others that you may be able to attach to. If Yelp results come up for a lot of your search terms you might identify that as an opportunity. But there are three main ways you can take advantage of this.


1. You can try to have the most visible profile on that third party page

If Yelp is ranking for LA Personal Injury Attorneys, it would suit you to figure out how the top users are showing up there. Maybe your customers are headed there and then doing some shopping and making a selection. Or maybe they’re using it for a research platform and then will visit your website. If your profile looks great and shows up high on the list, you just gave yourself a better chance at getting a conversion.


2. You can try to get your page to rank

Hey, just because you don’t own Yelp.com or whatever similar site you’ve found, doesn’t mean you shouldn’t put in the effort to have it rank. If Google is already showing you that they trust a third party site by ranking it, you can use similar organic ranking techniques that you would use on your own site to make your profile page stronger. Over time you might add this to your bio on interviews or other websites to earn links. If you increase the visibility of your profile on search engines and they see your website on the same page you might increase conversions.


3. You can help your Google My Business

If the site you’re using passes link juice and you earn links to the third party profile page, you will start to see some strong results. Links are a big factor in local since Pigeon this year and it’s an opportunity that should not be missed.


So how can you use this advice?

Start by finding a list of potential barnacle SEO partners for your industry. As an example, I did a search for “Personal Injury Attorneys” in Los Angeles. In addition to the law firms that showed up in the results on the first page, I also identified four additional places I may be able to show up on.

  1. Yelp
  2.  Thumbtack
  3. Avvo
  4. Wikipedia

If you were attorney, it would be worth your while to explore these and see if any make sense for you to contribute to.

Practical tip 4: earn some good links

Most people get too carried away with link building. I know because I used to do it. The key with link building is to change your approach to understand that
it’s always better to get fewer high quality links than hundreds or thousands of low quality links

For example, a link like this one that one of our clients earned is what I’m talking about. 

If you want to increase your local rankings you can do so by earning these links to your associated Google My Business landing page.

Do you know the URL you entered in your Google My Business page when you set it up? That’s the one I’m talking about. In most cases this will be linked to either a local landing page for that location or the home page. It’s essential to your success that you earn solid links to this page.


Simple resources for link building

Practical tip 5: have consistent citations and remove duplicates

Identifying and correcting incorrect or duplicate citations has been getting easier and easier over the years. Even if you don’t want to pay someone to do it, you can sign up for some great do-it-yourself tools. Your goal with any citation cleanup program is this:

  1. Ensure there are no duplicate citations
  2. Ensure there are no incorrect citations with wrong phone numbers, old addresses, etc. 

You can ignore small differences and inconsistencies like St vs. Street. I believe the importance of citations has been greatly reduced over the past year. At the same time, you still want to be the least imperfect and provide your customers with accurate information if they’re looking on third party websites.  

Let’s do good things in 2015

2014 was a tough year in search altogether. We had ups like Penguin refreshes and we had downs like the removal of authorship. I’m guessing 2015 will be no different. Staying on the roller coaster and keeping with the idea of having the “least imperfect” site is the best way to ring out the new year and march on moving forward. If you had a tough year in local search, keep your head up high, fix any existing issues, and sprint through this year by making positive changes to your site. 

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 4 years ago from moz.com