Why We Can’t Do Keyword Research Like It’s 2010 – Whiteboard Friday

Posted by randfish

Keyword Research is a very different field than it was just five years ago, and if we don’t keep up with the times we might end up doing more harm than good. From the research itself to the selection and targeting process, in today’s Whiteboard Friday Rand explains what has changed and what we all need to do to conduct effective keyword research today.

For reference, here’s a still of this week’s whiteboard. Click on it to open a high resolution image in a new tab!

What do we need to change to keep up with the changing world of keyword research?

Howdy, Moz fans, and welcome to another edition of Whiteboard Friday. This week we’re going to chat a little bit about keyword research, why it’s changed from the last five, six years and what we need to do differently now that things have changed. So I want to talk about changing up not just the research but also the selection and targeting process.

There are three big areas that I’ll cover here. There’s lots more in-depth stuff, but I think we should start with these three.

1) The Adwords keyword tool hides data!

This is where almost all of us in the SEO world start and oftentimes end with our keyword research. We go to AdWords Keyword Tool, what used to be the external keyword tool and now is inside AdWords Ad Planner. We go inside that tool, and we look at the volume that’s reported and we sort of record that as, well, it’s not good, but it’s the best we’re going to do.

However, I think there are a few things to consider here. First off, that tool is hiding data. What I mean by that is not that they’re not telling the truth, but they’re not telling the whole truth. They’re not telling nothing but the truth, because those rounded off numbers that you always see, you know that those are inaccurate. Anytime you’ve bought keywords, you’ve seen that the impression count never matches the count that you see in the AdWords tool. It’s not usually massively off, but it’s often off by a good degree, and the only thing it’s great for is telling relative volume from one from another.

But because AdWords hides data essentially by saying like, “Hey, you’re going to type in . . .” Let’s say I’m going to type in “college tuition,” and Google knows that a lot of people search for how to reduce college tuition, but that doesn’t come up in the suggestions because it’s not a commercial term, or they don’t think that an advertiser who bids on that is going to do particularly well and so they don’t show it in there. I’m giving an example. They might indeed show that one.

But because that data is hidden, we need to go deeper. We need to go beyond and look at things like Google Suggest and related searches, which are down at the bottom. We need to start conducting customer interviews and staff interviews, which hopefully has always been part of your brainstorming process but really needs to be now. Then you can apply that to AdWords. You can apply that to suggest and related.

The beautiful thing is once you get these tools from places like visiting forums or communities, discussion boards and seeing what terms and phrases people are using, you can collect all this stuff up, plug it back into AdWords, and now they will tell you how much volume they’ve got. So you take that how to lower college tuition term, you plug it into AdWords, they will show you a number, a non-zero number. They were just hiding it in the suggestions because they thought, “Hey, you probably don’t want to bid on that. That won’t bring you a good ROI.” So you’ve got to be careful with that, especially when it comes to SEO kinds of keyword research.

2) Building separate pages for each term or phrase doesn’t make sense

It used to be the case that we built separate pages for every single term and phrase that was in there, because we wanted to have the maximum keyword targeting that we could. So it didn’t matter to us that college scholarship and university scholarships were essentially people looking for exactly the same thing, just using different terminology. We would make one page for one and one page for the other. That’s not the case anymore.

Today, we need to group by the same searcher intent. If two searchers are searching for two different terms or phrases but both of them have exactly the same intent, they want the same information, they’re looking for the same answers, their query is going to be resolved by the same content, we want one page to serve those, and that’s changed up a little bit of how we’ve done keyword research and how we do selection and targeting as well.

3) Build your keyword consideration and prioritization spreadsheet with the right metrics

Everybody’s got an Excel version of this, because I think there’s just no awesome tool out there that everyone loves yet that kind of solves this problem for us, and Excel is very, very flexible. So we go into Excel, we put in our keyword, the volume, and then a lot of times we almost stop there. We did keyword volume and then like value to the business and then we prioritize.

What are all these new columns you’re showing me, Rand? Well, here I think is how sophisticated, modern SEOs that I’m seeing in the more advanced agencies, the more advanced in-house practitioners, this is what I’m seeing them add to the keyword process.

Difficulty

A lot of folks have done this, but difficulty helps us say, “Hey, this has a lot of volume, but it’s going to be tremendously hard to rank.”

The difficulty score that Moz uses and attempts to calculate is a weighted average of the top 10 domain authorities. It also uses page authority, so it’s kind of a weighted stack out of the two. If you’re seeing very, very challenging pages, very challenging domains to get in there, it’s going to be super hard to rank against them. The difficulty is high. For all of these ones it’s going to be high because college and university terms are just incredibly lucrative.

That difficulty can help bias you against chasing after terms and phrases for which you are very unlikely to rank for at least early on. If you feel like, “Hey, I already have a powerful domain. I can rank for everything I want. I am the thousand pound gorilla in my space,” great. Go after the difficulty of your choice, but this helps prioritize.

Opportunity

This is actually very rarely used, but I think sophisticated marketers are using it extremely intelligently. Essentially what they’re saying is, “Hey, if you look at a set of search results, sometimes there are two or three ads at the top instead of just the ones on the sidebar, and that’s biasing some of the click-through rate curve.” Sometimes there’s an instant answer or a Knowledge Graph or a news box or images or video, or all these kinds of things that search results can be marked up with, that are not just the classic 10 web results. Unfortunately, if you’re building a spreadsheet like this and treating every single search result like it’s just 10 blue links, well you’re going to lose out. You’re missing the potential opportunity and the opportunity cost that comes with ads at the top or all of these kinds of features that will bias the click-through rate curve.

So what I’ve seen some really smart marketers do is essentially build some kind of a framework to say, “Hey, you know what? When we see that there’s a top ad and an instant answer, we’re saying the opportunity if I was ranking number 1 is not 10 out of 10. I don’t expect to get whatever the average traffic for the number 1 position is. I expect to get something considerably less than that. Maybe something around 60% of that, because of this instant answer and these top ads.” So I’m going to mark this opportunity as a 6 out of 10.

There are 2 top ads here, so I’m giving this a 7 out of 10. This has two top ads and then it has a news block below the first position. So again, I’m going to reduce that click-through rate. I think that’s going down to a 6 out of 10.

You can get more and less scientific and specific with this. Click-through rate curves are imperfect by nature because we truly can’t measure exactly how those things change. However, I think smart marketers can make some good assumptions from general click-through rate data, which there are several resources out there on that to build a model like this and then include it in their keyword research.

This does mean that you have to run a query for every keyword you’re thinking about, but you should be doing that anyway. You want to get a good look at who’s ranking in those search results and what kind of content they’re building . If you’re running a keyword difficulty tool, you are already getting something like that.

Business value

This is a classic one. Business value is essentially saying, “What’s it worth to us if visitors come through with this search term?” You can get that from bidding through AdWords. That’s the most sort of scientific, mathematically sound way to get it. Then, of course, you can also get it through your own intuition. It’s better to start with your intuition than nothing if you don’t already have AdWords data or you haven’t started bidding, and then you can refine your sort of estimate over time as you see search visitors visit the pages that are ranking, as you potentially buy those ads, and those kinds of things.

You can get more sophisticated around this. I think a 10 point scale is just fine. You could also use a one, two, or three there, that’s also fine.

Requirements or Options

Then I don’t exactly know what to call this column. I can’t remember the person who’ve showed me theirs that had it in there. I think they called it Optional Data or Additional SERPs Data, but I’m going to call it Requirements or Options. Requirements because this is essentially saying, “Hey, if I want to rank in these search results, am I seeing that the top two or three are all video? Oh, they’re all video. They’re all coming from YouTube. If I want to be in there, I’ve got to be video.”

Or something like, “Hey, I’m seeing that most of the top results have been produced or updated in the last six months. Google appears to be biasing to very fresh information here.” So, for example, if I were searching for “university scholarships Cambridge 2015,” well, guess what? Google probably wants to bias to show results that have been either from the official page on Cambridge’s website or articles from this year about getting into that university and the scholarships that are available or offered. I saw those in two of these search results, both the college and university scholarships had a significant number of the SERPs where a fresh bump appeared to be required. You can see that a lot because the date will be shown ahead of the description, and the date will be very fresh, sometime in the last six months or a year.

Prioritization

Then finally I can build my prioritization. So based on all the data I had here, I essentially said, “Hey, you know what? These are not 1 and 2. This is actually 1A and 1B, because these are the same concepts. I’m going to build a single page to target both of those keyword phrases.” I think that makes good sense. Someone who is looking for college scholarships, university scholarships, same intent.

I am giving it a slight prioritization, 1A versus 1B, and the reason I do this is because I always have one keyword phrase that I’m leaning on a little more heavily. Because Google isn’t perfect around this, the search results will be a little different. I want to bias to one versus the other. In this case, my title tag, since I more targeting university over college, I might say something like college and university scholarships so that university and scholarships are nicely together, near the front of the title, that kind of thing. Then 1B, 2, 3.

This is kind of the way that modern SEOs are building a more sophisticated process with better data, more inclusive data that helps them select the right kinds of keywords and prioritize to the right ones. I’m sure you guys have built some awesome stuff. The Moz community is filled with very advanced marketers, probably plenty of you who’ve done even more than this.

I look forward to hearing from you in the comments. I would love to chat more about this topic, and we’ll see you again next week for another edition of Whiteboard Friday. Take care.

Video transcription by Speechpad.com

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 3 years ago from tracking.feedpress.it

Deconstructing the App Store Rankings Formula with a Little Mad Science

Posted by AlexApptentive

After seeing Rand’s “Mad Science Experiments in SEO” presented at last year’s MozCon, I was inspired to put on the lab coat and goggles and do a few experiments of my own—not in SEO, but in SEO’s up-and-coming younger sister, ASO (app store optimization).

Working with Apptentive to guide enterprise apps and small startup apps alike to increase their discoverability in the app stores, I’ve learned a thing or two about app store optimization and what goes into an app’s ranking. It’s been my personal goal for some time now to pull back the curtains on Google and Apple. Yet, the deeper into the rabbit hole I go, the more untested assumptions I leave in my way.

Hence, I thought it was due time to put some longstanding hypotheses through the gauntlet.

As SEOs, we know how much of an impact a single ranking can mean on a SERP. One tiny rank up or down can make all the difference when it comes to your website’s traffic—and revenue.

In the world of apps, ranking is just as important when it comes to standing out in a sea of more than 1.3 million apps. Apptentive’s recent mobile consumer survey shed a little more light this claim, revealing that nearly half of all mobile app users identified browsing the app store charts and search results (the placement on either of which depends on rankings) as a preferred method for finding new apps in the app stores. Simply put, better rankings mean more downloads and easier discovery.

Like Google and Bing, the two leading app stores (the Apple App Store and Google Play) have a complex and highly guarded algorithms for determining rankings for both keyword-based app store searches and composite top charts.

Unlike SEO, however, very little research and theory has been conducted around what goes into these rankings.

Until now, that is.

Over the course of five studies analyzing various publicly available data points for a cross-section of the top 500 iOS (U.S. Apple App Store) and the top 500 Android (U.S. Google Play) apps, I’ll attempt to set the record straight with a little myth-busting around ASO. In the process, I hope to assess and quantify any perceived correlations between app store ranks, ranking volatility, and a few of the factors commonly thought of as influential to an app’s ranking.

But first, a little context

Image credit: Josh Tuininga, Apptentive

Both the Apple App Store and Google Play have roughly 1.3 million apps each, and both stores feature a similar breakdown by app category. Apps ranking in the two stores should, theoretically, be on a fairly level playing field in terms of search volume and competition.

Of these apps, nearly two-thirds have not received a single rating and 99% are considered unprofitable. These studies, therefore, single out the rare exceptions to the rule—the top 500 ranked apps in each store.

While neither Apple nor Google have revealed specifics about how they calculate search rankings, it is generally accepted that both app store algorithms factor in:

  • Average app store rating
  • Rating/review volume
  • Download and install counts
  • Uninstalls (what retention and churn look like for the app)
  • App usage statistics (how engaged an app’s users are and how frequently they launch the app)
  • Growth trends weighted toward recency (how daily download counts changed over time and how today’s ratings compare to last week’s)
  • Keyword density of the app’s landing page (Ian did a great job covering this factor in a previous Moz post)

I’ve simplified this formula to a function highlighting the four elements with sufficient data (or at least proxy data) for our analysis:

Ranking = fn(Rating, Rating Count, Installs, Trends)

Of course, right now, this generalized function doesn’t say much. Over the next five studies, however, we’ll revisit this function before ultimately attempting to compare the weights of each of these four variables on app store rankings.

(For the purpose of brevity, I’ll stop here with the assumptions, but I’ve gone into far greater depth into how I’ve reached these conclusions in a 55-page report on app store rankings.)

Now, for the Mad Science.

Study #1: App-les to app-les app store ranking volatility

The first, and most straight forward of the five studies involves tracking daily movement in app store rankings across iOS and Android versions of the same apps to determine any trends of differences between ranking volatility in the two stores.

I went with a small sample of five apps for this study, the only criteria for which were that:

  • They were all apps I actively use (a criterion for coming up with the five apps but not one that influences rank in the U.S. app stores)
  • They were ranked in the top 500 (but not the top 25, as I assumed app store rankings would be stickier at the top—an assumption I’ll test in study #2)
  • They had an almost identical version of the app in both Google Play and the App Store, meaning they should (theoretically) rank similarly
  • They covered a spectrum of app categories

The apps I ultimately chose were Lyft, Venmo, Duolingo, Chase Mobile, and LinkedIn. These five apps represent the travel, finance, education banking, and social networking categories.

Hypothesis

Going into this analysis, I predicted slightly more volatility in Apple App Store rankings, based on two statistics:

Both of these assumptions will be tested in later analysis.

Results

7-Day App Store Ranking Volatility in the App Store and Google Play

Among these five apps, Google Play rankings were, indeed, significantly less volatile than App Store rankings. Among the 35 data points recorded, rankings within Google Play moved by as much as 23 positions/ranks per day while App Store rankings moved up to 89 positions/ranks. The standard deviation of ranking volatility in the App Store was, furthermore, 4.45 times greater than that of Google Play.

Of course, the same apps varied fairly dramatically in their rankings in the two app stores, so I then standardized the ranking volatility in terms of percent change to control for the effect of numeric rank on volatility. When cast in this light, App Store rankings changed by as much as 72% within a 24-hour period while Google Play rankings changed by no more than 9%.

Also of note, daily rankings tended to move in the same direction across the two app stores approximately two-thirds of the time, suggesting that the two stores, and their customers, may have more in common than we think.

Study #2: App store ranking volatility across the top charts

Testing the assumption implicit in standardizing the data in study No. 1, this one was designed to see if app store ranking volatility is correlated with an app’s current rank. The sample for this study consisted of the top 500 ranked apps in both Google Play and the App Store, with special attention given to those on both ends of the spectrum (ranks 1–100 and 401–500).

Hypothesis

I anticipated rankings to be more volatile the higher an app is ranked—meaning an app ranked No. 450 should be able to move more ranks in any given day than an app ranked No. 50. This hypothesis is based on the assumption that higher ranked apps have more installs, active users, and ratings, and that it would take a large margin to produce a noticeable shift in any of these factors.

Results

App Store Ranking Volatility of Top 500 Apps

One look at the chart above shows that apps in both stores have increasingly more volatile rankings (based on how many ranks they moved in the last 24 hours) the lower on the list they’re ranked.

This is particularly true when comparing either end of the spectrum—with a seemingly straight volatility line among Google Play’s Top 100 apps and very few blips within the App Store’s Top 100. Compare this section to the lower end, ranks 401–)500, where both stores experience much more turbulence in their rankings. Across the gamut, I found a 24% correlation between rank and ranking volatility in the Play Store and 28% correlation in the App Store.

To put this into perspective, the average app in Google Play’s 401–)500 ranks moved 12.1 ranks in the last 24 hours while the average app in the Top 100 moved a mere 1.4 ranks. For the App Store, these numbers were 64.28 and 11.26, making slightly lower-ranked apps more than five times as volatile as the highest ranked apps. (I say slightly as these “lower-ranked” apps are still ranked higher than 99.96% of all apps.)

The relationship between rank and volatility is pretty consistent across the App Store charts, while rank has a much greater impact on volatility at the lower end of Google Play charts (ranks 1-100 have a 35% correlation) than it does at the upper end (ranks 401-500 have a 1% correlation).

Study #3: App store rankings across the stars

The next study looks at the relationship between rank and star ratings to determine any trends that set the top chart apps apart from the rest and explore any ties to app store ranking volatility.

Hypothesis

Ranking = fn(Rating, Rating Count, Installs, Trends)

As discussed in the introduction, this study relates directly to one of the factors commonly accepted as influential to app store rankings: average rating.

Getting started, I hypothesized that higher ranks generally correspond to higher ratings, cementing the role of star ratings in the ranking algorithm.

As far as volatility goes, I did not anticipate average rating to play a role in app store ranking volatility, as I saw no reason for higher rated apps to be less volatile than lower rated apps, or vice versa. Instead, I believed volatility to be tied to rating volume (as we’ll explore in our last study).

Results

Average App Store Ratings of Top Apps

The chart above plots the top 100 ranked apps in either store with their average rating (both historic and current, for App Store apps). If it looks a little chaotic, it’s just one indicator of the complexity of ranking algorithm in Google Play and the App Store.

If our hypothesis was correct, we’d see a downward trend in ratings. We’d expect to see the No. 1 ranked app with a significantly higher rating than the No. 100 ranked app. Yet, in neither store is this the case. Instead, we get a seemingly random plot with no obvious trends that jump off the chart.

A closer examination, in tandem with what we already know about the app stores, reveals two other interesting points:

  1. The average star rating of the top 100 apps is significantly higher than that of the average app. Across the top charts, the average rating of a top 100 Android app was 4.319 and the average top iOS app was 3.935. These ratings are 0.32 and 0.27 points, respectively, above the average rating of all rated apps in either store. The averages across apps in the 401–)500 ranks approximately split the difference between the ratings of the top ranked apps and the ratings of the average app.
  2. The rating distribution of top apps in Google Play was considerably more compact than the distribution of top iOS apps. The standard deviation of ratings in the Apple App Store top chart was over 2.5 times greater than that of the Google Play top chart, likely meaning that ratings are more heavily weighted in Google Play’s algorithm.

App Store Ranking Volatility and Average Rating

Looking next at the relationship between ratings and app store ranking volatility reveals a -15% correlation that is consistent across both app stores; meaning the higher an app is rated, the less its rank it likely to move in a 24-hour period. The exception to this rule is the Apple App Store’s calculation of an app’s current rating, for which I did not find a statistically significant correlation.

Study #4: App store rankings across versions

This next study looks at the relationship between the age of an app’s current version, its rank and its ranking volatility.

Hypothesis

Ranking = fn(Rating, Rating Count, Installs, Trends)

In alteration of the above function, I’m using the age of a current app’s version as a proxy (albeit not a very good one) for trends in app store ratings and app quality over time.

Making the assumptions that (a) apps that are updated more frequently are of higher quality and (b) each new update inspires a new wave of installs and ratings, I’m hypothesizing that the older the age of an app’s current version, the lower it will be ranked and the less volatile its rank will be.

Results

How update frequency correlates with app store rank

The first and possibly most important finding is that apps across the top charts in both Google Play and the App Store are updated remarkably often as compared to the average app.

At the time of conducting the study, the current version of the average iOS app on the top chart was only 28 days old; the current version of the average Android app was 38 days old.

As hypothesized, the age of the current version is negatively correlated with the app’s rank, with a 13% correlation in Google Play and a 10% correlation in the App Store.

How update frequency correlates with app store ranking volatility

The next part of the study maps the age of the current app version to its app store ranking volatility, finding that recently updated Android apps have less volatile rankings (correlation: 8.7%) while recently updated iOS apps have more volatile rankings (correlation: -3%).

Study #5: App store rankings across monthly active users

In the final study, I wanted to examine the role of an app’s popularity on its ranking. In an ideal world, popularity would be measured by an app’s monthly active users (MAUs), but since few mobile app developers have released this information, I’ve settled for two publicly available proxies: Rating Count and Installs.

Hypothesis

Ranking = fn(Rating, Rating Count, Installs, Trends)

For the same reasons indicated in the second study, I anticipated that more popular apps (e.g., apps with more ratings and more installs) would be higher ranked and less volatile in rank. This, again, takes into consideration that it takes more of a shift to produce a noticeable impact in average rating or any of the other commonly accepted influencers of an app’s ranking.

Results

Apps with more ratings and reviews typically rank higher

The first finding leaps straight off of the chart above: Android apps have been rated more times than iOS apps, 15.8x more, in fact.

The average app in Google Play’s Top 100 had a whopping 3.1 million ratings while the average app in the Apple App Store’s Top 100 had 196,000 ratings. In contrast, apps in the 401–)500 ranks (still tremendously successful apps in the 99.96 percentile of all apps) tended to have between one-tenth (Android) and one-fifth (iOS) of the ratings count as that of those apps in the top 100 ranks.

Considering that almost two-thirds of apps don’t have a single rating, reaching rating counts this high is a huge feat, and a very strong indicator of the influence of rating count in the app store ranking algorithms.

To even out the playing field a bit and help us visualize any correlation between ratings and rankings (and to give more credit to the still-staggering 196k ratings for the average top ranked iOS app), I’ve applied a logarithmic scale to the chart above:

The relationship between app store ratings and rankings in the top 100 apps

From this chart, we can see a correlation between ratings and rankings, such that apps with more ratings tend to rank higher. This equates to a 29% correlation in the App Store and a 40% correlation in Google Play.

Apps with more ratings typically experience less app store ranking volatility

Next up, I looked at how ratings count influenced app store ranking volatility, finding that apps with more ratings had less volatile rankings in the Apple App Store (correlation: 17%). No conclusive evidence was found within the Top 100 Google Play apps.

Apps with more installs and active users tend to rank higher in the app stores

And last but not least, I looked at install counts as an additional proxy for MAUs. (Sadly, this is a statistic only listed in Google Play. so any resulting conclusions are applicable only to Android apps.)

Among the top 100 Android apps, this last study found that installs were heavily correlated with ranks (correlation: -35.5%), meaning that apps with more installs are likely to rank higher in Google Play. Android apps with more installs also tended to have less volatile app store rankings, with a correlation of -16.5%.

Unfortunately, these numbers are slightly skewed as Google Play only provides install counts in broad ranges (e.g., 500k–)1M). For each app, I took the low end of the range, meaning we can likely expect the correlation to be a little stronger since the low end was further away from the midpoint for apps with more installs.

Summary

To make a long post ever so slightly shorter, here are the nuts and bolts unearthed in these five mad science studies in app store optimization:

  1. Across the top charts, Apple App Store rankings are 4.45x more volatile than those of Google Play
  2. Rankings become increasingly volatile the lower an app is ranked. This is particularly true across the Apple App Store’s top charts.
  3. In both stores, higher ranked apps tend to have an app store ratings count that far exceeds that of the average app.
  4. Ratings appear to matter more to the Google Play algorithm, especially as the Apple App Store top charts experience a much wider ratings distribution than that of Google Play’s top charts.
  5. The higher an app is rated, the less volatile its rankings are.
  6. The 100 highest ranked apps in either store are updated much more frequently than the average app, and apps with older current versions are correlated with lower ratings.
  7. An app’s update frequency is negatively correlated with Google Play’s ranking volatility but positively correlated with ranking volatility in the App Store. This likely due to how Apple weighs an app’s most recent ratings and reviews.
  8. The highest ranked Google Play apps receive, on average, 15.8x more ratings than the highest ranked App Store apps.
  9. In both stores, apps that fall under the 401–500 ranks receive, on average, 10–20% of the rating volume seen by apps in the top 100.
  10. Rating volume and, by extension, installs or MAUs, is perhaps the best indicator of ranks, with a 29–40% correlation between the two.

Revisiting our first (albeit oversimplified) guess at the app stores’ ranking algorithm gives us this loosely defined function:

Ranking = fn(Rating, Rating Count, Installs, Trends)

I’d now re-write the function into a formula by weighing each of these four factors, where a, b, c, & d are unknown multipliers, or weights:

Ranking = (Rating * a) + (Rating Count * b) + (Installs * c) + (Trends * d)

These five studies on ASO shed a little more light on these multipliers, showing Rating Count to have the strongest correlation with rank, followed closely by Installs, in either app store.

It’s with the other two factors—rating and trends—that the two stores show the greatest discrepancy. I’d hazard a guess to say that the App Store prioritizes growth trends over ratings, given the importance it places on an app’s current version and the wide distribution of ratings across the top charts. Google Play, on the other hand, seems to favor ratings, with an unwritten rule that apps just about have to have at least four stars to make the top 100 ranks.

Thus, we conclude our mad science with this final glimpse into what it takes to make the top charts in either store:

Weight of factors in the Apple App Store ranking algorithm

Rating Count > Installs > Trends > Rating

Weight of factors in the Google Play ranking algorithm

Rating Count > Installs > Rating > Trends


Again, we’re oversimplifying for the sake of keeping this post to a mere 3,000 words, but additional factors including keyword density and in-app engagement statistics continue to be strong indicators of ranks. They simply lie outside the scope of these studies.

I hope you found this deep-dive both helpful and interesting. Moving forward, I also hope to see ASOs conducting the same experiments that have brought SEO to the center stage, and encourage you to enhance or refute these findings with your own ASO mad science experiments.

Please share your thoughts in the comments below, and let’s deconstruct the ranking formula together, one experiment at a time.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 3 years ago from tracking.feedpress.it

The Long Click and the Quality of Search Success

Posted by billslawski

“On the most basic level, Google could see how satisfied users were. To paraphrase Tolstoy, happy users were all the same. The best sign of their happiness was the “Long Click” — This occurred when someone went to a search result, ideally the top one, and did not return. That meant Google has successfully fulfilled the query.”

~ Steven Levy. In the Plex: How Google Thinks, Works, and Shapes our Lives

I often explore and read patents and papers from the search engines to try to get a sense of how they may approach different issues, and learn about the assumptions they make about search, searchers, and the Web. Lately, I’ve been keeping an eye open for papers and patents from the search engines where they talk about a metric known as the “long click.”

A recently granted Google patent uses the metric of a “Long Click” as the center of a process Google may use to track results for queries that were selected by searchers for long visits in a set of search results.

This concept isn’t new. In 2011, I wrote about a Yahoo patent in How a Search Engine May Measure the Quality of Its Search Results, where they discussed a metric that they refer to as a “target page success metric.” It included “dwell time” upon a result as a sign of search success (Yes, search engines have goals, too).

5543947f5bb408.24541747.jpg

Another Google patent described assigning web pages “reachability scores” based upon the quality of pages linked to from those initially visited pages. In the post Does Google Use Reachability Scores in Ranking Resources? I described how a Google patent that might view a long click metric as a sign to see if visitors to that page are engaged by the links to content they find those links pointing to, including links to videos. Google tells us in that patent that it might consider a “long click” to have been made on a video if someone watches at least half the video or 30 seconds of it. The patent suggests that a high reachability score on a page may mean that page could be boosted in Google search results.

554394a877e8c8.30299132.jpg

But the patent I’m writing about today is focused primarily upon looking at and tracking a search success metric like a long click or long dwell time. Here’s the abstract:

Modifying ranking data based on document changes

Invented by Henele I. Adams, and Hyung-Jin Kim

Assigned to Google

US Patent 9,002,867

Granted April 7, 2015

Filed: December 30, 2010

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media for determining a weighted overall quality of result statistic for a document.

One method includes receiving quality of result data for a query and a plurality of versions of a document, determining a weighted overall quality of result statistic for the document with respect to the query including weighting each version specific quality of result statistic and combining the weighted version-specific quality of result statistics, wherein each quality of result statistic is weighted by a weight determined from at least a difference between content of a reference version of the document and content of the version of the document corresponding to the version specific quality of result statistic, and storing the weighted overall quality of result statistic and data associating the query and the document with the weighted overall quality of result statistic.

This patent tells us that search results may be be ranked in an order, according to scores assigned to the search results by a scoring function or process that would be based upon things such as:

  • Where, and how often, query terms appear in the given document,
  • How common the query terms are in the documents indexed by the search engine, or
  • A query-independent measure of quality of the document itself.

Last September, I wrote about how Google might identify a category associated with a query term base upon clicks, in the post Using Query User Data To Classify Queries. In a query for “Lincoln.” the results that appear in response might be about the former US President, the town of Lincoln, Nebraska, and the model of automobile. When someone searches for [Lincoln], Google returning all three of those responses as a top result could be said to be reasonable. The patent I wrote about in that post told us that Google might collect information about “Lincoln” as a search entity, and track which category of results people clicked upon most when they performed that search, to determine what categories of pages to show other searchers. Again, that’s another “search success” based upon a past search history.

There likely is some value in working to find ways to increase the amount of dwell time someone spends upon the pages of your site, if you are already having some success in crafting page titles and snippets that persuade people to click on your pages when they those appear in search results. These approaches can include such things as:

  1. Making visiting your page a positive experience in terms of things like site speed, readability, and scannability.
  2. Making visiting your page a positive experience in terms of things like the quality of the content published on your pages including spelling, grammar, writing style, interest, quality of images, and the links you share to other resources.
  3. Providing a positive experience by offering ideas worth sharing with others, and offering opportunities for commenting and interacting with others, and by being responsive to people who do leave comments.

Here are some resources I found that discuss this long click metric in terms of “dwell time”:

Your ability to create pages that can end up in a “long click” from someone who has come to your site in response to a query, is also a “search success” metric on the search engine’s part, and you both succeed. Just be warned that as the most recent patent from Google on Long Clicks shows us, Google will be watching to make sure that the content of your page doesn’t change too much, and that people are continuing to click upon it in search results, and spend a fair amount to time upon it.

(Images for this post are from my Go Fish Digital Design Lead Devin Holmes @DevinGoFish. Thank you, Devin!)

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 4 years ago from tracking.feedpress.it

Try Your Hand at A/B Testing for a Chance to Win the Email Subject Line Contest

Posted by danielburstein

This blog post ends with an opportunity for you to win a stay at the ARIA in Vegas and a ticket to
Email Summit, but it begins with an essential question for marketers…

How can you improve already successful marketing, advertising, websites and copywriting?

Today’s Moz blog post is unique. Not only are we going to teach you how to address this challenge, we’re going to offer an example that you can dig into to help drive home the lesson.

Give the people what they want

Some copy and design is so bad, the fixes are obvious. Maybe you shouldn’t insult the customer in the headline. Maybe you should update the website that still uses a dot matrix font.

But when you’re already doing well, how can you continue to improve?

I don’t have the answer for you, but I’ll tell you who does – your customers.

There are many tricks, gimmicks and technology you can use in marketing, but when you strip away all the hype and rhetoric, successful marketing is pretty straightforward –
clearly communicate the value your offer provides to people who will pay you for that value.

Easier said than done, of course.

So how do you determine what customers want? And the best way to deliver it to them?

Well, there are many ways to learn from customers, such as focus groups, surveys and social listening. While there is value in asking people what they want, there is also a major challenge in it. “People’s ability to understand the factors that affect their behavior is surprisingly poor,” according to research from Dr. Noah J. Goldstein, Associate Professor of Management and Organizations, UCLA Anderson School of Management.

Or, as Malcolm Gladwell more glibly puts it when referring to coffee choices, “The mind knows not what the tongue wants.”

Not to say that opinion-based customer preference research is bad. It can be helpful. However, it should be the beginning and not the end of your quest.

…by seeing what they actually do

You can use what you learn from opinion-based research to create a hypothesis about what customers want, and then
run an experiment to see how they actually behave in real-world customer interactions with your product, marketing messages, and website.

The technique that powers this kind of research is often known as A/B testing, split testing, landing page optimization, and/or website optimization. If you are testing more than one thing at a time, it may also be referred to as multi-variate testing.

To offer a simple example, you might assume that customers buy your product because it tastes great. Or because it’s less filling. So you could create two landing pages – one with a headline that promotes that taste (treatment A) and another that mentions the low carbs (treatment B). You then send half the traffic that visits that URL to each version and see which performs better.

Here is a simple visual that Joey Taravella, Content Writer, MECLABS create to illustrate the concept…

That’s just one test. To really learn about your customers, you must continue the process and create a testing-optimization cycle in your organization – continue to run A/B tests, record the findings, learn from them, create more hypotheses, and test again based on these hypotheses.

This is true marketing experimentation, and helps you build your theory of the customer.

But you probably know all that already. So here’s your chance to practice while helping us shape an A/B test. You might even win a prize in the process.

The email subject line contest

The Moz Blog and MarketingExperiments Blog have joined forces to run a unique marketing experimentation contest. We’re presenting you with a real challenge from a real organization (VolunteerMatch) and
asking you to write a subject line to test (it’s simple, just leave your subject line as a comment in this blog post).

We’re going to pick three subject lines suggested by readers of The Moz Blog and three from the MarketingExperiments Blog and run a test with this organization’s customers. Whoever writes the best performing subject line will
win a stay at the ARIA Resort in Las Vegas as well as a two-day ticket to MarketingSherpa Email Summit 2015 to help them gain lessons to further improve their marketing.

Sound good? OK, let’s dive in and tell you more about your “client”…

Craft the best-performing subject line to win the prize

Every year at Email Summit, we run a live A/B test where the audience helps craft the experiment. We then run, validate, close the experiment, and share the results during Summit as a way to teach about marketing experimentation. We have typically run the experiment using MarketingSherpa as the “client” website to test (MarketingExperiments and MarketingSherpa are sister publications, both owned by MECLABS Institute).

However, this year we wanted to try something different and interviewed three national non-profits to find a new “client” for our tests.

We chose
VolunteerMatch – a nonprofit organization that uses the power of technology to make it easier for good people and good causes to connect. One of the key reasons we chose VolunteerMatch is because it is an already successful organization looking to further improve. (Here is a case study explaining one of its successful implementations – Lead Management: How a B2B SaaS nonprofit decreased its sales cycle 99%).

Another reason we chose VolunteerMatch for this opportunity is that it has three types of customers, so the lessons from the content we create can help marketers across a wide range of sales models. VolunteerMatch’s customers are:

  • People who want to volunteer (B2C)
  • Non-profit organizations looking for volunteers (non-profit)
  • Businesses looking for corporate volunteering solutions (B2B) to which it offers a Software-as-a-Service product through VolunteerMatch Solutions

Designing the experiment

After we took VolunteerMatch on as the Research Partner “client,” Jon Powell, Senior Executive Research and Development Manager, MECLABS, worked with Shari Tishman, Director of Engagement and Lauren Wagner, Senior Manager of Engagement, VolunteerMatch, to understand their challenges, take a look at their current assets and performance, and craft a design of experiments to determine what further knowledge about its customers would help VolunteerMatch improve performance.

That design of experiments includes a series of split tests – including the live test we’re going to run at Email Summit, as well as the one you have an opportunity to take part in by writing a subject line in the comments section of this blog post. Let’s take a look at that experiment…

The challenge

VolunteerMatch wants to increase the response rate of the corporate email list (B2B) by discovering the best possible messaging to use. In order to find out, MarketingExperiments wants to run an A/B split test to determine the
best messaging.

However the B2B list is relatively smaller than the volunteer/cause list (B2C) which makes it harder to test in (and gain
statistical significance) and determine which messaging is most effective.

So we’re going to run a messaging test to the B2C list. This isn’t without its challenges though, because most individuals on the B2C list are not likely to immediately connect with B2B corporate solutions messaging.

So the question is…

How do we create an email that is relevant (to the B2C list), which doesn’t ask too much, that simultaneously helps us discover the most relevant aspect of the solutions (B2B) product (if any)?

The approach – Here’s where you come in

This is where the Moz and MarketingExperiments community comes in to help.

We would like you to craft subject lines relevant to the B2C list, which highlight various benefits of the corporate solutions tool.

We have broken down the corporate solutions tool into three main categories of benefit for the SaaS product.
In the comments section below, include which category you are writing a subject line for along with what you think is an effective subject line.

The crew at Moz and MarketingExperiments will then choose the top subject line in each category to test. Below you will find the emails that will be sent as part of the test. They are identical, except for the subject lines (which you will write) and the bolded line in the third paragraph (that ties into that category of value).

Category #1: Proof, recognition, credibility


Category #2: Better, more opportunities to choose from


Category #3: Ease-of-use

About VolunteerMatch’s brand

Since we’re asking you to try your hand at crafting messaging for this example “client,” here is some more information about the brand to inform your messaging…


VolunteerMatch’s brand identity


VolunteerMatch’s core values

Ten things VolunteerMatch believes:

  1. People want to do good
  2. Every great cause should be able to find the help it needs
  3. People want to improve their lives and communities through volunteering
  4. You can’t make a difference without making a connection
  5. In putting the power of technology to good use
  6. Businesses are serious about making a difference
  7. In building relationships based on trust and excellent service
  8. In partnering with like-minded organizations to create systems that result in even greater impact
  9. The passion of our employees drives the success of our products, services and mission
  10. In being great at what we do

And now, we test…

To participate, you must leave your comment with your idea for a subject line before midnight on Tuesday, January 13, 2015. The contest is open to all residents of the 50 US states, the District of Columbia, and Canada (excluding Quebec), 18 or older. If you want more info, here are the
official rules.

When you enter your subject line in the comments section, also include which category you’re entering for (and if you have an idea outside these categories, let us know…we just might drop it in the test).

Next, the Moz marketing team will pick the subject lines they think will perform best in each category from all the comments on The Moz Blog, and the MarketingExperiments team will pick the subject lines we think will perform the best in each category from all the comments on the MarketingExperiments Blog.

We’ll give the VolunteerMatch team a chance to approve the subject lines based on their brand standards, then test all six to eight subject lines and report back to you through the Moz and MarketingExperiments blogs which subject lines won and why they won to help you improve your already successful marketing.

So, what have you got? Write your best subject lines in the comments section below. I look forward to seeing what you come up with.

Related resources

If you’re interested in learning more about marketing experimentation and A/B testing, you might find these links helpful…

And here’s a look at a previous subject line writing contest we’ve run to give you some ideas for your entry…


Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 4 years ago from moz.com

International SEO Study: How Searchers Perceive Country Code Top-Level Domains

Posted by 5le

The decision to focus your site on an international audience is a big step and one fraught with complexities. There are, of course, issues to deal with around language and user experience, but in addition there are some big technical choices to make including what domains to use.

Any authoritative
international SEO guide will elaborate on the differences between the options of subdirectory, subdomain, and country-code top level domain (CCTLD). One of the most common suggestions is for a site to opt to use a ccTLD (e.g. domain.co.uk) as the domain extension. The reasoning behind this is the theory that the ccTLD extension will “hint” to search engines and users exactly who your target audience should be versus the other, less explicit options. For example, a search engine and human user would know, even without clicking into a site, that a site that ends with .co.uk is targeting a user looking for UK content. 

We have solid data from
Google that a ccTLD does indicate country targeting; however, when it comes to users there is only an assumption that users even notice and make choices based on the ccTLD. However, this is a fairly broad assumption that doesn’t address whether a ccTLD is more important than a brand name in the domain or the quality of a website’s content. To test this theory, we ran a survey to discover what users really thought.

User knowledge of TLDs

Even before trying to understand how users related to ccTLDs it is essential to validate the assumption that users even know that general TLDs exist. To establish this fact, we asked respondents to pick which TLD might be the one in use by a non-profit. Close to
100% of respondents correctly identified a TLD ending with .org as the one most likely to be used by a non-profit. Interestingly, only 4% of people in the US stated that they were unsure of the correct TLD compared to 13% of Australians. Predictably, nearly all marketers (98%) chose the .org answer.

Another popular TLD is the .edu in use by educational assumptions, and we wanted to understand if users thought that content coming from a .edu domain might be more trustworthy. We asked users if they received an unsolicited email about water quality in their town whether they would place more trust in a sender’s email address that ended with .edu or .com.
89% of respondents in the US chose the .edu as more trustworthy, while only 79% said the same in Australia. Quite interestingly, the marketer responses (from the survey posted on Inbound.org were exactly the same as the Australians with 79% declaring the .edu to be more trustworthy.

.org cctld survey australia

If users can identify a .org as the correct TLD for a non-profit, and a .edu as a TLD that might be more trustworthy, it is likely that users are familiar with the existence of TLDs and how they might be used. The next question to answer is if users are aware of the connection between TLDs and locations.

Country relationship awareness

Next, we asked respondents to identify the location of a local business using a .ca TLD extension. The majority of respondents across all three surveys correctly chose Canada; and nearly all marketers (92%) got this correct. Oddly, more Australians (67%) correctly identified Canada than Americans (62%). We would have thought Americans should have been more familiar with the TLD of a neighboring country. Additionally, more Americans (23%) fell for the trick answer of California than Australians (15%). Regardless, we were able to conclude that most Internet users are aware of TLDs and that they are tied to a specific country.

canada cctld survey

To really gauge how much users know about TLDs and countries, we asked users to pick the right domain extension for a website in another country. In the US survey, we asked users to pick the correct TLD for an Australian company, and in the Australian survey we used a British company. In each of the questions we gave one correct answer possibility, one almost correct, and two entire wrong choices.For example, we gave .co.uk and .uk as answer choices to Australians.

In both the US and Australia, the majority of respondents chose the correct TLD, although Americans seem to have been confused by whether Australia’s TLD was .AU (35%) or .com.AU (24%).

There is a common practice of using country-code domain extensions as a vanity URL for content that is not geotargeted. For example, .ly is the domain extension for Libya, but it is frequently used on domains that have a word that ends with “ly.” Additionally, .me is the domain extension for Montenegro; however, the TLD is used for many purposes other than Montenegro content.

We wanted to understand if users noticed this type of TLD usage or if they thought the content might still be related to another country. We asked respondents what might be on a website that ended with .TV which is the TLD for the island nation of Tuvalu and is also a popular TLD for TV show websites. 51% of US respondents thought it might be a TV show and 42% chose the “it could be anything” answer. In Australia, 43% thought the site would be a TV show, and 44% said “it could be anything”.

tuvalu cctld survey

One of the answer options was that it could be a website in Tuvalu and interestingly twice as many Australian (9%) chose this option vs US respondents (4.5%). This question was one of the areas where marketers’ answers were very different from those in the US and Australia. 77% of marketers chose the TV show option and only 19% said it could be anything.

Based on the these three results, it is apparent that
users recognize TLDs, know that they are from other countries, and appear to make some judgments around the content based on the TLD.

Decision making using TLDs

Since users know that TLDs are an important part of a URL that is tied to a country of origin, it is important to understand how the TLD factors into their decision-making processes about whether or not they visit certain websites.

We asked users whether they thought medical content on a foreign TLD would be as reliable as similar content found on their local TLD. In the US, only 24% thought the content on the non-local TLD (.co.uk) was less reliable than content on a .com. In Australia, the results were nearly identical to what we saw in the US with only 28% answering that the non-local TLD (.co.uk) was less reliable than the content on a .com.au. Even 24% of marketers answered that the content was less reliable. The remaining respondents chose either that the content equally reliable or they just didn’t know. Based on these results, the TLD (at least as long as it was a reputable one)
does not seem to impact user trust.

UK cctld survey

Digging into the idea of trust and TLD a bit further, we asked the same reliability question about results on Google.com vs Google.de. In the US, 56% of respondents said that the results on Google.de are equally reliable to those on Google.com, and in Australia, 51% said the same thing when compared to Google.com.au. In the marketer survey, 66% of respondents said the results were equally reliable. The fact that the majority of respondents stated that results are equally reliable should mean that users are more focused on the brand portion of a domain rather than its country extension.

CcTLD’s impact on ecommerce

Making the decision to use a ccTLD on a website can be costly, so it is important to justify this cost with an actual revenue benefit. Therefore the real test of TLD choice is how it impacts revenue. This type of answer is of course hard to gauge in a survey where customers are not actually buying products, but we did want to try to see if there might be a way to measure purchasing decisions.

To achieve this result, we compared two different online retailers and asked respondents to choose the establishment that they thought would have the most reliable express shipping. In the US survey, we compared Amazon.co.jp to BestBuy.com. In the Australian survey, we compared Bigw.com.au (a well known online retailer) to Target.com. (Interesting fact: there is a Target in Australia that is not affiliated with Target in the US and their website is target.com.au) The intent of the question was to see if users zeroed in on the recognizable brand name or the domain extension.

cctld trust survey

In the US, while 39% said that both websites would offer reliable shipping, 42% still said that Best Buy would be the better option. Australians may have been confused by the incorrect Target website, since 61% said both websites would have reliable shipping, but 34% chose Big W. Even marketers didn’t seem oblivious to domain names with only 34% choosing the equally reliable option, and 49% choosing Best Buy. The data in this question is a bit inconclusive, but we can definitively say that while a large portion of users are blind to domain names, however, when selling online it would be best to use a familiar domain extension.

cctld trust survey australia

New TLDs

Late last year, ICANN (the Internet governing body) announced that they would be releasing dozens of new
GTLDs, which opened up a new domain name land grab harkening back to the early days of the Internet. Many of these domain names can be quite expensive, and we wanted to discover whether they even mattered to users.

gtld survey

We asked users if, based solely on the domain name, they were more likely to trust an insurance quote from a website ending in .insurance.
62% of Americans, 53% of Australians, and 67% of marketers said they were unlikely to trust the quote based on the domain alone. Based on this result, if you’re looking to invest in a new TLD simply to drive more conversions, you should probably do more research first. 

A new gTLD is probably not a silver bullet.

Methodology

For this survey, I collaborated with
Sam Mallikarjunan at HubSpot and we decided that the two assumptions we absolutely needed to validate where 1) whether users even notice ccTLDs and 2) if so do they really prefer the TLD of their country. While we received 101 responses from a version of the survey targeted at marketers on an Inbound.org discussion, we primarily used SurveyMonkey Audience, which allowed us to get answers from a statistically significant random selection of people in both the United States and Australia.

We created two nearly identical surveys with one targeted to a US-only audience and the other targeted to an Australian-only audience. A proper sample set is essential when conducting any survey that attempts to draw conclusions about people’s general behavior and preferences. And in this case, the minimum number of respondents we needed in order to capture a representative example was 350 for the U.S. and 300 for Australia.

Additionally, in order for a sample to be valid, the respondents have to be chosen completely at random. SurveyMonkey Audience recruits its 4-million+ members from SurveyMonkey’s 40 million annual unique visitors, and members are not paid for their participation. Instead, they are rewarded for taking surveys with charitable donations, made on their behalf by SurveyMonkey.

When tested against much larger research projects, Audience data has been exactly in line with larger sample sizes. For example, an Audience survey with just 400 respondents about a new Lay’s potato chip flavor had the same results as a wider contest that had 3 million participants.

SurveyMonkey’s survey research team was also able to use SurveyMonkey Audience to accurately predict election results in both 2012 and 2013. With a US sample size of 458 respondents and an Australian one of 312 all drawn at random, our ccTLD user preferences should reliably mirror the actual reality.

Summary

There will be many reasons that you may or may not want to use ccTLDs for your website, and a survey alone can never answer whether a ccTLD is the right strategy for any particular site. If you are thinking about making any big decisions about TLDs on your site, you should absolutely conduct some testing or surveying of your own before relying on just the recommendations of those who advise a TLD as the best strategy or the others that tell you it doesn’t matter at all.

Launching a PPC campaign with a landing page on a ccTLD and measuring CTRs against a control is far cheaper than replicating your entire site on a new TLD.

Based on our survey results, here’s what you should keep in mind when it comes to whether or not investing your time and money in a ccTLD is worth it:

  1. Users are absolutely aware of the TLDs and how they might relate to the contents of a website
  2. Users are aware of the connection between TLDs and countries
  3. Users do make decisions about websites based on the TLD; however there are no absolutes. Brand and content absolutely matter.

As to whether a ccTLD will work for you on your own site, give it a try and report back!

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 4 years ago from moz.com

Lessons from the Front Line of Front-End Content Development

Posted by richardbaxterseo

As content marketing evolves, the list of media you could choose to communicate your message expands. So does the list of technologies at your disposal. But without a process, a project plan and a tried and tested approach, you might struggle to gain any traction at all.

In this post, based on my
MozCon 2014 presentation, I’d like to share the high level approach we take while developing content for our clients, and the lessons we’ve learned from initial research to final delivery. Hopefully there are some takeaways for you to enhance your own approach or make your first project a little less difficult.

This stuff is hard to do

I hate to break it to you, but the first few times you attempt to develop something
a little more innovative, you’re going to get burned. Making things is pretty tough and there are lots of lessons to learn. Sometimes you’ll think your work is going to be huge, and it flops. That sucks, move on, learn and maybe come back later to revisit your approach.

To structure and execute a genuinely innovative, successful content marketing campaign, you need to understand what’s possible, especially within the context of your available skills, process, budget, available time and scope.

You’ll have a few failures along the journey, but when something goes viral, when people respond positively to your work – that, friends, feels amazing.

What this post is designed to address

In the early days of SEO, we built links. Email outreach, guest posting, eventually, infographics. It was easy, for a time. Then,
Penguin came and changed everything.

Our industry learned that we should be finding creative and inventive ways to solve our customers’ problems, inspire, guide, help – whatever the solution, an outcome had to be justified. Yet still, a classic habit of the SEO remained: the need to decide in what form the content should be executed before deciding on the message to tell.

I think we’ve evolved from “let’s do an infographic on something!” to “I’ve got a concept that people will love should this be long form, an interactive, a data visualization, an infographic, a video, or something else?”

This post is designed to outline the foundations on an approach you can use to enhance your approach to content development. If you take one thing away from this article, let it be this:

The first rule of almost anything: be prepared or prepare to fail. This rule definitely applies to content development!

Understand the technical environment you’re hosting your content in

Never make assumptions about the technical environment your content will be hosted in. We’ve learned to ask more about technical setup of a client’s website. You see, big enterprise class sites usually have load balancing, 
pre-rendering, and very custom JavaScript that could introduce technical surprises much too late in the process. Better to be aware of what’s in store than hope your work will be compatible with its eventual home.

Before you get started on any development or design, make sure you’ve built an awareness of your client’s development and production environments. Find out more about their CMS, code base, and ask what they can and cannot host.

Knowing more about the client’s development schedule, for example how quickly a project can be uploaded, will help you plan lead times into your project documentation.

We’ve found that discussing early stage ideas with your client’s development team will help them visualise the level of task required to get something live. Involving them at this early stage means you’re informed on any potential risk in technology choice that will harm your project integrity later down the line.

Initial stakeholder outreach and ideation

Way back at MozCon 2013, I presented an idea called “really targeted outreach“. The concept was simple: find influential people in your space, learn more about the people they influence, and build content that appeals to both.

We’ve been using a similar methodology for larger content development projects: using social data to inspire the creative process gathered from the Twitter Firehose and
other freely available tools, reaching out to identified influencers and ask them to contribute or feedback on an idea. The trick is to execute your social research at a critical, early stage of the content development process. Essentially, you’re collecting data to gain a sense of confidence in the appeal of your content.

We’ve made content with such a broad range of people involved, from astronauts to butlers working at well known, historic hotels. With a little of the right approach to outreach, it’s amazing how helpful people can be. Supplemented by the confidence you’ve gained from your data, some positive results from your early stage outreach can really set a content project on the right course.

My tip: outreach and research several ideas and tell your clients which was most popular. If you can get them excited and behind the idea with the biggest response then you’ll find it easier to get everyone on the same page throughout your project.

Asset collection and research

Now, the real work begins. As I’ve
written elsewhere, I believe that the depth of your content, it’s accuracy and integrity is an absolute must if it is to be taken seriously by those it’s intended for.

Each project tends to be approached a little differently, although I tend to see these steps in almost every one: research, asset collection, storyboarding and conceptual illustration.

For asset collection and research, we use a tool called
Mural.ly – a wonderful collaborative tool to help speed up the creative process. Members of the project team begin by collecting relevant information and assets (think: images, quotes, video snippets) and adding them to the project. As the collection evolves, we begin to arrange the data into something that might resemble a timeline:

After a while, the story begins to take shape. Depending on how complex the concept is, we’ll either go ahead with some basic illustration (a “white board session”) or we’ll detail the storyboard in a written form. Here’s the Word document that summarised the chronological order of the content we’d planned for our
Messages in the Deep project:

messages-in-the-deep-storyboard

And, if the brief is more complex, we’ll create a more visual outline in a whiteboard session with our designers:

interactive-map-sketch

How do you decide on the level of brief needed to describe your project? Generally, the more complex the project, the more important a full array of briefing materials and project scoping will be. If, however, we’re talking simpler, like “long form” article content, the chances are a written storyboard and a collection of assets should be enough.

schema-guide

Over time, we’ve learned how to roll out content that’s partially template based, rather than having to re-invent the wheel each time. Dan’s amazing
Log File Analysis guide was reused when we decided to re-skin the Schema Guide, and as a result we’ve decided to give Kaitlin’s Google Analytics Guide the same treatment.

Whichever process you choose, it helps to re-engage your original contributors, influencers and publishers for feedback. Remember to keep them involved at key stages – if for no other reason than to make sure you’re meeting their expectations on content they’d be willing to share.

Going into development

Obviously we could talk all day about the development process. I think I’ll save the detail for my next post, but suffice it to say we’ve learned some big things along the way.

Firstly, it’s good to brief your developers well before the design and content is finalised. Particularly if there are features that might need some thought and experimental prototyping. I’ve found over time that a conversation with a developer leads to a better understanding of what’s easily possible with existing libraries and code. If you don’t involve the developers in the design process, you may find yourself committed to building something extremely custom, and your project timeline can become drastically underestimated.

It’s also really important to make sure that your developers have had the opportunity to specify how they’d like the design work to be delivered; file format; layers and sizing for different break points are all really important to an efficient development schedule and make a huge difference to the agility of your work.

Our developers like to have a logical structure of layers and groups in a PSD. Layers and groups should all be named and it’s a good idea to attach different UI states for interactive elements (buttons, links, tabs, etc.), too.

Grid layouts are much preferred although it doesn’t matter if it’s 1200px or 960px, or 12/16/24 columns. As long as the content has some structure, development is easier.

As our developers like to say: Because structure = patterns = abstraction = good things and in an ideal world they prefer to work with
style tiles.

Launching

Big content takes more promotion to get that all important initial traction. Your outreach strategy has already been set, you’ve defined your influencers, and you have buy in from publishers. So, as soon as your work is ready, go ahead and tell your stakeholders it’s live and get that flywheel turning!

My pro tip for a successful launch is be prepared to offer customised content for certain publishers. Simple touches, like
The Washington Post’s animated GIF idea was a real touch of genius – I think some people liked the GIF more than the actual interactive! This post on Mashable was made possible by our development of some of the interactive to be iFramed – publishers seem to love a different approach, so try to design that concept in right at the beginning of your plan. From there, stand back, measure, learn and never give up!

That’s it for today’s post. I hope you’ve found it informative, and I look forward to your comments below.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 4 years ago from moz.com

Panda 4.1: The Devil Is in the Aggregate

Posted by russvirante

I wish I didn’t have to say this. I wish I could look in the eyes of every victim of the last Panda 4.1 update and tell them it was something new, something unforeseeable, something out of their control. I wish I could tell them that Google pulled a fast one that no one saw coming. But I can’t.

Like many in the industry, I have been studying Panda closely since its inception. Google gave us a rare glimpse behind the curtain by providing us with the very guidelines they set in place to build their massive machine-learned algorithm which came to be known as Panda. Three and a half years later, Panda is still with us and seems to still catch us off guard.
Enough is enough.

What I intend to show you throughout this piece is that the original Panda questionnaire still remains a powerful predictive tool to wield in defense of what can be a painful organic traffic loss. By analyzing the winner/loser reports of Panda 4.1 using standard Panda surveys, we can determine whether Google’s choices are still in line with their original vision. So let’s dive in.

The process

The first thing we need to do is acquire a winners and losers list. I picked this excellent
one from SearchMetrics although any list would do as long as it is accurate. Second, I proceeded to run a Panda questionnaire with 10 questions on random pages from each of the sites (both the winners and losers). You can run your own Panda survey by following Distilled and Moz’s instructions here or just use PandaRisk like I did. After completing these analyses, we simply compare the scores across the board to determine whether they continue to reflect what we would expect given the original goals of the Panda algorithm.

The aggregate results

I actually want to do this a little bit backwards to drive home a point. Normally we would build to the aggregate results, starting with the details and leaving you with the big picture. But Panda
is a big-picture kind of algorithmic update. It is specially focused on the intersection of myriad features, the sum is greater than the parts. While breaking down these features can give us some insight, at the end of the day we need to stay acutely aware that unless we do well across the board, we are at risk.

Below is a graph of the average cumulative scores across the winners and losers. The top row are winners, the bottom row are losers. The left and right red circles indicate the lowest and highest scores within those categories, and the blue circle represents the average. There is something very important that I want to point out on this graph.
The highest individual average score of all the losers is less than the lowest average score of the winners. This means that in our randomly selected data set, not a single loser averaged as high a score as the worst winner. When we aggregate the data together, even with a crude system of averages rather than the far more sophisticated machine learning techniques employed by Google, there is a clear disparity between the sites that survive Panda and those that do not.

It is also worth pointing out here that there is no
positive Panda algorithm to our knowledge. Sites that perform well on Panda do not see boosts because they are being given ranking preference by Google, rather their competitors have seen rankings loss or their own previous Panda penalties have been lifted. In either scenario, we should remember that performing well on Panda assessments isn’t going to necessarily increase your rankings, but it should help you sustain them.

Now, let’s move on to some of the individual questions. We are going to start with the least correlated questions and move to those which most strongly correlate with performance in Panda 4.1. While all of the questions had positive correlations, a few lacked statistical significance.


Insignificant correlation

The first question which was not statistically significant in its correlation with Panda performance was “This page has visible errors on it”. The scores have been inverted here so that the higher the score, the fewer the number of people who reported that the page has errors. You can see that while more respondents did say that the winners had no visible errors, the difference was very slight. In fact, there was only a 5.35% difference between the two. I will save comment on this until after we discuss the next question.

The second question which was not statistically significant in its correlation with Panda performance was “This page has too many ads”. The scores have once again been inverted here so that the higher the score, the fewer the number of people who reported that the page has too many ads. This was even closer. The winners performed only 2.3% better than the losers in Panda 4.1.

I think there is a clear takeaway from these two questions. Nearly everyone gets the easy stuff right, but that isn’t enough. First, a lot of pages just have no ads whatsoever because that isn’t their business model. Even those that do have ads have caught on for the most part and optimized their pages accordingly, especially given that Google has other layout algorithms in place aside from Panda. Moreover, content inaccuracy is more likely to impact scrapers and content spinners than most sites, so it is unsurprising that few if any reported that the pages were filled with errors. If you score poorly on either of these, you have only begun to scratch the surface, because most websites get these right enough.


Moderate correlation

A number of Panda questions drew statistically significant difference in means but there was still substantial crossover between the winners and losers.
Whenever the average of the losers was greater than the lowest of the winners, I considered it only a moderate correlation. While the difference between means remained strong, there was still a good deal of variance in the scores. 

The first of these to consider was the question as to whether the content was “trustworthy”. You will notice a trend in a lot of these questions that there is a great deal of subjective human opinion. This subjectivity plays itself out quite a bit when the topics of the site might deal with very different categories of knowledge. For example, a celebrity fact site might be very trustworthy (although the site might be ad-laden) and an opinion piece in the New Yorker on the same celebrity might not be seen as trustworthy – even though it is plainly labeled as opinion. The trustworthy question ties back to the “does this page have errors” question quite nicely, drawing attention to the difference between a subjective and objective question and the way it can spread the means out nicely when you ask a respondent to give more of a personal opinion. This might seem unfair, but in the real world your site and Google itself is being judged by that subjective opinion, so it is understandable why Google wants to get at it algorithmically. Nevertheless, there was a strong difference in means between winners and losers of 12.57%, more than double the difference we saw between winners and losers on the question of Errors.

Original content has long been a known requirement of organic search success, so no one was surprised when it made its way into the Panda questionnaire. It still remains an influential piece of the puzzle with a difference in mean of nearly 20%. It was barely ruled out from being a heavily correlated feature due to one loser edging out a loss against the losers’ average mean. Notice though that one of the winners scored a perfect 100% on the survey. This perfect score was received despite hundreds of respondents.
It can be done.

As you can imagine, perception on what is and is not an authority is very subjective. This question is powerful because it pulls in all kinds of assumptions and presuppositions about brand, subject matter, content quality, design, justification, citations, etc. This likely explains why this question is beleaguered by one of the highest variances on the survey. Nevertheless, there was a 13.42% difference in means. And, on the other side of the scale, we did see what it is like to have a site that is clearly not an authority, scoring the worst possible 0% on this question. This is what happens when you include highly irrelevant content on your site just for the purpose of picking up either links or traffic. Be wary.

Everyone hates the credit card question, and luckily there is huge variance in answers. At least one site survived Panda despite scoring 5% on this question. Notice that there is a huge overlap between the lowest winner and the average of the losing sites. Also, if you notice by the placement of the mean (blue circle) in the winners category, the average wasn’t skewed to the right indicating just one outlier. There was strong variance in the responses across the board. The same was true of the losers. However, with a +15% difference in means, there was a clear average differentiation between the performance of winners and losers. Once again, though, we are drawn back to that aggregate score at the top, where we see how Google can use all these questions together to build a much clearer picture of site and content quality. For example, it is possible that Google pays more attention to this question when it is analyzing a site that has other features like the words “shopping cart” or “check out” on the homepage. 

I must admit that the bookmarking question surprised me. I always considered it to be the most subjective of the bunch. It seemed unfair that a site might be judged because it has material that simply doesn’t appeal to the masses. The survey just didn’t bear this out though. There was a clear difference in means, but after comparing the sites that were from similar content categories, there just wasn’t any reason to believe that a bias was created by subject matter. The 14.64% difference seemed to be, editorially speaking, related more to the construction of the page and the quality of the content, not the topic being discussed. Perhaps a better way to think about this question is:
would you be embarrassed if your friends knew THIS was the site you were getting your information from rather than another.

This wraps up the 5 questions that had good correlations but substantial enough variance that it was possible for the highest loser to beat out the average winner. I think one clear takeaway from this section is that these questions, while harder to improve upon than the Low Ads and No Errors questions before, are completely within the webmaster’s grasp. Making your content and site appear original, trustworthy, authoritative, and worthy of bookmarking aren’t terribly difficult. Sure, it takes some time and effort, but these goals, unlike the next, don’t appear that far out of reach.


Heavy correlation

The final three questions that seemed to distinguish the most between the winners and losers of Panda 4.1 all had high difference-in-means and, more importantly, had little to no crossover between the highest loser and lowest winner. In my opinion, these questions are also the hardest for the webmaster to address. They require thoughtful design, high quality content, and real, expert human authors.

The first question that met this classification was “could this content could appear in print”. With a difference in mean of 22.62%, the winners thoroughly trounced the losers in this category. Their sites and content were just better designed and better written. They showed the kind of editorial oversight you would expect in a print publication. The content wasn’t trite and unimportant, it was thorough and timely. 

The next heavily correlated question was whether the page was written by experts. With over a 34% difference in means between the winners and losers, and
literally no overlap at all between the winners’ and losers’ individual averages, it was clearly the strongest question. You can see why Google would want to look into things like authorship when they knew that expertise was such a powerful distinguisher between Panda winners and losers. This really begs the question – who is writing your content and do your readers know it?

Finally, insightful analysis had a huge difference in means of +32% between winners and losers. It is worth noting that the highest loser is an outlier, which is typified by the skewed mean (blue circle) being closer to the bottom that the top. Most of the answers were closer to the lower score than the top. Thus, the overlap is exaggerated a bit. But once again, this just draws us back to the original conclusion – that the devil is not in the details, the devil is in the aggregate. You might be able to score highly on one or two of the questions, but it won’t be enough to carry you through.


The takeaways

OK, so hopefully it is clear that Panda really hasn’t changed all that much. The same questions we looked at for Panda 1.0 still matter. In fact, I would argue that Google is just getting better at algorithmically answering those same questions, not changing them. They are still the right way to judge a site in Google’s eyes. So how should you respond?

The first and most obvious thing is you should run a Panda survey on your (or your clients’) sites. Select a random sample of pages from the site. The easiest way to do this is get an export of all of the pages of your site, perhaps from Open Site Explorer, put them in Excel and shuffle them. Then choose the top 10 that come up.  You can follow the Moz instructions I linked to above, do it at PandaRisk, or just survey your employees, friends, colleagues, etc. While the latter probably will be positively biased, it is still better than nothing. Go ahead and get yourself a benchmark.

The next step is to start pushing those scores up one at a time. I
give some solid examples on the Panda 4.0 release article about improving press release sites, but there is another better resource that just came out as well. Josh Bachynski released an amazing set of known Panda factors over at his website The Moral Concept. It is well worth a thorough read. There is a lot to take in, but there are tons of easy-to-implement improvements that could help you out quite a bit. Once you have knocked out a few for each of your low-scoring questions, run the exact same survey again and see how you improve. Keep iterating this process until you beat out each of the question averages for winners. At that point, you can rest assured that your site is safe from the Panda by beating the devil in the aggregate. 

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 4 years ago from feedproxy.google.com

Customer Journey Maps – Whiteboard Friday

Posted by kerrybodine

At every stage in the marketing funnel, it’s crucially important to empathize with your customers’ interactions with your business, feeling great about the high points and frustrated by the lows. In today’s Whiteboard Friday, MozCon 2014 speaker Kerry Bodine shows us all about customer journey mapping—a tool that allows us to visualize and learn from those experiences.

Video transcription

Hi, I’m Kerry Bodine. I am a customer experience consultant, and I am the co-author of a book called “Outside In.” The subtitle of the book is “The Power of Putting Customers at the Center of Your Business.” That’s really what I am all about. I try and help companies to take customer knowledge, customer insights and really bring it into their organization, so that they can become more customer-centric.

What I’d like to share with you today is a tool from the customer experience world that I think is really critical for every marketer out there to understand. It is called a “customer journey map.” Really simply, all a journey map is, is it’s an illustration that shows all the different steps that your customers go through as they do business with you over time.

In addition to showing just what they do, it also shows customers’ thoughts, their feeling, and their emotions. The goal of the customer journey map is really to get a holistic view of what the customer is going through from their point of view and really what it’s like for them on a personal level, that human level. I’ll share a little bit about how customer journey maps work, and we’ll wrap up with how you can do this yourselves within your own organizations.

What I’ve got behind me here is the start of a customer journey map, what this typically looks like. As you can see, as customers interact with you, it’s not just a straight line. Some of those experiences are going to be better, and some of those experiences are going to be worse. What you want to do is you want to track what those actually look like over time. Now ideally, you are going to be understanding where those bright spots are. Those are the things that your company is really doing well to help meet your customers’ goals.

You’ve also got to understand where things aren’t going so well for your customers, where you’re not delivering the value that they’re looking for, where you’re making it really difficult just to do business with you, or where you’re just not treating them as a human being. You’re treating them as just kind of a line in a spreadsheet or maybe a record in your CRM system. We’ve got to really understand our customers at a human level.

Why is a journey map like this so important for marketers? Well, part of the reason is that, at some point as we go along this journey, we’re going through that typical marketing funnel. The customer first learns about your products and services. Then there’s consideration, and they move into actually purchasing whatever it is that you’re providing. We’re not talking with those words when we’re doing a journey map, because no customer is out there saying, “Oh, I’m in the awareness phase right now of buying shoes.”

No, they’re just saying, “Hey, I’m out there researching shoes.”

Those are the types of steps that you put on here. As you go along, your customers are learning about your products and services, and then they’re buying them hopefully. At some point, the traditional role of the marketer ends. The rest of the customer journey, maybe receiving those shoes in the mail if they’ve ordered them online and then trying them on, and if they don’t fit, maybe the process of returning them, that all happens after that purchase point. We’ve got half of this customer journey that’s really all about making promises to the customer.

This is what marketing is traditionally set up to do. They are set up to help customers to understand why it’s going to be so amazing to spend money with their particular company. All of these different touch points here are in the service of making a promise to the customer about what they’re going to get after they’ve purchased from you. All of the touch points that follow are really about delivering on that promise. As you can see in this journey, the organization really didn’t deliver well on whatever it was that was promised during this phase over here.

The interesting thing is that not only do marketers need to care about these journey maps, but everyone else in the organization does as well. While marketers might be primarily responsible for this process of making promises, there are many, many other parts of the organization that are primarily responsible for delivering on those promises. You’ve got people who are working in customer service, in retail, in finance, in operations, behind the scenes, in parts of the organization like legal and IT, parts of the organization that never even talked to a customer typically during their employment at that company or maybe in their entire careers.

These journey maps can help to unite all of the different parts of the organization. It can help someone in marketing understand really what they need to be promising in order to have expectations set correctly for the end of this process. It can also help people who are responsible for delivering the rest of the customer experience. It can help them understand really what that pre-purchase experience is like and really what is being promised to customers.

This is really an effective tool at helping to break people out of their organizational silos, getting them to understand that holistic customer viewpoint across all the different touch points, and getting people within the organization to have empathy for each other, their fellow colleagues, or perhaps external partners, who are all playing a role in delivering this journey behind me.

How can you do this yourself within your organization? What I want to do is share with you a very simple method for doing journey mapping with any group. All you really need is to have a whiteboard like this, or maybe you’re going to have a big sheet of butcher paper that you can get at any office supply store. You want to have some markers. I typically like using Sharpie markers, because you can read them from a distance. My very, very favorite tool for doing this, packs of sticky notes.

All you’re going to do is you’re going to write down each step in the customer journey on a different sticky note. Then all you need to do is put them up on your whiteboard or your piece of white butcher paper in the order that the customer would go through their particular journey.

I mentioned buying shoes before, and what I’m putting up here are all the different steps that a customer might go through if they were buying shoes from your company. They’re going to search for the shoes. They’re going to follow a link to a website. They’re going to learn about the product. They’re going to buy the shoes. They’re going to wait to receive them. Then they’ll finally receive them. They’re going to try on the shoes, and they’re not going to fit here. They’re going to go to the website, but they can’t find the returns information. They’re going to call customer service. They’re going to get the return information. I’m running out of room here. They’re going to print a return label. They’re going to box up their shoes, and then they’re going to drop the box off at the shipper, UPS or USPS, whatever it is that they’re using.

That’s really the basic building blocks of creating a journey map. It’s just going through and mapping out step by step what the customer is going through. I like using stickers for this. You can get red and green stickers at your office supply store. You can use markers. The idea is that you’re going to note where the different steps in that process are going well and then maybe where those steps start to go south. This will give you a really good depiction of where the problem points are in your customer journey and where you need to focus on improving interactions to better meet your customer’s needs.

You can go a lot further with this. You can start detailing what your customers are thinking and what they’re feeling. You can add those in on different colors of Post-it notes. You can also denote all of the different touch points that they’re interacting with. Are they talking to the call center? Are they on the website? Are they on Google? Whatever those touch points happen to be. You can even dig down deeper into the organization to start to identify who is responsible for all of those different interactions, so that again you really know where you need to be focusing on fixing the systemic problems within your organization.

What I would recommend that you do is conduct this type of exercise with people from across your organization. I mentioned that this is a really great tool for breaking down organizational silos. Really, that’s only going to happen if you get the people from all of those different organizational silos involved in creating this diagram. Hold a half-day workshop. Bring in people from all the different parts of your organization, maybe some of your key partners, and map out what you think this journey is based on your best assumptions about customers.

But don’t stop there, because, often, what we find are that our assumptions are either wrong or they’ve got big gaps in them. The second step to this process is to bring customers into the workshop and have them validate this. The beauty of this is that when you’ve created this out of sticky notes, your customers are going to have no problem going up and removing sticky notes, adding new sticky notes, moving them around so that the journey more accurately reflects what it is that they go through when they do business with you.

That is Journey Mapping 101. I hope that I’ve introduced you to a tool that you can use within your organization. If you would like more information about customer journey maps, please visit my website. It’s KerryBodine.com/CustomerJourneyMaps. Thanks very much.

Video transcription by Speechpad.com

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Reblogged 4 years ago from feedproxy.google.com