Brad Fallon
Smart Marketing, Inc.

How to Spy on Your Site Visitors and Win the Marketing War through Better Intelligence and Faster Response

Any Internet marketer worth her salt knows the importance of tracking web site statistics. It’s called many things – web analytics, site stats, or just “How many ‘hits’ did we get this month versus last month?” To do this, software is used to compile raw server log file data into pretty charts, graphs and navigation trackers.

After you accumulate some data, you can learn a lot about user behavior on your landing pages, web sites, landing pages and checkout pages. This is like the general back at the Pentagon analyzing data delivered to him from people on the ground all over the war zone. And of course there is much value in paying attention and analyzing this data on a macro scale.

Recently, however, I picked up a rifle, fixed my bayonet and charged into the front lines myself. You see, I’ve discovered that for getting a feel for the users’ experience on a web site, there’s nothing like being there with them while they surf through your site.

Ok, Brad, what are you talking about? One of those focus/usability/oberve-the-users-and take-notes groups? Ahh . . . I wish my clients had budgets for such endeavors. Actually, I’ve discovered a way anyone can accomplish the same thing and more. Plus, it works like a champ, and it’s free or close to free.

Let me share with you an idea that I have started using on the last several web initiatives I have launched. I’ll use two examples, one a large B2B software company and the other my wife’s new Yahoo store that sells wedding favors.

See You See Me

In a nutshell, I started using the LivePerson software to spy on all of our site visitors. You know LivePerson, right? That’s the application that lets your site visitors click to chat live with a sales person or customer service rep. Perhaps you’re like me and you haven’t tried it because you couldn’t imagine staffing people to sit there and chat with site visitors all day. And it’s not exactly the high-end feel for a site is it? I mean, you’re not going to see “click here to chat” on Siebel’s home page.

But the great thing is you don’t have to use Liver Person’s live chat feature to use their cool back office. Here’s how it works:

That’s it! Now, whenever someone visits your web site, the application rings a doorbell and tracks them all through the site. (When you first launch a web site, those “ding-dong’s” are exciting. Later, you must turn the sound off before you go nuts.)

View from the Foxhole

So here’s how it plays out on the front lines. The doorbell alerts you that someone just hit your site – excellent. So you stop reading email and alt-tab over to the window with the whole back office view.

The screen is divided into three parts, top and bottom. The top half stretches across the screen, and the bottom half is divided into left and right. The way it’s laid out you can see a lot of information at a glance. The first thing you notice is the page the person landed on and where they came from.

Most of the time, if they came from a search engine, the referring code shows you which search engine and what they typed in to find your web page in the search results. In fact, the application displays an image of the actual referring search engine page.

So not only can you see at a glance which search engine it was, but you can click on the image and pull up the exact page that they just left to come to your site. In a new window, you will see the results page, and your site’s listing as it is in the results – whether it’s an Overture ad at the top or the second page of Google with your listing halfway down. So you check that out to see what your search results are starting to look like. As you continue to do this while you’re at your computer you really develop a feel for your SEO progress.

You get the idea. If you’re studying and testing Google AdWords (you are, right?) then you know that one of the keys is to get as many “little” keyword phrases as you can. All those little infrequent searches can really add-up. And they are cheap, so it’s the best thing for your ROI.

Of course this software is made so a salesperson can see what sorts of products the prospect was looking at before visiting the site. It’s a much more efficient live sales process if you know the context of what they were looking at when they clicked on your link. But I like it because it gives you a feel for what’s going on with regard to all the different search terms and search listings for your site.

Why the view is better up front

Now of course you can look at your site stats after the fact and do much the same thing. You don’t have to be on the front lines, for example, to see what search terms people are using to find your site and adjust your AdWords campaign. You can see the whole list from your web analytics software anytime you want. But as we’ll see in the next section, there’s just something about the real-time nature of the information gathering that makes it special.

For SEO issues, every time you click on the Google image and go look at another search results page, you learn things you just can’t see, or perhaps “feel” with a search word list and a rankings report. For example, when you go look at the actual pages you get a real sense of what your competitors are doing and other valuable information.

As we’ll show by example, spying on your web site visitors in real time allows you to get a feel for what’s going on and make improvements – faster.

Get better faster

You see, one of the advantages of being on the front lines is that you can make faster changes to relevant real-time information. For me, this “Live Spying” works especially well at the launch of a new site. In these cases, I have limited data about the reactions site visitors will have and I’m not getting hammered with traffic the first week.

Like many marketers, I usually use paid clicks (usually AdWords) for initial traffic and tests. However, my focus is generally to get in the free search results as soon as possible. When we launch a new site we use what we call a Googlefast technique to get all of our sites spidered by Googlebot within 24 hours and showing in the actual Google search results within 72 hours.

For many terms, we’re able to get very high Google rankings quickly. For example the brand-new wedding favor site, after being live for only two weeks is already #1 on Google for beach wedding favors, golf wedding favors, wine wedding favors, silver wedding favors, and doing well for some other fairly competitive terms as well.

The Google optimization methods are beyond the scope of this article, but by watching carefully where we first come up, as the first free search visitors start hitting the site, we can make immediate adjustments to capitalize on the initial successes. Not only that, but watching closely clues you in to other low-cost paid search opportunities that might not otherwise be on your immediate hit list.

There’s never enough time to do everything, and sometimes a couple of visitors are enough to make you add another pay-per-click engine or buy an inexpensive ad link on a related site. For example, sometimes a reciprocal link partner actually sends you some traffic as well as Page Rank and even more valuable Reputation. In this case, you can often pay a little and buy a better placement on that particular site. Otherwise, you’d be purely guessing where to commit link ad dollars. Sites already sending traffic are a great place to start.

It’s almost scary: spying on the site visitors themselves

Search-related data refers to where the visitors were and what they did before they got to your site. In the remainder of this article, we’ll look at what we can learn after they get there.

The Live Person software shows all the visitors currently in the site in the top half of the screen. All the visitors are stacked in rows; in each row going across you see their IP address, the page they’re currently on, how much total time they’ve been in the site, and how many pages they’ve visited.

If you have the “click to chat” button installed you can “Invite them to chat” which sends a horribly gaudy big box floating across their screen, right on top of what they were trying to read.

I did try this twice, and both times they closed the box and left the site. I know, I know, it’s not personal, but I did feel a little rejected – I was so ready to strike up a conversation and make friends with my potential customer. But my hunch that the annoying big box would be . . . well, annoying, appeared to be confirmed. Admittedly, that was the end of my “Let’s Invite a Stranger to Chat” test.

When you click on one of the visitors at the top, his individual surfing trail appears at the bottom of the screen on the left. You see each page they’ve been to, in order, and how much time they spent on each page. This is the fun part.

It’s just as good as watching them live in a laboratory. Well better, actually, because they don’t know they’re being watched and you don’t have to try and peek over their shoulder to see what they’re doing. You can watch it right from your monitor.

The funny thing is, the sample size is small (n always equals one). But by “watching” them live as they surf your site, you really do get a feel for what’s going on. I’ll give you a couple examples of learning by watching individual site visitors and being able to make changes fast.

Large software company: Good “intelligence” pays off

When we launched the new product-specific web site for a large software company, we had two goals in mind:

Goal number one began quickly. Using the Googlefast technique mentioned above, the new site on a new domain was spidered by Googlebot approximately 12 hours after launch, and it was #8 on the first page of Google for its most-competitive search term exactly seven days later.

Unfortunately, the site was launched November 5, 2003 and seven days later was four days before the Florida update when we got “Scroogled” for that one search term. (We hadn’t had time to get enough of the right kind of links to survive that update.)

As we launched, and visitors started making the doorbell ring in Live Person, we were continually amazed at the search terms we hadn’t included in our AdWords keyword list. Throughout the first weeks, we constantly added keywords to the paid campaign based on the traffic through the Live Person software.

The second goal, conversion, involved a three step process. First, we designed the entire site as a “landing site” rather than using certain “landing pages” for the paid click ads. We did this because no matter which page came up in the search results, we wanted the highest chance of capturing the lead. Because the home page usually comes up, we designed the main page itself to tell the story.

Second, we offered as many quid pro quo’s, or reasons for visitors to give us their email addresses as possible, and we had them in the right-hand column on every page. For this client, we researched and wrote their first white paper to exchange for email addresses, designed and built an interactive ROI calculator, and moved their software demo online for the first time.

We had decided to require a fairly lengthy form for access to the online demo and other tools so we carefully monitored the visitors in real time to get a feel for what was happening. Again, in each case you’re only looking at one individual, but it sure felt like we were gaining an understanding.

Quietly looking over their shoulder, bayonet ready

For example, in many cases, the doorbell would ring, I’d pop open the window and “watch” the visitor. While he was reading the text on the home page and deciding what to do, I’d click on the Google page showing in the bottom right of the window. Then I’d go see what he just searched and where we were coming up in the results, if at all, or if it was an ad.

Then, I’d go back and watch him. Often, he would click on the “online demo” right away, where he quickly learned that he had to fill out a form to get in. “Naaaahh,” I could hear him say, while he’d quickly click on Products to get more information before he decided if it was worth his time to fill out the form.

Sure enough, he’d read for a few minutes. Sometimes he would look at some of the other pages.

I had been prepared to lessen the requirements to get the demo in order to get more leads albeit with less info, but as we were having over 12 percent of all site visitors fill out the long form, we left it the way it was.

Battlefield decision-making

Here’s one thing that we did change as a direct result of this process that I think would have taken longer and been much harder to see and fix if we had depended on looking at static site statistics later, after the battle was over, so to speak.

As we were watching people on the very first day, we noticed that many were coming to the site after searching for the exact product the site was all about. Then, they would click on “products,” but leave the site very soon after that. Watching them do this once or twice, I immediately looked at the product page and realized that we had put the company’s other products there. The main product that was the subject of this site was covered completely in a page from a different navigation link with the name of that product.

Unfortunately, people were used to just clicking on “product” and when they didn’t find what they wanted, they left. Fortunately, by watching this in real time (it only took a couple of times) we merely and immediately added some text to the top of the product page and the problem was fixed – on the morning of the launch. I’m convinced that if we hadn’t been spying over their shoulder in real time, it would have taken much longer to recognize the problem. We would have had lower conversions and who knows what we would have tried to improve it.

Selling products

Sales people love to use war analogies, but without bringing up YASTA (Yet Another Sun Tsu Anecdote), I’ll give you an example of using this technique to close more sales in a simple Yahoo store.

When my wife first launched MyWeddingFavors.com a couple weeks ago, we were all over the Live Person monitor. Sure, we could have “analyzed” the click trails later and tried to figure out what they really meant, and what changes we should make.

But it’s not the same as being on the front lines.

For example, as we launched the store, we would sit there and watch all the visitors. How long did they stay on the home page (not long). Which category pages did they go to first? (We moved those higher up on the navigation bar.) Which products were more popular? (We moved those up on the product page.) Which ones did people really seem to like? (We featured those on the home page.)

I just don’t believe I could learn the same things by staring at the static reports later on. Yes, you can see that “more people exit the site on this page.” Well ok, but maybe that’s the most popular page. You really have to try harder to decipher the data.

On the contrary, when you watch it in real time, you sense what’s happening. Call it battlefield instincts. Example: I’m minding my own business, writing an email, when the doorbell rings. Ding dong. I go to the site, and quickly glance down at the search he just arrived from – just “wedding favors.” Not much there, but he clicks on “Golf Wedding Favors” first. We only have a couple measly products there so far; he spends a few seconds and leaves.

Any doubt what’s happening? Not after watching them go to the sites with lots of cool products and spend 30 minutes. So what do I do? Maybe nothing – until I click on one of the visitors who came from a search for “golf wedding favors.” I go to the exact Google page and I realize we’re #1 for “golf wedding favors.” A quick check on Overture’s term suggestion tool to see how many people are searching for that, and hey, maybe it’s time to get serious about adding some golf wedding favors.

In short, this simple technique of keeping a simple window open really keeps you in touch with what’s going on with your site. Like the VP riding with a sales rep in the field, or a general walking amongst the troops, you have to be on the front lines to really get a feel for what’s going on.

And one final tip. If you do SEO for a living, and your site’s been up for awhile . . . If you’re ever in a bad mood, just turn your sound back on and the barrage of doorbells can really make you feel better.