Moving Forward by Stepping Back

In April of 2005 I sat at my usual corner table at Krispy Kreme donuts in San Antonio.  It had become my “thinking place” and every few weeks I would ponder life over a couple of donuts and a cup of coffee. I was reading a series of articles on the growing problem of “click fraud”.  Our former parent company, Optimal iQ, had recently found a group of fake clicks and we were intrigued by the problem.  Perhaps it was the highly caffeinated sugar rush but at that moment, I had an epiphany.  I realized the problem of click fraud was only going to get worse and more complex.  Measure this challenge against the rapid growth of search marketing and there was an opportunity staring me in the face.

Click Forensics was born in January of 2006 through the hard work of my co-founders and our dedicated team. We raised over $15,000,000 to solve a problem, execute a plan and lead the industry.  Today Click Forensics is the clear leader in traffic quality management. Our original vision was to “help ensure advertisers get what they pay for” and today that is exactly what we do.  I’m very proud of Click Forensics and excited about the opportunity to continue to serve and improve the online advertising community.

In June of last year I made the decision to begin my exit from the company.  Click Forensics is growing rapidly and developing cutting edge technology our competitors can’t touch.  My role has deliberately been changing over the last year and I feel I owed it to myself and my family to explore “what’s next” for me.  So last week, I formally stepped back to a board of director position.  I will continue to help the company in any way I can and remain an active advisor, board member and chairman of the Click Quality Council.

As I reflect on the last four years or so of our growth, there are eight things that come to mind that I am most proud of:

1. We raised awareness about click fraud – Sure we may have ruffled a few feathers when we launched our Click Fraud Index ™ in the spring of 2006, but those numbers caused everyone to stand up and listen.  Google called us out in August of that year and dismissed the problem of click fraud as insignificant, claiming their own efforts to stop it were "reasonable".  The problem was hugely significant and no standards existed to stop it.  Something had to be done so we took the lead.  With the support of the advertising community, we “encouraged” Google to take the problem seriously and listen.


2. We brought “both sides to the table – No one wanted to talk about the problem of click fraud in 2006.  We recognized that solving this problem was in the interest of everyone in the advertising ecosystem.  We worked hard to listen to advertisers and work with search engines to develop sustaining solutions that are communally beneficial.


3. We built a bridge – In a groundbreaking partnership with Yahoo!, we built the FACTr™ system (Fully Automated Click Tracking and Reconciliation).  FACTr allows advertisers to communicate directly to the ad providers via Click Forensics.  Yahoo! led the way and by the fall of 2007, Looksmart, Google and others followed suit.


4. We saved advertisers money – Our tools and insight are effective in identifying unwanted traffic and allow an advertiser to work with the ad provider to get what they pay for.  Hundreds of customers have used our technology to find click fraud and improve their campaign ROI.


5. We led the charge for standards – Working closely with the Interactive Advertising Bureau (IAB) and dozens of online advertising companies we developed standards that were released in June of last year.  It was a team effort from Google, Microsoft, Yahoo!, Click Forensics and many others.  The result is a document that can be built upon for years to come.


6. We worked with some of the best – Our customer list includes industry-leading advertisers like Vegas.com, Progressive Insurance, Experian and EBay.  On the ad network side our clients include leaders like Adknowledge, Turn, Yahoo! and Lycos.  The relationships I have made through customers, partner companies and industry leaders means a lot to me (even Shuman!).  The online advertising industry is vibrant and growing.  Some of the brightest people I have ever been associated make this industry great.


7. We caught click fraud – We have an incredible team of scientists, engineers and developers. Many of them have PhD’s and tremendous industry experience.  We have put their knowledge to work by innovating ahead of the fast moving bad guys.  One example of this is our identification of the “Bahama Botnet”, an organized group of fraudsters using a highly sophisticated approach.  Our discovery of this (subsequently alerting Microsoft, Yahoo! and Google) saved advertisers from paying for these clicks.  Additionally, it forced the bad guys to change their approach.


8. We changed online advertising… for good – The work that we have done and continue to do is meant to have a sustaining effect on online advertising.  Improving traffic quality in real time means that publishers and ad networks can send out highly qualified clicks to advertisers and enhance trust.  This relationship, built on standards and delivered on quality, will accelerate the growth of online advertising for years to come.  We are proud to play an important part in the ecosystem.

I have enjoyed the opportunity afforded me to play a role in our company and the industry.  I'm deeply appreciative of the support, encouragement and hard work from our employees, partners and board of directors.  It's been an amazing five year ride!

And so now I look ahead.  Currently, I’m enjoying an “entrepreneurial sabbatical” to consider options and explore opportunities over the next few months.  I’m enjoying time with friends and family, traveling and learning about new businesses.  I’m keeping busy by consulting with an advertising technology company, joined the advisory board of an NBA agent firm and of course reading, writing and learning.  At some point, I’ll head back to Krispy Kreme.  There are an unlimited supply of donuts, coffee and problems to solve.  I can’t wait~

Tom Cuthbert

Click Fraud Gets Slicker

Andy Greensberg of Forbes has written about the problem of click fraud for years.  In fact it was Forbes who published a "debate" between Shuman Ghosemajumder of Google and me that helped set the record straight about discrepancies.  While the problem of click fraud has been widely covered by media outlets ranging from the Wall St. Journal to Search Engine Watch, Forbes (and Greenberg) have always been on the leading edge.

This is still the case today as Andy wrote and excellent piece on the problem titled, "Google Faces The Slickest Click Fraud Yet."  The article focuses on the work of Harvard professor Ben Edelman.  Ben's site (www.benedelman.org) is an impressive compilation of research and reports on a wide range of topics with a focus on spyware, click fraud and online advertising.

The article highlights "a new form of click fraud that accomplishes what online fraud watchdogs might have once thought impossible: a scam that not only simulates valid clicks on a Google ad sold to an advertiser, but seems to result in a real customer who spends money on the advertiser's site."  This is the worst case scenario for an advertiser and Edelman is right on in his method of identifying the threat.  While this approach is limited in it's ability to scale, it is the most significant finding since Click Forensics identified the "Bahama Botnet".

I applaud the work of Ben Edelman and others as well as Andy and the folks at Forbes for continuing to shine the light on this ongoing problem for our industry.  For over four years now, we have been on a mission to "ensure advertisers get what they pay for".  As long as Google lacks transparency, there will be click fraud.  Ben ends by saying, "Google owes its advertisers something better than that."  I agree and we need more guys like Ben to help hold their feet to the fire.

Tom Cuthbert

Beware the “Bahama” Botnet

By Steve O'Brien, Click Forensics

Just when you thought the fraudsters couldn’t get any more sophisticated … they surprise you.  Click Forensics researchers have recently discovered one of the most advanced sources of click fraud we’ve seen.  We’ve named it the “Bahama botnet” because when first discovered it was redirecting traffic through 200,000 parked domain sites located in the Bahamas.  It has since been reprogrammed to redirect through other intermediate sites hosted in Amsterdam, the U.K., and even San Jose, CA, but the Bahama name stuck.

Interestingly, the Bahama botnet appears to be closely related to the recent spate of “scareware” attacks, such as the one perpetrated against The New York Times digital site just a few days ago, reported by ComputerWorld.  Visitors to the NYTimes.com site were greeted with a pop-up informing them their computer was infected and directed to an authentic-looking site where they could install a program called Personal Antivirus.  Users duped into purchasing this phony software were then infected with a Trojan that gave control of their computer to an unknown third party that we now know to be part of a gang in the Ukraine.

We believe the Bahama botnet is controlled by this same gang, or their neighbors down the street.  More info about the “Ukranian fan club” can be found in Dancho Danchev’s excellent security blog.  We’re pretty sure the Bahama botnet is related to the Ukranian fan club and the NYTimes.com scareware because they each phone back to a bogus “Windows protection” domain located on the same IP address.

These sources were originally identified by the Black Hat community, but we believe Click Forensics is the first to discover the breadth and depth of click fraud being perpetrated by the botnets it controls.  And the botnet is incredibly insidious.

The video below shows the botnet in action, caught on film and narrated by Click Forensic’s own Matt Graham, the infected machine will exhibit some really funky behavior.  Clicks on organic search results are redirected through a series of parked domains across a number of top-tier ad providers (search engines and ad networks), eventually arriving at an advertiser unrelated to the original query.  The user is momentarily confused, but likely just performs the search again, this time with easy success.



What makes the botnet so insidious is that it operates intermittently so that the user doesn’t really know that anything is wrong.  Additionally, it can operate independently of the user because the authors appear to be building a large database of authentically user-generated search queries.

[caption id="attachment_718" align="alignright" width="500" caption="Seemingly random clicks discovered through advanced pattern detection"][/caption]

And because the queries come from many different machines (IPs) across a broad segment of the Internet population, it is very difficult to find and identify these clicks as fraudulent.  But these auto-generated clicks were not able to disguise themselves well enough to escape Click Forensics anomaly detection algorithms.  Additionally, large amounts of non-converting clicks were spotted in the data we receive from advertisers.  From there, our team was able to hone in on the source of the Bahama botnet.

The Doctors Are ‘In’

In February of 2006, Click Forensics was just getting off the ground.  We recognized the problem of click fraud was a big problem and that building a solution would be tough technical challenge.

[caption id="attachment_709" align="alignright" width="248" caption="Dr. Tuzhilin with the Click Forensics founding team in 2006"][/caption]

We decided to bring in an expert in the field of data mining and anomaly detection in click stream analysis.  That expert was Dr. Alex Tuzhilin.  Alex spent the day with us at our offices in San Antonio and provided us a road map for the evolution of our approach to identifying invalid traffic.  His contribution to us at that point was essential and provided tremendous insight.  After reviewing our approach he commented,

"Click Forensics has good data and this is a source of their advantage over the search engines. My role is to work with them to refine the scoring methodology to improve accuracy. Their approach is to incorporate as much data as possible to improve accuracy. The search providers simply don't have enough data to have the most accurate approach."

Shortly after Alex visit to Texas, I received a call from the lead attorney representing Lane’s Gifts in their lawsuit against Google.  He said, “Tom, I just hired your Ph.D!”  He told me that the judge in that case had mandated that an outside consultant review Google’s click fraud detection methods and publish paper on the efficacy.  Alex spent many weeks at Google and wrote an insightful paper detailing their approach, ultimately describing it as “reasonable”.  The Lane’s Gift case was settled and Alex returned to his role as a professor at NYU.

Today we are thrilled to announce that Dr. Tuzhilin has joined the Click Forensics Advisory Board.  Few individuals have had more real-world and academic experience in the measurement of online traffic quality and its effect on advertisers.  His work has helped move the industry toward standards and cooperation.  After visiting us in Austin a few weeks ago and meeting with our technology team, Alex said,

“Having firsthand experience reviewing the state of the art in ad network traffic management, I was impressed with the level of technical sophistication the team exhibits and I was impressed with the directions they are going, Click Forensics has played a leadership role in helping the online advertising community to monitor quality of clicks on ads, including identification of invalid clicks. I look forward to continuing to work with the team.”

In additional to Dr. Tuzhilin, we have also added Dr. William Wright, the Chief Scientist at Paypal.  Dr. Wright, a Ph.D. in cognitive science, is an artificial intelligence expert who has built numerous analytical and predictive systems over the past twenty years, including the Falcon Credit Card Fraud Detection System at HNC, the Advanced Fraud Screen system at CyberSource, and numerous adversarial modeling systems for the U.S. military.  After spending time with our team, William concluded,

“Click Forensics has built a strong team of developers using very advanced machine learning and data mining techniques to detect fraud and measure traffic quality, they are pioneering a new area of fraud detection and I’m finding it satisfying to work closely with them on leveraging lessons from my past experience combating credit card and banking fraud.”

One out of every five employees at Click Forensics hold a Ph.D.  Adding the expertise of Alex and William dramatically enhances our ability to meet our goal of providing the state of the art approach to traffic quality management.  I appreciate their contributions and look forward to benefiting from their knowledge in the future.

Tom Cuthbert

Building on a Foundation of Success: IAB Guidelines

Over the past week four major players in the online media space have announced accreditation to the Interactive Advertising Bureau’s Click Measurement Guidelines.  This list includes Yahoo!, Google, Microsoft and Business.com.  I wanted to take a moment and explore why you should care about this development and what accreditation means for advertisers.

The IAB is a publisher-focused organization that has led the process to develop click measurement guidelines.  The task force is made up of thirty or so companies representing the online advertising community.  Click Forensics has been a member since day one and participated in every step of the process.

There are three main benefits for advertisers and conversely, three concerns advertisers need to keep in mind associated with the entire process.  First, the benefits;

IAB Accreditation Represents a Commitment
The process to become accredited to the IAB guidelines is time consuming and certainly not free.  At Click Forensics, we have first hand knowledge of this and can assure you that any company that takes time and spends the money to become accredited is committed to their customers.  The level of detail the auditors go in to is amazing.  Our community is fortunate to have auditors that have demonstrated a deep commitment to both the development of the process and the implementation of the guidelines.

IAB Accreditation Demonstrates Leadership
The IAB established a gating period to allow member companies and others to become accredited to the guidelines.  The companies mentioned above were the first to announce compliance.  This is important because it represents a sense of urgency among these four that enhances the urgency for others.  As an advertiser, you should reward these leaders with business.  They were first out of the gate and in my book that demonstrates leadership.

IAB Accreditation Means Better Quality Traffic
The IAB Guidelines are a lengthy narrative of “best practices” and rules in delivering quality traffic to advertisers.  While it is not intended to be a complete list, it serves as a firm foundation and includes practical steps to help ensure advertisers get what they pay for.  By working with an accredited ad provider, advertisers will be assured that the clicks they are buying have met the guidelines established by the industry.  This is a good thing and an excellent first step.

While we applaud the efforts of the IAB, Media Rating Council and member companies who participated in this process, there are things advertisers need to keep in mind.  There was a great deal of discussion and debate during the nearly three years of meetings it took to develop these guidelines.  In that process, there were a lot of valuable and important items that fell to the floor.  This is a good start, not a perfect process.  Keep in mind the following;

IAB Accreditation is a “Moment in Time” Process
The process for an ad provider to become accredited is a long one.  The auditor is invited in for a pre-assessment then the actual audit begins.  At the end of the process accreditation is awarded.  The problem is there is no mechanism for ongoing compliance.  When we buy gas at the gas station there is a meter that is routinely calibrated to ensure that when we fill our tank with 20 gallons of gasoline, we get 20 gallons.  This approach is not taken nor addressed in the guidelines.  While an annual audit is suggested in the guidelines, it is still important for advertisers to be monitoring their campaigns and holding the ad providers feet to the fire for every click.

IAB Accreditation Does Not Cover Everything
The 27 page Guideline document is quite comprehensive.  Our task force worked hard to ensure that both the guidelines are made clear and that the standard for measurement is defined.  However, when you consider that the dominant constituency in this process was multibillion-dollar ad providers, you might imagine not everything met their liking.  A few examples of chaff that hit the threshing room floor included:

Click ID – Each click should have a unique identifier so investigations can be “apples to apples”
Persistent Cookie – It’s important that ad providers can identify unique visitors to ensure they are billed for only once.
Standards for Investigation – Advertisers deserve to feel confident that they get what they pay for.  By setting an investigation format and agreeing to a timeline, ad providers can build trust with customers.

IAB Accreditation is a Roadmap
There is a Japanese proverb that says, “Beginning is easy and continuing is hard”.  There is truth in this as it relates to the guidelines.  We have begun the process.  We have released guidelines that will make the world of online advertising a better place.  Now we should look to leadership to take the next step and continue what we have begun.  The current guidelines will serve as a roadmap to the future standards.  We need to examine the items removed, listen to the community and think of better ways to ensure advertisers get what they pay for in the future.  The roadmap has been built.  Now we need to move on.

In January of 2006 as Click Forensics was just beginning as a company, I wrote the following challenge to our industry:

“Define standards for what an unwanted click looks like. We believe that there are certain characteristics or attributes that are common to a large percentage of click fraud. We are working with publishers and advertisers to agree on common ground and work together to expose it. Once this is developed it should be published so that the entire community can benefit from it.”

Today, over three years later, we have the cooperation of community leaders, the foundation of technical standards and the desire to continue to improve on what we have built.  I invite you, to join us as we build a future of ongoing growth and improving effectiveness by enhancing the process of online advertising.  I can assure you that both the Click Quality Council and Click Forensics will continue to support the work of the IAB and other industry organizations to work together to make our community a better place.  Let's not stop with the foundation.

Tom Cuthbert

Scareware… the Next Internet Ripoff

From spyware to bots to viruses and other unimaginable hazards… the web can be a scary place.  As far back as Prodigy in the early days of the online world, scams have been a part of the party.  The Internet is simply a new way for the bad guys to rip off unsuspecting consumers.  The key difference though is that the reach is enormous and the damage can spread to more people, more quickly than ever before.

Enter scareware, new way to trick unsuspecting consumers into parting with their money.  USA Today recently had an article about the tricks and tactics used to perpetrate this latest rip off.  Unfortunately, online advertising has become an accomplice to the crime.

Scareware is worthless software that allegedly removes viruses from your computer.  Anyone who has surfed the web knows how easy it can be to become infected with a virus.  The damage to the users computer is often measured in slowed performance, unwanted clicking and potentially even more nefarious things like key logging and password swiping.  Now, the bad guys are selling “scareware” to solve a problem that may not actually exist.

The first such program was called “SpySheriff,” built by a team of cyber crooks from Russia.  The Anti-Phishing Working Group recently reported that scareware infections rose 48% in the second half of 2008.  The growth is tied to the ease of distribution and weaknesses in online advertising and the web in general.

There are several ways these fake products are being distributed.  Phony pages are created using hot search key words such as “American Idol” or “iPhone” and drive the unsuspecting consumer to the infected page.  Recently the Facebook email scam was used to send people to a page by promoting things like “best video.”  Since these emails came from your friends, millions clicked.  Twitter has become a vehicle for distribution. Phony Twitter accounts are created and enticing titles of posts encourage people to click.

Additionally, the bad guys are simply buying display or search ads.  They rotate in infected pages to the landing page.  It is virtually impossible for an ad provider to scan every ad impression and linking page.  This loophole creates an opportunity for the bad guys to drive significant traffic to infected pages at a very low cost.  Microsoft reported finding 4.4M installations of one such program, so the scale is enormous.  Do the math… at $49 or $79, that is big business.

Once someone lands on the page, getting off is nearly impossible.  Immediately upon landing, a “system scan” begins.  The results are, of course, showing that your computer is infected with a number of viruses.  Conveniently you can buy the product at that point and they take your money and run.  If you try to move away from the page, or cancel, an endless number of scans take over your screen.  Essentially, users must “control/alt/delete” their way out or restart.

The danger in this scam is not limited to monetary damage to the consumer.  These type of pages and methods to attract clicks are the same methods used to install spyware, malware and perpetrate click fraud.  To their credit, USA Today has done a good job over the last few years of highlighting the dangers of the web to the average consumer.

The FTC is cracking down.  They have identified products like WinFixer, DriveCleaner and XP AntiVirus as worthless and they are going after the owners.  The problem is that like the click fraud crooks, these guys are in remote locations and move their servers often. Tracking them is a full time job and extremely difficult.  The search engines are trying to help as well.  Bing has a neat feature that highlights “at risk” url’s.  Yahoo has similar product built with McAfee.

Trust is what keeps consumers clicking on ads.  Without stepped up industry efforts from organizations, like the Anti Phishing Working Groups and others, trust could be diminished.  Like click fraud, scareware is damaging trust.  It takes a community effort to stay after the problem and build solutions to take the scare out of the internet.

Tom Cuthbert

The Buzz on Click Fraud

The New York Times ran a feature article this week on click fraud.  Why you ask?  Because, like spam, click fraud is still a big problem for advertisers. The article pointed out that as the economy tilts downward, advertisers cannot afford to waste dollars. This is a good news, bad news scenario for online advertising.

The good news is that online advertising is highly measurable.  Large advertisers that traditionally have been offline are now shifting dollars online.  This fact has contributed to online advertising continuing to grow as traditional media is in decline.

The bad news however, is that this window of opportunity is narrow.  The online advertising community must embrace measurability and enhance trust to gain share of spend from the big guys. 

There was a significant event this week that helped in that effort.  The Interactive Advertising Bureau (IAB) released from draft the Click Measurement Guidelines.  This document, three years in the making, is a great start for our community to come together around standards and enhance trust. Dozens of ad providers are busily working with third party audit firms to become accredited to the new guidelines.  Advertisers will have a way to gauge the level of commitment from ad provides when this list is made public.

Click Forensics was proud to represent advertisers in this process.  In fact, we were the only traffic quality management firm to participate and were quoted in the press release from the IAB.  Many thanks are in order for the 38 members of the working group for a job well done.

Now, we find ourselves at the beginning.  An opportunity exists to build on the foundation laid by the IAB member companies.  Click fraud is going to be a problem for a long time to come.  Progress is being made.  But in order to re-accelerate the growth of online advertising we need more than standards.  We need a community effort to work together to ensure advertisers have confidence that they get what they pay for.  Articles raise awareness, documents create a process and awareness builds urgency.  But ultimately it will take the effort of everyone in the community to get to the day where trust is commonplace and online advertising becomes the marvelous, measurable media it can be.  We look forward to continuing our efforts toward that goal.

Tom Cuthbert

SES New York 2009

I attended the Search Engine Strategies conference in New York this week.  It was a very different show than last year.  The crowds were down, keynotes were less engaging and more people that ever are dressing up.  

One other notable change, there was no click fraud panel.  It was just a year ago that the Search Engine Watch said the top story coming out of SES Day 1 was "Yahoo cozies up its click fraud critics".  Too bad, because click fraud is certainly a topic on the minds of advertisers, agencies and publishers.  One blogger noted, " In these hard times, it's more fun to talk about silly tweets than evil cheats."  Maybe true but "silly tweets" are only wasting time, not advertising dollars...

I did enjoy participating on my panel titled, “Pay-for-Performance: Winning Strategies for Advertisers and Agencies.”   I've posted my presentation below.  Additionally, a video recap can be found here.
[slideshare id=1195434&doc=sesnyc090324-090325080029-phpapp01]

 

Tom Cuthbert

How Botnets Take Control

The problem of botnets is getting worse, not better.  In fact, over 30% of all click fraud comes from botnets, maybe even YOUR computer!  Ever wonder how botnets work?  The BBC has acquired control of 22,000 computers and have been demonstrating how it works.  The videos are easy to understand and  very interesting.

Cyber crime risk exposed
How cyber criminals attack websites
Is your PC doing a hacker's dirty work?

Click fraud is costing advertisers millions of dollars a year.  So how can you protect your computer from becoming a party to the crime?  Again the BBC site has an excellent article with practical steps called, "How to keep your computer secure".  Take time to read it and be sure you are doing your part to reduce click fraud.  

Tom Cuthbert

IAB Click Measurement Guidelines

On the first day Click Forensics was formed, January 2nd, 2006, we called for standards to be developed around click measurement.  Our work and our focus has been to ensure advertisers get what they pay for.  There has been a need for standards and guidelines to help ensure this happens.  Today, the Interactive Advertising Bureau (IAB) released the highly anticipated Click Measurement Guidelines.  This document is the result of a task force that took form in September of 2006.  Click Forensics was a founding member and along with Google, Yahoo, Microsoft, LookSmart and other members of the IAB, the document is ready for public comment... your comment.

Since the IAB is a publisher driven organization, we have felt a responsibility to advertisers to ensure your voice is heard throughout this process.  During the public comment phase, we want to help encourage and inject advertiser comments into the process.  It is important for you to take time and review the document.  Think about the benefits that it may have to you as a buyer and question aspects that are missing.  All ideas are good ideas and the IAB and our entire task force are eager to get feedback.  You can download a copy of the document and provide comments directly to the IAB here.  You are also welcome to provide feedback to us directly by commenting in the box below.  

[contact-form]

 The Guidelines are an important  foundational step for our industry.  While no one document can solve the problem of click fraud or mitigate invalid traffic, it is encouraging to us to be at this point.  We look forward to continuing to work to represent your voice and contribute toward building standards that enhance the value for online advertisers.  We are grateful to the IAB, the Media Rating Council and all the members of the task force for the hard work put in over the last few years.  Together we are making meaningful progress in building trust and making online advertising more transparent and effective.

Tom Cuthbert
President and Founder

Insight into the Jump in Click Fraud

This morning, Click Forensics released the Click Fraud Index numbers for Q4 2008.  The overall rate jumped to 17.1%.  This means that across our community of advertiser data, we find that one out of every six clicks are invalid and not worth paying for.  There were three factors that drove the increase.

[caption id="attachment_303" align="alignright" width="300" caption="Source: Click Fraud Index"][/caption]

First, the increase lines up with past increases in Q4, somewhat of a seasonal bump.  This year however, was magnified by the economic downturn and a correlating increase in cybercrime.  I wrote about this last week.

Secondly, there was a 14% increase in the botnet rate, surpassing 30% for the first time.  There have been quite a few reports from McAfee, Symantec and others noting a similar trend.  Botnet attacks are becoming more sophisticated and difficult to stop.

Finally, for the first time in awhile, we saw an uptick in click farm activity.  Unfortunately, there lots of sites that pay people to click on ads.  This type of activity is difficult to catch and unfortunately, advertisers lose when this occurs.

While the jump is alarming, there is a growing current of cooperation to address the problem.  At Click Forensics, we continue to work with advertisers, agencies and ad providers to build solutions to ensure advertisers get what they pay for.  

Coverage of the news can be found here:

CNET

WebProNews

MediaPost

TechCrunch

New York Times 

Search Engine Land 

Search Engine Watch 

Tom

Getting an Uptick in a Downturn

It's no secret our county’s economy is slowing and times are getting tough.  While there is great optimism, there is an undercurrent of concern.  In anytime of economic downturn, one unwanted byproduct is that crime increases ("Statistics point to increase in crime").  Unfortunately, this includes cyber crime and specifically, click fraud.

For almost three years now Click Forensics has been tracking click fraud.  While the overall rate somewhat stabilized during 2008, the number of advertisers affected and the dollars lost continue to rise.  As we face more sophisticated attacks in 2009, I wanted to highlight some recent advances in the battle and share some specific steps advertisers can do to ensure they get what they pay for.

Cooperation has been a theme for 2008.  Progress is being made on three fronts by leaders in the industry.  First, the Interactive Advertising Bureau (IAB) continues its work on defining guidelines for counting clicks.  Click Forensics continues to take an active role in this important effort and supports it as a good first step.  Secondly, leading search providers, including Google and Yahoo!, have enhanced their products with better tools to manage campaigns and announced traffic quality centers to provide additional resources to advertisers.

The third initiative has been the joint effort of Click Forensics and Yahoo! to build the FACTr process (Fully Automated Click Tracking Reconciliation).  For years, advertisers have been frustrated when they have found invalid activity in their pay per click campaigns.  Now there is a simple, automated process that connects advertisers to the ad providers.  The FACTr system was built by Yahoo! and Click Forensics and launched publicly in July of 2008.  By the fall, other ad providers including LookSmart, Miva and Google were added.

While progress continues, here are specific steps advertisers can take to protect their online investment.  First, advertisers should monitor campaign performance at the most granular level. Click fraud attacks come in spikes of activity.  By watching campaign performance on a daily basis, advertisers can see anomalies and alert the ad provider quickly.  Looking for spikes in clicks, drops in conversion ratios, and higher than normal impression levels are all signs something may be out of line.

Secondly, now is a good time to double check campaign settings.  A large number of unwanted clicks come as a result of campaigns not being set up correctly.  Look at geo-target settings to ensure they match your campaign goals.  Review day part settings to ensure you have optimal ad delivery for delivering quality traffic.

Finally, avoid paying for clicks from low quality traffic sources.  By utilizing the site exclusion functionality made available by ad providers, you can block bad clicks from ever getting to your campaign.  Click Forensics has a process called, Intelligent Exclusion™ that dynamically identifies bad traffic sources and eliminates them from the campaign.  We see a 43.5% decline in the overall invalid rate for advertisers who use this process.  Keeping the money in your pocket is always better than having to go back and ask for a refund!

Like you, we are hopeful that the economy will make a quick recovery.  As it does, we will all benefit.  In the meantime, it’s more important than ever that advertisers are on the lookout for threats to their ad budgets.  You can count on all of us at Click Forensics to continue to work on behalf of the entire industry to bring solutions to the marketplace to ensure advertisers get what they pay for.

Tom

“The Happiest Place on Earth”

The domain industry is fascinating.  I have enjoyed being involved in events that are both educational and productive for Click Forensics and me.  The latest event is the TRAFFIC conference held this week in “The Happiest Place on Earth”, Disneyworld of course!  Everyone seemed happy (even though there wasn’t a Starbucks anywhere to be found!).

The domain industry appears to me to be at a crossroads.  While the value of names continues to rise, the earnings from monetization programs are falling.  Domain owners and parking companies are struggling to gain more transparency from Google and Yahoo.  Today, there is essentially no transparency.  Traffic from domains is sent up to Google and some amount of money is paid for the traffic.  While agreements regarding revenue shares can be negotiated up front, without the ability to see inside the black box, the monetization metrics are a mystery.

One speaker at this conference has a deep understanding of this issue.  Michael Gilmour runs whizzbangsblog.com from his home in Australia.  I heard Michael’s presentation and had a chance to speak to him at an after party.

He spoke candidly about the risks that the industry faces from the search engines black box approach.  “The lack of transparency in the whole process means that they are accountable to no one.”Gilmour said.  He accurately pointed out that, “Google has been progressively reducing its network traffic margins from a high of 22.1% (Q1 ‘06) to low of 11.9% (Q1 '08).”

What this means is that parked domain companies and site owners are being squeezed.  This is a trend that will continue.  Advertisers are demanding higher quality traffic and Google has had a hard time delivering that from low quality traffic sites like MySpace.  Enter the parked domain channel.

Gilmour has written a series of blogs addressing this issue.  The eight part series can be found on his site at www.whizzbangsblog.com.  In it Gilmour says, “Google is able to launder a lot of bad traffic with good traffic and make it all pay the same while they themselves can discriminate on what they pay out.”

There is a lot of great quality traffic that comes from direct navigation domains.  This is an industry with lots of smart folks and great ideas to help advertisers sell more stuff.  It will take cooperation and transparency to build value in the domain space.  As Gilmour says, “Unless they (the parking companies) are able to audit Google then they can't ever be assured of their share of the revenue.”

Without cooperation, transparency and standards, future conferences may not be as happy as this one was.

Tom

Under the Iceberg

Over the past two years we have been trying to bring attention to the real danger of click fraud.  It is a real problem that is getting worse not better.  Since we began reporting our Click Fraud Index, the overall rate has climbed over 20%.  This problem has been highlighted in mainstream publications including Business Week, USA Today and the Wall St. Journal.  No one today denies that click fraud is a problem and that it is having a negative effect on the growth of the online industry.

What may be less obvious is that click fraud is only the tip of the iceberg.  The bigger issue for our industry is the overall decline of traffic quality.  Advertisers want to get what they pay for and know that the traffic they buy has value.  Problems including: the growth of botnets, out of country traffic and other low quality traffic sources like made-for-ad sites and parked domains are hurting ROI.  Advertisers know this and have been demanding action from ad providers.  One recent example is the poor quality traffic that comes from social network sites like MySpace.  Google surprised Wall St by missing Q4 earnings due, in part, to their inability to monetize MySapce traffic.  Social network sites are notorious for having very low SiteScore’s.

So what is happening?  Smart ad providers are taking matters into their own hands.  They can’t afford to lose business and they are listening to their customers.  By using real time tools to detect invalid traffic, publishers can block it, redirect it or re-price it.  This is the way the industry is moving and we are working hard to lead the way.  It is encouraging to see ad providers like Yahoo see the dangers ahead and help their clients steer clear.  On the other hand, it is concerning others are on a collision course. 

Why Yahoo! Matters

A couple of weeks ago at Search Engine Strategies in New York, Click Fraud made big news.  Yahoo announced a partnership with Click Forensics that changes the tone of the ongoing “Click Fraud Debate”.  Since late 2005 there has been denial, litigation, finger pointing, 17 page reports and lots and lots of media coverage around the topic of click fraud.  In March of 2006 I wrote that, “It will take a community approach to solve the problem”.  Since then the community of advertisers, agencies, third parties, publishers, ad networks, industry organizations and search providers has grown.  We have all been drawn closer together to help solve the problem; not deny its existence.  A visible result of that progress was the Yahoo announcement.

It was just over a year ago that I first met Reggie Davis.  Yahoo was the first search engine to name a Vice President over Marketplace Quality.  Reggie’s approach has been refreshing.  He listened carefully and took notes to what advertisers were saying.  He worked with his team to implement solutions that helped improve transparency.  And now, less than a year later, Yahoo is the first search engine to work with a third-party to fight click fraud.

This is real progress and a sign of things to come.  For quite some time we have been drawing the distinction between traditional media (TV and Radio) and online.  Advertisers are used to having standards, auditable invoices and notarized affidavit’s confirming they get what they paid for. It’s especially important because as pay-per-click industry expert, Dr. Tuzhilin, has noted: third-parties have access to data search engines don’t have. And that information is helpful for identifying quality issues such as click fraud. Yahoo! understands this and we’re now able to share this information to help Yahoo! help its advertisers.

Stealing Clicks

Forbes reporter Andy Greenberg recently published a story with dueling perspectives from my friend Shuman, of Google, and me highlighting our differing views of the click fraud problem. It got me thinking about how far the community has come toward solving the click fraud problem and how far we still have to go.

It was just a little over a year ago when search engine giants – Google, Yahoo! and Microsoft – publicly pronounced their support for the development of industry standards for measuring click fraud with the Interactive Advertising Bureau (IAB). Around the same time they also promised to work directly with third-party click fraud auditors on solutions to the click fraud problem. Although many were skeptical at the time, we thought these promises were important steps in the right direction. After all, just a few months earlier search engines, like Google, labeled click fraud “immaterial.”

Looking back, however, it’s clear not much has been done by search engines to turn these two important promises to advertisers into action. The standards development process, which we also joined to help ensure the voice of advertisers was heard, has lumbered along. We’re still waiting for the IAB to deliver its recommendations for click fraud measurement standards and open them up for public comment. We’re also waiting for Google and the other search engines to work with third-parties. While they have launched traffic quality resource centers and done lots of talking, there has been little meaningful action.  

Meanwhile, back at the ranch, traffic in the publisher networks (including Google AdSense) is abysmal.  Over 70% of the sites that make up these networks are made-for-ad sites or parked domains. Well over 60% of the traffic from these types of sites is traffic advertisers should not be paying for.  Instead, advertisers should be actively excluding poor performing sites and avoiding low quality ad networks.  But it’s is daunting without the help of search engines.  Knowing which ones to exclude is a dynamic process that is tough to do when your information is limited.

So the question is, why the delay by search engines on all these fronts? There are many possible reasons but I believe the main one is economic incentive. When a company doesn’t have an immediate or near-term financial incentive to change, then it won’t.  One only needs to look at the stock prices of certain search engines to see the economic incentive just isn’t there.

I do believe there is a light at the end of the tunnel. Advertisers are starting to pull back on their online spending because of quality issues. It’s only a matter of time before search engines feel the pinch. The spark may just well have been lit by Google’s recent success.

Related Article:

Forbes.com

Stealing Clicks

Forbes reporter Andy Greenberg - 09.24.07, 6:00 AM ET

Profiting from Click Fraud

A day or so ago, Shuman Ghosemajumder from Google posted two blogs on his personal site questioning Click Forensics data, methodologies and motivation. While it is really unfortunate that we have to take time to address these issues, I will. It is unfortunate because Shuman and I have had numerous phone conversations, email exchanges and meetings over the past nine months and we discussed many of the topics he brought up in his post.  Let me begin…

1) Many third parties have not counted clicks properly
Click Forensics counts clicks properly. We do not count back clicks. We count page views correctly. As a matter of fact, we use click ID’s (including Google’s) and other methods to ensure that our reports are the most accurate in the industry.

2) Inflated click counts result in even more inflated “click fraud” estimates
Google calls them “invalid clicks.” Click fraud is a subsection of this category and does include malicious attacks, competitive clicks and affiliate fraud. Shuman has said on many occasions that Google does not charge for invalid clicks. This is a game of semantics. Our report focuses on what we call “high threat level clicks.” These are clicks that our rating engine and algorithm identified as being unwanted by the advertiser. These are clicks that advertisers should not be paying for and the reality is that in Q4 of 2006 14.2% of all clicks we scored fell into this category.

3) Even if they fixed those problems, they’re not actually measuring click fraud
Actually the truth is Click Forensics measures and reports on all suspicious click activity and attempted click fraud across search engines and their content network advertising channels. Google has admitted that is has not charged for some clicks. Shuman told Andy Beal that this number was .2% of all clicks.  Our advertisers (and Google’s) can see the number of clicks that they credited and it is far lower than the number of unwanted clicks search engines credit them for in their campaigns. As a matter of fact, many of the 3,000 advertiser members of the Click Fraud Network report that they have never been discounted more than 2 percent for the invalid clicks. It is clear search engines are not catching all bad click activity. Part of the reason is their lack of advertiser-side information and the difficulty in policing content networks alone.

4) Industry metrics are not necessarily the same as Google’s metrics
True.  Our industry metrics a representative of virtually every search provider large and small in the industry. This is why we have a statistically accurate view of the true rate of industry click fraud levels. We used to separate our click fraud figures by search engines tiers but stopped reporting this way this quarter because there is a growing disparity between individual search providers as to the click fraud levels. We didn’t think it was fair to lump the good with the bad so we will begin to report on individual providers later this year. The results may surprise you.

5) ROI on the content network is the same as it is on search
That is an interesting way to spin the story.  The fact is that content networks produce a higher amount of fraudulent and invalid click activity.  We reported that 19.2% of clicks coming from content networks are fraudulent and fall into the high threat category. I’m glad Google agrees with us on this one.  Its clear advertisers have been justified in their views on the click quality of content networks because the threat level is so high. We help advertisers identify publishers in the content networks that they should avoid. This approach actually helps improve the ROI on all content networks and gives advertisers peace of mind, thereby accelerating the growth of pay per click advertising.

Shuman’s first post ends with this, “The key point here is not that their (Click Forensics) numbers are "too high". The point is that their data collection methods are inherently flawed and any resemblance their numbers could have to reality would be coincidental. Even so, given that they are not measuring click fraud, they apparently don't intend their numbers to reflect reality.”

At Click Forensics we stand by our numbers, our methodology, our people, our clients, our partners and the 3,000 plus members of the Click Fraud Network.  And we are not alone in defending our approach.   In March 2006, our methodology was examined by Dr. Tuzhilin, the same expert who examined Google's click fraud practices and wrote the report as part of the $90 million click fraud settlement case.  Dr. Tuzhilin said:

“Click Forensics has good data and this is a source of their advantage over the search engines. My role is to work with them to refine the scoring methodology to improve accuracy. Their approach is to incorporate as much data as possible to improve accuracy. The search providers simply don't have enough data to have the most accurate approach."

The final false accusation from Shuman, responding to a post, charged us with releasing data to  “…promote flawed industry estimates in order to attract business.” 

Nothing could be further from the truth. 

We are not compensated based on how much fraud we find.  But each time a fraudulent click occurs and is missed by a search engine, they get paid for it.

Our goal is to continue to build a great company than helps advertisers ensure they get what they pay for in the tradition of Nielsen and Arbitron.  Google would do well to take a less confrontational and more cooperative approach embraced by their competitors.  That said, I look forward to discussing these issues with Shuman and continuing to work together to accelerate the growth of online advertising by fostering trust and accountability.

 

 

SES Chicago 2006

I am on my way back from Search Engine Strategies (SES) in Chicago.  What could have been more fun than another click fraud panel this time in sunny Chicago in December!  Honestly, I have been looking forward to participating with the same group we had in San Jose last summer.  The discussion was lively and I have a deep respect for the other panelists.  

Jeff Rohrs did another exceptional job laying out the issues surrounding click fraud.  Since we last met there have been several major developments around this topic.  I thought I would update you on those as well as provide you a recap of SES.

First is the work of the IAB and the Click Measurement Working Group.  The process is underway to help define industry standards surrounding click fraud.  Click Forensics is deeply involved, having met in person with both the IAB and the Media Rating Council (MRC).  We take our role seriously and have dedicated time and resources to help this important initiative.  All four major search engines are engaged as well as other industry players.  John Slade from Yahoo! gave a brief recap saying that two meetings have been held and all the stakeholders are working well together.

Secondly, the formation of the Click Quality Council (CQC) signals advertiser interest in the issue of click fraud.  The CQC held its first meeting the week of SES and will be focused on developing solutions from an advertiser perspective to improve click quality for our industry.  The CQC and the 3,000+ members of the Click Fraud Network represent a large percentage of all online spending.  It is critical that their collective voices are heard as we work together to improve the industry.  I pointed out on the panel that the Wall Street Journal recently reported that online advertising revenue growth is flattening.  Clearly, part of this is due to a lack of standards, clear measurement standards and a deficit of transparency into the process.  My hope is ’07 will provide some movement on these issues.

As you know, Q4 represents the most active quarter of the year for pay per click advertising.  The Click Fraud Index released November numbers indicating the click fraud activity increased to above 14%.  This is not surprising given the increase in activity.  If you have looked at the increased spam in your inbox you will understand!  Stay tuned for more information on this.

There was a great deal of discussion around the “friction” that exists for advertisers who find click fraud in their campaigns in submitting that information.  Several questions from audience members made known their frustration.  While both Shuman and John commented that they want to work with advertisers, it was not enough.  Our feeling is that there needs to be a process for advertisers to easily submit findings, understand the levels of invalid activity and have a defined reconciliation process.  Click Forensics will continue to work on this.

Lori Weiman and Jessie Stricchiola did a great job discussing the lawsuit updates.  Jeff mentioned the recent BusinessWeek article titled “The Vanishing Click-Fraud Case” (http://www.businessweek.com/technology/content/dec2006/tc20061204_923336.htm) and that sparked some good discussion. 

We all survived the 14 degree temperature in Chicago and all in all felt that SES was well worth the trip.  Thanks to my fellow panelists and I look forward to New York!

Announcing the Click Quality Council

I am pleased to announce the formation of the Click Quality Council.  Around twenty forward thinking advertisers and agencies will convene quarterly to review news, share ideas and comment on recent industry developments.  The complete list of members will be released prior to our first meeting in mid October.

Why is this important?  Because developing click measurement standards is a critical initiative and something we have been working toward for a long time.  In March of this year I wrote in this blog, “We need search providers to accept responsibility and work to build an industry standard.”  That process is well underway with the IAB Click Measurement Working Group.  Yahoo!, Microsoft along with other leading companies had executives engaged and in person and actively participating at the first meeting.  This shows a strong commitment from these companies in the effort of solving this problem.

I am in New York as I write this attending the MIXX and OMMA events and very pleased with the support we have been getting regarding the CQC.  The Business Week cover story made clear the risks involved in pay per click advertising.  We have been aware of those threats and continue to work full time on catching click fraud.  However, the real news is the progress being made together, as an industry, in developing standards to be enforced by independent third parties.

Thanks to the advertisers and agencies that have stepped forward to help with the CQC.  I look forward to reporting our progress in the coming months.

Tom Cuthbert

The Google Guy

It was the SES show in Chicago last December.  Not surprisingly, by the time the conference was over it was freezing cold and snowing sideways.  Welcome to Chicago!  I had just come from listening to the last session of the three day event, “The Click Fraud Debate”.  The panel included moderator Jeff Rohrs, John Slade from Yahoo!, Jessie Stricchiola, Lori Weiman and Shuman Ghosemajumder of Google.  It would be an understatement to say the discussion was lively!  I went up and met Shuman after the panel and we exchanged cards.  

I left the conference room ran upstairs to change into the warmest clothes I could find (jeans and my MIT sweatshirt).  I flew downstairs to jump in the car service to head off through the blinding snow to O’Hare.  Much to my surprise there was Shuman trying to hail a cab to the airport.  (Don’t ask me why a $6B company makes their executives take taxicabs to the airport, but that seems to be the case~)

Shuman (an MIT alum) noticed my sweatshirt and we began chatting.  I offered him a ride to the airport and so off we went through the snow hoping to make our flights out.  So there we were… the guy that has been handling the click fraud conundrum for Google locked in a car with and the president of a click fraud detection company.  Small world, huh?

  He strikes me as sincere and I have felt from the beginning that Google probably does have good intentions relating to click fraud.

This chance car ride was the beginning of an ongoing dialogue.  At our last meeting in San Francisco, we spent a great deal of time discussing things our companies agree on.  Somewhat surprisingly, it is a long list.  I believe eventually all the major search providers will open up to a third party solution for this growing problem.  It is inevitable and simply the right thing to do.  Becky Quick from CNBC keeps asking me when this will occur and who I think will be first.  I have no idea… but while Click Forensics patiently builds our company, online advertisers (including our API beta testers) seem to be growing more impatient.  

I am not expecting snow at SES in San Jose this week.  I am expecting another lively discussion tomorrow as I sit with Shuman on the click fraud panel.  And Shuman, if at the end of day you need a lift, I am here!  As a matter of fact, you can rest assured that we are not going anywhere.  Have car, will travel.  Have algorithm, will catch click fraud~

Tom