Bot Filtering

Every number a report suite produces rests on one assumption: that a hit arrived because a person did something. A meaningful share of them did not. What reached Adobe was software, and it was never going to convert, never going to bounce, and never going to read a word of what you published.

Almost none of it is malicious. Search engines have to crawl your site or nobody finds it. Uptime monitors have to request a page every minute or nobody knows when you go down. Your own testing tools, your own security scanners, and a long tail of scripts written by people you will never meet all knock on the same door a customer uses. The problem is not that they visit. The problem is that a page view is a page view, and nothing about a machine request announces itself as one.

Left in place, that traffic does not just add noise. It moves every number that has a denominator.

What automated traffic actually does to a report

Bots inflate the things that are easy to count and dilute the things that matter. Page views rise. Visits rise. And because almost no bot ever buys anything, fills in a form, or reaches step four of a checkout, every rate you calculate against those inflated totals falls.

Conversion rate drops without a single customer changing their behaviour. Bounce rate climbs, because a crawler takes one page and leaves. Average time on site collapses, because a machine does not read. None of these numbers look broken. They look like a bad quarter.

Nobody sold anything differently. The orders are identical. Only the denominator moved, and the denominator is the part nobody looks at.

The setting itself, which takes about ten seconds

Here is the part that surprises people who expect a hard problem: switching on bot filtering is a checkbox. Adobe subscribes to the bot list published by the IAB, the Interactive Advertising Bureau, an industry body that maintains a shared register of known automated agents. You tick the box, and from that moment every incoming hit is checked against that list before it is counted.

That is the whole configuration. There is no tuning, no sensitivity, and nothing to maintain: the list is Adobe's to keep current, not yours. Any hit matched against it is diverted before it reaches your reports, and it lands instead in a set of bot reports where you can see what was taken out.

Why it is a checkbox at all

A fair question sits underneath that setting. If removing automated traffic is obviously the right thing to do, and Adobe already maintains the list, why is this a decision anyone has to make? Why is the box not simply ticked the day a report suite is created?

Adobe had three ways to handle it, and the one it chose is worth reading properly.

It could have shipped nothing, which would say the product cannot recognise automated traffic at all. It could have switched filtering on by default, which would mean Adobe deciding on your behalf which of your visitors count, and removing them from your reports before you knew a decision had been taken. Or it could build the mechanism, hand you the switch, and step back.

That third option is what a checkbox is. It is not a missing default, and it is not laziness in the interface. It is a transfer of ownership, made deliberately.

Because the underlying question is not technical. Deciding that a class of traffic does not count is a judgment about what your organisation considers a visitor, and that judgment belongs to whoever has to defend the numbers when someone asks why the figures moved. Adobe supplies the capability and stays out of the decision. You tick the box, so the cleaner conversion rate is yours to explain, and so is the traffic that vanished from the year-on-year comparison.

It is a small piece of interface carrying a large message, and it is worth recognising, because it tells you what the product expects of you. The tool is built to be directed. Somebody has to sit in the driver's seat and say what counts, and the checkbox is Adobe making it unambiguous that the somebody is you.

What the list catches, and what walks straight past it

Now the part that matters, and the reason this section is longer than the configuration deserves.

The IAB list is a list of known bots, and known is doing a great deal of work in that sentence. A bot appears on it because it identifies itself: it sends a user agent that names what it is, it publishes its address ranges, and it behaves like a good citizen of the web. Search engine crawlers do this. Large, reputable monitoring services do this. They want to be recognised, because being recognised is how they avoid being blocked.

The traffic that distorts your reports most is usually the traffic with no reason to announce itself. A scraper lifting your pricing does not want to be identified. A script someone wrote to poll a page every thirty seconds has a default user agent nobody bothered to change. An internal load test fired from a colleague's laptop looks exactly like a browser, because it is one.

This is not a criticism of Adobe, and it is not a mistake anyone made in your implementation. A shared industry list can only contain what the industry has agreed to name. Recognising traffic that is actively trying not to be recognised is a genuinely harder problem, and it is one the product has not solved yet. Tick the box, because it is free and it removes real noise. Just do not walk away believing the job is finished.

Which is where exclude by IP earns its place

When you find automated traffic the list never caught, the practical answer is usually not a clever user agent rule. It is the blunter tool sitting a few rows up in the same settings panel: exclude by IP address.

It works because most of this traffic has a property the user agent lacks. It comes from somewhere consistent. A scraper runs from a server, and servers have stable addresses. An internal testing tool runs from your own office range. A monitoring script runs from wherever it was deployed and stays there. The behaviour hides; the origin usually does not.

Once you have a candidate, the exclusion itself is as simple as the checkbox was. You add the address, or a range, and every future hit from it is dropped before processing. Exclusion by IP is decided per report suite, like almost everything else in this module, so a rule added to one suite protects that suite and nothing else.

Filtering only ever looks forward

A rule takes effect from the moment it is saved. It does not reach backwards. Data already collected and already processed stays exactly as it is, bots included, permanently, because there is no reprocessing pass in Adobe Analytics that can revisit it.

In practice this is a smaller problem than it sounds, because the traffic you are removing is usually a steady background hum rather than a single dramatic event. Your history is not ruined. It is just slightly noisier than your present, and the step down on the day you enabled the rule is visible in any long trend.

Worth knowing before someone asks why last year looks busier than this one.

The report that tells you whether any of this is working

Filtered traffic is not deleted. It is set aside, and Adobe keeps a small set of bot reports showing what was removed and which rule removed it. This is the only feedback the feature gives you, and it is worth opening twice: once when you first enable filtering, and once a few weeks later.

The first look tells you the rules are matching something. The second tells you whether the volume is stable, climbing, or has suddenly stopped, and a bot report that falls to zero is far more likely to mean a setting was changed than that the bots went away.

What you have now

Bot filtering is a ten second setting wrapped around a problem that is not ten seconds deep. The checkbox handles the automated traffic that was willing to identify itself, which is worth having and costs nothing. What it leaves behind is the traffic that had a reason to stay quiet, and that part is found by reading your own reports for patterns no human would produce, then excluding it by address.

Neither half tells you it is working. The bot reports are the only instrument you get, and an exclusion that goes too far leaves no trace at all. That is the quiet theme running under every setting in this module: the panel accepts whatever you type, and the consequences arrive months later in a number nobody can explain.

One group of settings in the same panel is different in kind, because getting those wrong is not merely a reporting problem. Data retention periods, IP obfuscation, and the controls deciding what Adobe is permitted to keep about a person carry obligations that sit outside analytics entirely, and they are covered in Privacy and Data Retention.