Thread: How can I detect a search engine spider using PHP

Basically I need a function that I can call that returns true or false if a spider is viewing the page. What is the best way to do this? I know I can compare user-agents, but I figured I'd check with you guys before I go out and spend time creating a solution to a problem that might already be solved here. Any code I can grab that will do this for me?

I'm entirely new to this, so please share even the basic ideas that I am missing, Thanks!

Do you want it in real time?

I did think that cpanel has this as a built in feature when viewing your websites traffic?

user agent is the only way

What are you using this for?

If it's for giving extremely different content to bots and users you're going to want to get a good list of bot IPs and identify them that way. Otherwise, you risk getting found out and blacklisted.

If it is simply for internal metrics, or some other small on-page modifications, then go for user agent. (But don't expect it to catch all the bots.)

use Cpanel, or if you want to put it on ur site, maybe ..

just gugling for PHP script for SpiderBot visit

The most common way on detecting bots/spiders is obviously via user agent. (PHP: $_SERVER['HTTP_USER_AGENT'])
But you can also usually detect via reverse dns. (PHP: gethostbyaddr). The hostname will usually contains strings should as "google", "bot", "spider", "yahoo", etc.
Hope this helps.