SpikeStrip is a module for Apache 2.x that implements a new form of content access control for websites. SpikeStrip encrypts hyperlinks within pages served by Apache using the current browser's session key and a server-side secret key. Incoming requests for encrypted urls are decrypted and served as normal. This link encryption procedure creates a unique "view" of the protected website for each browser. Since each view is tied to a specific client browser, this prevents the collection of deep-links by web crawlers. This capability is useful for website operators wishing to prevent deep-linking to their content.
Because each "view" is unique to a specific browing session, SpikeStrip can perform rare-limiting on HTTP requests on a per-session basis. This approach to request rate-limiting is superior to IP based tracking since it can disambiguate users who are behind NATs and proxies. SpikeStrip's rate-limiting prevents unauthorized web crawlers from being able to index the protected website's content in a timely fashion.
The protection offered by SpikeStrip is 100% configurable. The IP addresses and hostnames of known good crawlers, such as Googlebot, can be whitelisted so that they will not be rate-limited. This enables your website to still be indexed in a timely manner by search engines. The URLs that SpikeStrip encrypts are also configurable through the use of regex statements. This enables website administrators to configure precisely which portions of their site receive SpikeStrip protection.
SpikeStrip is designed for high efficiency. In most cases, SpikeStrip imposes less than 2% CPU performance penalty on protected webservers, and uses less than 30 megabytes of RAM.