Cloudflare (NET) said in a blog post that it is observing stealth crawling behavior from Perplexity, an AI-powered answer engine. “Although Perplexity initially crawls from their declared user agent, when they are presented with a network block, they appear to obscure their crawling identity in an attempt to circumvent the website’s preferences,” the blog psot reads. “We see continued evidence that Perplexity is repeatedly modifying their user agent and changing their source ASNs to hide their crawling activity, as well as ignoring – or sometimes failing to even fetch – robots.txt files. The Internet as we have known it for the past three decades is rapidly changing, but one thing remains constant: it is built on trust. There are clear preferences that crawlers should be transparent, serve a clear purpose, perform a specific activity, and, most importantly, follow website directives and preferences. Based on Perplexity’s observed behavior, which is incompatible with those preferences, we have de-listed them as a verified bot and added heuristics to our managed rules that block this stealth crawling.”
Elevate Your Investing Strategy:
- Take advantage of TipRanks Premium at 50% off! Unlock powerful investing tools, advanced data, and expert analyst insights to help you invest with confidence.
Published first on TheFly – the ultimate source for real-time, market-moving breaking financial news. Try Now>>
Read More on NET:
- Cloudflare’s Earnings Call Highlights Robust Growth
- Cloudflare price target raised to $230 from $175 at Truist
- Cloudflare price target raised to $240 from $220 at Mizuho
- Cloudflare price target raised to $199 from $162 at Cantor Fitzgerald
- Cloudflare price target raised to $239 from $200 at Oppenheimer