Cloudflare + Pantheon WordPress: Free Tier Configuration for Security, Performance, and AI Bots

The Cloudflare + Pantheon Stack

Pantheon provides exceptional managed WordPress hosting with built-in caching, CDN, and performance optimizations. Adding Cloudflare’s free tier on top creates a powerful security and performance layer—but only when configured correctly to complement rather than conflict with Pantheon’s infrastructure.

Many developers make critical mistakes when pairing Cloudflare with Pantheon:

• Cache conflicts that break WordPress functionality
• Over-aggressive bot blocking that hurts legitimate traffic and SEO
• Misconfigured security rules that either block users or leave vulnerabilities
• AI bot settings that either tank performance or harm search visibility

This guide shows you how to configure Cloudflare’s free tier to work seamlessly with Pantheon, focusing on practical rules that enhance security, optimize performance, and intelligently manage AI crawlers.

Prerequisites

This guide assumes you have:

• WordPress site hosted on Pantheon
• Cloudflare account (free tier is sufficient)
• DNS already pointed to Cloudflare
• Basic familiarity with Cloudflare dashboard
• Understanding that Pantheon already provides caching and CDN

Important Context: As a Pantheon Platinum Partner managing multiple client sites, we’ve tested these configurations extensively across different traffic patterns and use cases.

Understanding the Cache Layer Strategy

Critical Principle: Cloudflare’s cache must complement—not replace—Pantheon’s cache layer.

How Pantheon Caching Works:

• Page-level caching for anonymous users
• Object caching (Redis) for database queries
• Edge caching via built-in CDN
• Automatic cache clearing on content updates

How Cloudflare Should Layer On Top:

• Cache static assets (CSS, JS, images, fonts) globally
• Add security filtering before requests hit Pantheon
• Block malicious traffic at the edge
• Provide additional DDoS protection

What NOT to Do:

• Don’t cache WordPress admin pages
• Don’t cache logged-in user sessions
• Don’t cache dynamic endpoints (AJAX, API)
• Don’t override Pantheon’s cache headers unnecessarily

Cache Configuration Rules

Cloudflare’s free tier cache rules complement Pantheon perfectly when configured correctly. Here are the two essential rules:

Cache Rule #1: Bypass WP Admin and Login

Purpose: Prevent caching of administrative areas and login pages.

Configuration:

1. Go to Caching > Cache Rules in Cloudflare dashboard
2. Click Create Rule
3. Name: “Bypass WP Admin and Login”
4. Set up the expression:

(http.request.uri.path contains "/wp-admin") or
(http.request.uri.path eq "/wp-login.php")

5. Cache eligibility: Bypass cache
6. Save and deploy

Why this matters: Caching admin pages breaks WordPress functionality and can expose sensitive data. This rule ensures dynamic admin content always hits Pantheon fresh.

Cache Rule #2: Cache Static Assets (Pantheon Safe)

Purpose: Aggressively cache static files that rarely change.

Configuration:

1. Create a new Cache Rule
2. Name: “Cache Static Assets”
3. Expression:

(http.request.uri.path contains ".css") or
(http.request.uri.path contains ".js") or
(http.request.uri.path contains ".jpg") or
(http.request.uri.path contains ".jpeg") or
(http.request.uri.path contains ".png") or
(http.request.uri.path contains ".gif") or
(http.request.uri.path contains ".webp") or
(http.request.uri.path contains ".avif") or
(http.request.uri.path contains ".svg") or
(http.request.uri.path contains ".ico") or
(http.request.uri.path contains ".woff") or
(http.request.uri.path contains ".woff2") or
(http.request.uri.path contains ".ttf") or
(http.request.uri.path contains ".otf") or
(http.request.uri.path contains ".eot")

4. Cache eligibility: Eligible for cache
5. Edge TTL: 1 month or longer
6. Save and deploy

Why this matters: Static assets benefit from global edge caching. This includes modern formats like AVIF and WebP along with traditional images, fonts, and code files.

Pro Tip: These files rarely change, so aggressive caching here significantly reduces origin requests to Pantheon and improves global performance.

Security Rule Configuration

Cloudflare’s free tier allows 5 custom WAF rules. We recommend using 3 slots for WordPress-specific security, leaving 2 slots available for site-specific needs or AI bot management.

Security Rule #1: Block WordPress Junk Requests

Purpose: Block common WordPress vulnerability scans and exploit attempts.

Configuration:

1. Go to Security > WAF > Custom rules
2. Click Create rule
3. Name: “WP: Block Junk”
4. Expression:

(http.request.uri.path contains "/wp-config.php") or
(http.request.uri.path contains "/.env") or
(http.request.uri.path contains "/.git") or
(http.request.uri.path contains "/cgi-bin/") or
(http.request.uri.path contains "/phpinfo.php") or
(http.request.uri.path contains "/wp-content/debug.log") or
(http.request.uri.path contains "/xmlrpc.php") or
((http.request.uri.path contains "/wp-content/uploads/") and (http.request.uri.path contains ".php")) or
(http.request.uri.path contains ".phtml") or
(http.request.uri.path contains ".php5") or
(http.request.uri.path contains ".php7") or
(http.request.uri.path contains ".phps")

5. Action: Block
6. Save and deploy

What this blocks:

• Configuration file access attempts (wp-config.php, .env)
• Version control exposure (.git)
• PHP info disclosure
• XML-RPC abuse (common brute force vector)
• PHP files uploaded to media directories (common exploit)
• Alternative PHP extensions used to bypass filters

Why this matters: These requests are never legitimate for a production WordPress site. Blocking them at the edge reduces server load and prevents automated vulnerability scanners from probing your site.

Security Rule #2: Block Scanners and Enumeration

Purpose: Prevent attackers from discovering site structure, users, and vulnerabilities.

Configuration:

1. Create new WAF rule
2. Name: “WP: Block Scanners/Enumeration”
3. Expression:

(http.request.uri.path eq "/.well-known/security.txt")
or (http.request.uri.path eq "/wlwmanifest.xml")
or (http.request.uri.path eq "/readme.html")
or (http.request.uri.path eq "/license.txt")
or (http.request.uri.path eq "/wp-json/wp/v2/users")
or (http.request.uri.query contains "author=")
or (http.request.uri.query contains "rest_route=/wp/v2/users")

4. Action: Block
5. Save and deploy

What this blocks:

• REST API user enumeration (discovers usernames)
• Author archive queries (another username discovery method)
• WordPress version disclosure files
• Legacy Windows Live Writer manifest
• REST API endpoints that expose site structure

Why this matters: Username enumeration is the first step in brute force attacks. Blocking these discovery methods significantly hardens your WordPress security posture.

Important Note: If you use the REST API for legitimate purposes (headless WordPress, mobile apps, integrations), or have trouble publishing, you’ll want to modify this rule to allow specific endpoints or authenticated requests.

Security Rule #3: Protect WP Login/Admin

Purpose: Add an additional challenge layer to WordPress authentication endpoints.

Configuration:

1. Create new WAF rule
2. Name: “WP: Protect wp-login/admin”
3. Expression:

(http.request.uri.path eq "/wp-login.php") or 
(starts_with(http.request.uri.path, "/wp-admin"))

4. Action: Managed Challenge
5. Save and deploy

Why Managed Challenge (not Block):

• Legitimate users can still access admin
• Automated bots get challenged and typically fail
• Reduces brute force attack effectiveness
• Cloudflare adapts challenge difficulty based on threat level

What this protects against:

• Automated brute force login attempts
• Credential stuffing attacks
• Bot-driven admin scanning

User Experience Note: Legitimate administrators may occasionally see a brief challenge screen before accessing wp-admin. This is a small trade-off for significantly enhanced security.

Slots 4 & 5: Custom Rules for Your Needs

You have 2 remaining WAF rule slots available. Common uses:

Geographic Restrictions: Block countries where you don’t do business to reduce attack surface.

Rate Limiting: Limit requests per IP for contact forms or search endpoints.

AI Bot Management: Our preferred use for slot #4 (detailed below).

AI Bot Management Strategy

AI crawlers present a unique challenge: some are legitimate (improving search, training useful models), while others are purely extractive or disruptive. Our approach balances being crawlable while blocking problematic bots.

Our Philosophy on AI Crawling

We prefer to keep websites AI crawl-friendly for several reasons:

• Search engines use AI for indexing and ranking
• Legitimate AI tools can drive discovery and traffic
• Blanket blocking risks SEO penalties
• Selective blocking is more effective than total blockage

However, some AI bots are genuinely disruptive:

• Aggressive crawling that overwhelms servers
• Bots that ignore robots.txt and crawl directives
• Content scrapers with no value exchange
• Bots from platforms with questionable data practices

Custom AI Bot Blocking Rule (Slot #4)

Configuration:

1. Create new WAF rule
2. Name: “Block Specific AI Bots”
3. Expression:

(http.request.uri.path ne "/robots.txt") and 
((http.user_agent contains "Bytespider") or 
(http.user_agent contains "meta-externalagent") or 
(http.user_agent contains "PetalBot"))

4. Action: Block
5. Save and deploy

Why these specific bots:

• Bytespider (ByteDance/TikTok): Aggressive crawling behavior
• meta-externalagent (Meta/Facebook): Data collection concerns
• PetalBot (Huawei): Often ignores crawl directives

Important: We still allow access to robots.txt so these bots can see our crawl preferences (even if they ignore them).

Alternative Approach: Use Cloudflare’s built-in AI Scrapers and Crawlers list:

1. Go to Security > Bots
2. Enable AI Scrapers and Crawlers
3. Review the managed list of AI bots
4. Choose which to allow, challenge, or block

Pro Tip: Start with “Challenge” rather than “Block” to see how bots behave. Monitor your analytics for 1-2 weeks before making permanent blocking decisions.

Cloudflare AI Features We DON’T Recommend

❌ AI Labyrinth (BETA) – Keep This OFF

What it does: Modifies web pages by adding nofollow links containing AI-generated content to disrupt bots ignoring crawling standards.

Why we don’t recommend it:

• We tested this and saw massive gains in bot traffic (the opposite of intended effect)
• It created more disruption to our infrastructure than to the bots
• Potential SEO concerns: Some semi-legitimate bots ignore crawling standards, and Google’s crawlers could be affected
• The nofollow links, while invisible to users, may impact how search engines interpret page structure

Our experience: After running AI Labyrinth for several weeks, we disabled it due to increased server load and unpredictable bot behavior. The feature is still in beta for good reason.

❌ “Block AI Bots” (Managed Ruleset) – Be Selective

What it does: Deploys Cloudflare-managed rule to block all bots categorized as AI training crawlers.

Why blanket blocking is problematic:

• Search engines increasingly use AI for indexing
• You want to be discoverable in AI-powered search results
• Some legitimate services get caught in the net
• Better to be selective using custom rules (as shown above)

Our recommendation: Keep this OFF and use the custom rule approach for granular control over which AI bots to block.

✅ Bot Fight Mode – Turn This ON

What it does: Detects and challenges bot traffic on your domain using Cloudflare’s extensive threat intelligence.

Configuration:

1. Go to Security > Bots
2. Enable Bot Fight Mode

Why we recommend this:

• Not specifically for AI bots—handles general bot traffic
• Cloudflare’s threat intelligence is excellent
• Catches credential stuffing, scraping, and spam bots
• Minimal false positives in our experience
• Free tier feature with significant value

What it protects against:

• Automated form spam
• Content scrapers
• Brute force attempts
• DDoS botnet traffic
• Inventory hoarding bots

Monitoring Your Configuration

After implementing these rules, monitor your site for 1-2 weeks:

Key Metrics to Watch:

1. Security Events (Cloudflare > Security > Events)
   • Review blocked requests to verify rules work correctly
   • Look for false positives (legitimate traffic blocked)
2. Traffic Analytics (Cloudflare > Analytics > Traffic)
   • Compare bot vs. human traffic
   • Monitor origin request reduction
3. Cache Hit Ratio (Cloudflare > Caching > Configuration)
   • Aim for 85%+ cache hit ratio on static assets
   • Lower ratio may indicate cache rule issues
4. Pantheon Metrics (Pantheon Dashboard > Metrics)
   • Verify reduced origin requests
   • Check page load time improvements
   • Monitor server resource usage

Expected Results:

• 30-50% reduction in origin requests (more cached at edge)
• Significant drop in malicious scanning attempts
• Faster global page loads (static assets from edge)
• Reduced Pantheon resource consumption

To Pay or Not to Pay: Cloudflare WordPress Plugin

Cloudflare offers a WordPress plugin with enhanced features for approximately $20/month:

What You Get:

• Automatic cache purging when content changes
• One-click WordPress-specific hardening
• Improved bypass rules for logged-in users
• Easier configuration management
• Automatic Platform Optimization (APO)

Is It Worth It? For most sites, probably not immediately. The free tier configuration we’ve outlined provides excellent security and performance. Consider upgrading if:

• You publish content frequently and need automatic cache purging
• You have non-technical team members managing the site
• You want simplified management across multiple sites
• Budget allows for premium features

The Manual Alternative: You can achieve automatic cache purging on WordPress post updates without the paid plugin using Cloudflare’s API. This requires:

• Cloudflare Zone ID
• Cloudflare API Token
• WordPress function to trigger purge on content changes
• Secure storage of Cloudflare credentials

Coming Soon: We’ll publish a detailed tutorial on implementing automatic Cloudflare cache purging for WordPress without the paid plugin. Stay tuned.

Troubleshooting Common Issues

Problem: Admin area loads slowly or shows challenges

• Solution: Verify Cache Rule #1 is bypassing admin correctly
• Check that Security Rule #3 is using “Managed Challenge” not “Block”

Problem: Static assets not caching

• Solution: Review Cache Rule #2 expression for typos
• Check Cloudflare cache analytics for hit ratio
• Verify Pantheon isn’t sending no-cache headers for static files

Problem: Legitimate traffic getting blocked

• Solution: Review Security Events log in Cloudflare
• Identify the rule causing blocks
• Adjust expression or action (Challenge instead of Block)

Problem: Bot traffic still high after implementing rules

• Solution: Bot Fight Mode takes time to learn patterns
• Consider adding more specific bots to custom rule
• Review Analytics to identify bot user agents

Problem: Search engines not indexing properly

• Solution: Ensure you haven’t blocked Googlebot or Bingbot
• Verify robots.txt is accessible
• Don’t use AI Labyrinth
• Check Google Search Console for crawl errors

Best Practices Summary

Do:

• ✅ Layer Cloudflare caching on top of Pantheon (not replace)
• ✅ Use Managed Challenges instead of outright blocks when possible
• ✅ Monitor security events regularly for false positives
• ✅ Enable Bot Fight Mode
• ✅ Be selective about which AI bots to block
• ✅ Test rule changes in staging environment first

Don’t:

• ❌ Cache WordPress admin or login pages
• ❌ Enable AI Labyrinth (still too unpredictable)
• ❌ Block all AI bots indiscriminately
• ❌ Ignore Cloudflare analytics and security events
• ❌ Assume more aggressive = better security
• ❌ Forget to allow access to robots.txt

Performance and Security Impact

After implementing this configuration across multiple client sites, we consistently see:

Performance Improvements:

• 30-50% reduction in origin requests to Pantheon
• 200-400ms faster global page load times
• 85%+ cache hit ratio on static assets
• Reduced Pantheon resource consumption

Security Improvements:

• 90%+ reduction in WordPress vulnerability scans
• Elimination of xmlrpc.php brute force attempts
• Blocked user enumeration attempts
• Significantly reduced automated bot traffic

The Bottom Line: Cloudflare’s free tier, when configured correctly, provides enterprise-level security and performance benefits without the enterprise price tag.

---

Need help optimizing your WordPress hosting stack or configuring advanced security rules?  Contact Knihter for professional Pantheon and Cloudflare configuration services. As a Pantheon Platinum Partner, we specialize in high-performance WordPress infrastructure.

Related Services:

• Pantheon WordPress performance optimization
• Cloudflare security and CDN configuration
• WordPress security audits and hardening
• Advanced caching and CDN strategy consulting