LLM Crawler for WordPress

Changelog & Version History

Version 0.7.0 Latest Major Release
February 2026
New Standards Support
  • Spec-compliant /llms.txt following llmstxt.org format (Markdown with H1/H2/link lists)
  • New /llms-full.txt endpoint with inline full content
  • New /.well-known/agents.json for machine-readable API descriptions
  • MCP (Model Context Protocol) server at /wp-json/llm/v1/mcp with 6 tools
  • AI-specific XML sitemap provider for WordPress 5.5+ sitemaps
AI Crawler Intelligence
  • Automatic detection of 12+ AI crawlers (GPTBot, ClaudeBot, PerplexityBot, etc.)
  • X-LLM-Endpoints header sent to detected AI bots
  • Enhanced robots.txt with per-bot Allow rules for all AI crawlers
  • New Analytics dashboard tracking bot visits, top pages, and last-seen timestamps
Content Optimization
  • Schema.org JSON-LD output on singular posts (Article/WebPage)
  • LLM-specific meta tags: llm-content-type, llm-language, llm-last-modified
  • Markdown alternate link for each post via rel="alternate"
  • AIOSEO compatibility added (noindex detection)
Security & Privacy
  • Telemetry changed to opt-in only (GDPR compliance)
  • No data sent during activation — requires explicit user consent
  • SHA-384 package hash verification on updates
  • Ed25519 package signing infrastructure (WordPress 5.2+ native)
  • Site URL hashed (SHA-256) in update checks for privacy
  • Explicit SSL verification on all remote requests
Admin UI
  • New Discovery tab with toggles for llms.txt, llms-full.txt, agents.json, and MCP
  • New Analytics tab with bot visit charts and top crawled pages
  • Improved telemetry description listing exactly what data is sent
  • Configurable per-type limit for llms.txt entries
Version 0.6.8 Minor Release
October 2025
New Features
  • Added "Reset to Defaults" button for quick configuration
  • Confirmation dialog lists all settings that will be changed
  • Preserves important data like Client ID when resetting
Improvements
  • Better handling of settings persistence
  • Improved user experience with clear reset functionality
Version 0.6.7 Patch
October 2025
Bug Fixes
  • Fixed default checkboxes not showing as checked on General tab
  • Forced critical defaults on plugin upgrade
  • Completely removed Update Server field from UI
UI Improvements
  • Discovery URLs now display on individual lines
  • Added copy button next to each endpoint URL
  • Improved Discovery Endpoints section layout
  • JavaScript copy functionality with visual feedback
System Updates
  • Update server URL hardcoded to https://llm-crawler.exploredigital.network
  • Improved llms.txt file writing with better error handling
  • Dynamic serving of llms.txt if file creation fails
Version 0.6.6 Patch
October 2025
Settings Updates
  • Updated default settings for optimal LLM crawling
  • Cache TTL default changed from 5 to 60 minutes
  • Auto-update preference enabled by default
  • All discovery options enabled by default
Improvements
  • Better activation process for new and existing installs
  • Simplified file writing logic for llms.txt
  • Improved error messages
Version 0.6.5 Minor Release
October 2025
New Features
  • Added Settings link on plugins page for quick access
  • Added "About this plugin" link pointing to update server
  • Plugin action links for better navigation
Default Settings
  • Respect SEO noindex checked by default
  • Public discovery enabled by default
  • All discovery methods activated by default
  • Telemetry and auto-updates enabled by default
Version 0.6.4 Major Fix
October 2025
Critical Fixes
  • Fixed duplicate Client ID systems causing multiple registrations
  • Consolidated Client ID management to single system
  • Fixed Client ID disappearing when toggling auto-update
  • Removed duplicate settings storage
Technical Improvements
  • Update client now uses main plugin's Client ID via dependency injection
  • Improved settings persistence
  • Better integration between plugin components
  • Added public get_client_id() method
Versions 0.6.0 - 0.6.3 Initial Releases
September - October 2025
Core Features
  • Initial release of LLM Crawler plugin
  • REST API endpoints for manifest, content, and delta updates
  • Token-based authentication system
  • IP allowlisting and rate limiting
  • ACF field support with configurable allowlists
  • JSONL export functionality
  • Webhook notifications for content updates
  • Universal discovery via /.well-known and robots.txt
  • Self-update mechanism from dedicated update server
  • Per-post exclusion capabilities
  • SEO plugin integration (Yoast, RankMath)
  • Smart caching system