Netlify Integration

Netlify Edge Functions run on Deno at the CDN edge, before your site is served. This makes them ideal for intercepting bot requests and routing them through SeoRend.

How It Works

Bot → Netlify Edge Function → SeoRend API → Rendered HTML
User → Netlify Edge Function → Your Site (unchanged)

Requirements

• Netlify project (any framework: React, Vue, Next.js, Astro, SvelteKit, etc.)
• SeoRend API key from app.seorend.com

1. Add Your API Key

   In your Netlify project:

   1. Go to Site Settings → Environment Variables
   2. Add SEOREND_API_KEY with your key as the value
   3. Set scope to All scopes or Functions + Runtime

2. Create the Edge Function

   Create the file netlify/edge-functions/seorend.ts:

   netlify/edge-functions/seorend.ts

   import type { Config, Context } from 'netlify:edge';
   
   const BOT_PATTERNS = [
     // Google
     'googlebot', 'googleother', 'storebot-google', 'google-inspectiontool',
     'google-extended', 'google-cloudvertexbot', 'google-favicon',
     'apis-google', 'adsbot-google', 'mediapartners-google', 'feedfetcher-google', 'google page speed',
     // Bing / Microsoft
     'bingbot', 'bingpreview', 'msnbot', 'adidxbot', 'microsoftpreview',
     // Yandex
     'yandexbot', 'yandeximages', 'yandexvideo', 'yandexmobilebot', 'yandexmetrika', 'yandexaccessibilitybot', 'yabrowser',
     // Baidu & other search
     'baiduspider', 'duckduckbot', 'duckassistbot', 'applebot', 'naver',
     'sogouspider', '360spider', 'coccoc', 'seznambot', 'qwantbot',
     'ecosia', 'yahoo', 'exabot', 'petalbot', 'bravebot',
     // AI / LLM crawlers
     'gptbot', 'oai-searchbot', 'chatgpt-user', 'chatgpt',
     'claudebot', 'claude-web', 'claude-user', 'claude-searchbot', 'anthropic-ai',
     'perplexitybot', 'perplexity-user', 'amazonbot', 'ccbot',
     'meta-externalagent', 'meta-externalfetcher',
     'mistralai-user', 'cohere-ai', 'ai2bot',
     'diffbot', 'deepseekbot', 'firecrawlagent', 'kagi-fetcher',
     'bytespider', 'tiktokspider',
     'xai-crawler', 'timpibot', 'anchor browser', 'novellum ai crawl', 'proratainc',
     // Social media / link previews
     'facebookexternalhit', 'facebookcatalog', 'facebookbot', 'facebookplatform', 'instagram',
     'twitterbot', 'linkedinbot', 'pinterestbot', 'pinterest',
     'slackbot', 'slack-imgproxy', 'discordbot', 'telegrambot', 'whatsapp',
     'redditbot', 'snapchat', 'vkshare',
     'tumblr', 'flipboard', 'outbrain', 'embedly', 'bitlybot', 'quora link preview', 'line', 'viber',
     // SEO tools
     'semrushbot', 'splitsignalbot', 'ahrefsbot', 'ahrefssiteaudit',
     'mj12bot', 'rogerbot', 'dotbot', 'chrome-lighthouse', 'screaming frog',
     'oncrawlbot', 'botifybot', 'deepcrawl', 'lumar', 'dataforseobot', 'serpstatbot', 'w3c_validator',
     // Internet Archive
     'ia_archiver', 'archive.org_bot',
   ];
   
   const STATIC_EXT = /\.(js|css|png|jpg|jpeg|gif|ico|svg|webp|avif|woff2?|ttf|eot|pdf|zip|mp4|wasm|map)$/i;
   
   function isBot(ua: string): boolean {
     const lower = ua.toLowerCase();
     return BOT_PATTERNS.some(p => lower.includes(p));
   }
   
   export default async function handler(request: Request, context: Context) {
     const url = new URL(request.url);
     const ua = request.headers.get('user-agent') || '';
   
     // Only handle GET requests
     if (request.method !== 'GET') return context.next();
   
     // Skip static files and API paths
     if (STATIC_EXT.test(url.pathname)) return context.next();
     if (url.pathname.startsWith('/api/')) return context.next();
   
     // Loop guard
     if (request.headers.get('x-seorend-processed') === '1') return context.next();
   
     if (!isBot(ua)) return context.next();
   
     const apiKey = Deno.env.get('SEOREND_API_KEY');
     if (!apiKey) {
       console.error('SEOREND_API_KEY not configured');
       return context.next();
     }
   
     try {
       const renderRes = await fetch('https://render.seorend.com/v1/render', {
         method: 'POST',
         headers: {
           'Content-Type': 'application/json',
           'Authorization': `Bearer ${apiKey}`,
         },
         body: JSON.stringify({ url: request.url, userAgent: ua }),
       });
   
       if (!renderRes.ok) return context.next();
   
       const data = await renderRes.json();
   
       return new Response(data.html, {
         status: data.status || 200,
         headers: {
           'Content-Type': 'text/html; charset=utf-8',
           'X-Seorend-Cache': data.cache?.hit ? 'HIT' : 'MISS',
         },
       });
     } catch {
       return context.next();
     }
   }
   
   export const config: Config = {
     path: '/*',
     excludedPath: [
       '/_next/*',
       '/static/*',
       '/.netlify/*',
     ],
   };

3. Configure netlify.toml

   Add the edge function declaration to your netlify.toml:

   netlify.toml

   [[edge_functions]]
     path = "/*"
     function = "seorend"

   If you don’t have a netlify.toml, create one in the project root with the above content.

4. Deploy

   netlify deploy --prod

   Or push to your connected Git branch.

Testing locally

You can test edge functions locally with Netlify Dev:

netlify dev
curl -A "Googlebot/2.1" http://localhost:8888/

Verification

# Should return X-Seorend-Cache header
curl -s -I -A "Googlebot/2.1" https://your-site.netlify.app/

# Should NOT have X-Seorend headers
curl -s -I -A "Mozilla/5.0 Chrome/120" https://your-site.netlify.app/

Troubleshooting

Edge function not triggered: verify netlify.toml is in the project root and path = "/*" is correct.

Environment variable not found: Deno.env.get() is used (not process.env) since Edge Functions run on Deno. Confirm the variable is set in Site Settings → Environment Variables.

netlify/edge-functions directory not found: create it manually at the project root.

Next Steps

• Testing with curl — validate your setup
• Warm Bot — keep important pages always cached