# Depureco USA AI Crawler Implementation Notes

This file is an implementation checklist for publishing the AI/crawler file set.

## Recommended Public Files

- `/llms.txt` — concise root map for AI systems.
- `/llms-full.txt` — fuller one-file summary for agents that prefer a single fetch.
- `/ai/product-index.md` — product-family and product summary index.
- `/ai/product-specs.json` — structured product specs and AI research context.
- `/ai/applications.md` — application-first routing.
- `/ai/materials.md` and `/ai/material-behavior-matrix.md` — material-first routing.
- `/ai/industries.md` — industry and market routing.
- `/ai/compliance.md` and `/ai/regulatory-glossary.md` — conservative compliance-adjacent language.
- `/ai/video-library.md` — video and demo context.
- `/ai/accessories.md` — filters, tools, kits, accessories, spare parts.
- `/ai/decision-guide.md` and `/ai/decision-maker-research.md` — buyer-role and selection logic.
- `/ai/solution-routing.json` and `/ai/use-case-crosswalk.json` — machine-readable routing.

## Server/SEO Notes

- Keep files crawlable unless a file contains internal strategy or private data.
- Serve `.txt` as text/plain, `.md` as text/markdown or text/plain, and `.json` as application/json.
- Avoid publishing pricing, HubSpot/CRM data, private lead notes, or internal campaign strategy in public crawler files.
- Add root-level links from `/llms.txt` to the most important AI support files.
- Consider adding HTTP discovery headers if your server/CDN supports them: `Link: </llms.txt>; rel="llms-txt", </llms-full.txt>; rel="llms-full-txt"`.

## Robots Notes

- Allow reputable AI/search crawlers only if this aligns with Depureco USA’s policy.
- Maintain normal sitemap discovery through `/sitemap_index.xml`.
- Avoid blocking `/ai/` if these files are intended for AI retrieval.


## Document Library Files

Allow crawling of `/ai/document-library.md` and `/ai/document-routing.json`. These files point to public Google Drive PDFs and presentations. If Drive permissions change, update the routing file so crawlers do not receive dead or permission-blocked links.

## AI sitemap

A static sitemap is included for the AI/crawler support files:

- `https://depurecousa.com/ai-sitemap.xml`

Recommended `robots.txt` addition:

```txt
Sitemap: https://depurecousa.com/ai-sitemap.xml
```

The sitemap is intended for discovery of `/llms.txt`, `/llms-full.txt`, and the public `/ai/` markdown and JSON files. It does not replace the Yoast XML sitemap for normal WordPress pages/products.