How to Add a Domain

Prerequisites

• A registered account on SEO by Daniel
• A publicly accessible website (e.g., example.com)
• (Recommended) A robots.txt file with Sitemap directives for auto-discovery

Step 1: Navigate to Domains

From your dashboard, click Domains in the navigation or go directly to /domains. If this is your first time, you'll see an empty state prompting you to add your first domain.

Click Add Domain to open the creation form.

What you should see: the Domains list is empty on a new account, with a single Add Domain button. If you already have domains, they appear as cards with their last scan time and average score.

Step 2: Fill in domain details

The domain form asks for:

Required fields

• Name: A human-readable label for this domain. Use something you'll recognize, especially if you manage multiple client sites. Examples: "Client - Acme Corp", "My Blog", "E-commerce Store".
• Domain: The actual domain name without https://. Examples: example.com, blog.mysite.com. This is the hostname SEO by Daniel will crawl.

Optional fields

• Internal URL Prefix: Use this if your site's internal links use a different URL format than your domain. For example, if your domain is example.com but internal links point to www2.example.com, set the prefix to https://www2.example.com. This helps SEO by Daniel correctly classify links as internal vs. external.

Configuration settings

• Scan Frequency: How often your pages are re-scanned. Options: every 1 hour, every 6 hours, every 12 hours, daily, weekly, monthly. Daily is recommended for actively managed sites. Weekly works for stable content.
• Pages Per Batch: How many pages are scanned at once. Default is 10. For larger sites, increase this to scan faster. Note: this is also limited by your Pages Per Hour rate limit, which prevents overwhelming your server.
• Pages Per Hour: Maximum pages scanned per hour. Default is 100. This protects your site's server from excessive requests.

Step 3: Add sitemaps (optional, auto-discovery handles this)

After the domain is created, SEO by Daniel automatically fetches your robots.txt and extracts any Sitemap: directives. Supported formats:

• Standard XML sitemaps (/sitemap.xml)
• Sitemap index files (XML files that list other sitemaps)
• Gzipped sitemaps (/sitemap.xml.gz)
• Nested sitemap indexes (sitemap index pointing to another sitemap index)

If auto-discovery doesn't find your sitemaps, add them manually from the domain page. Paste the full sitemap URL (e.g., https://example.com/sitemap-products.xml) and click Add. Then click Parse to extract pages immediately, or click Rediscover to trigger auto-discovery again.

Step 4: Wait for page discovery

Sitemap parsing runs in the background as a queued job. For small sites (fewer than 500 URLs), discovery is nearly instant. For large sites with multiple sitemaps, it may take a few minutes. The domain page shows how many pages have been discovered so far.

Watch the status: Each sitemap shows its parse status. If a sitemap shows 0 pages, it may be unreachable, malformed, or use a format SEO by Daniel doesn't support. Try downloading it manually and checking it in a browser.

What you should see: the domain page lists each discovered sitemap with a parsed URL count. A sitemap index expands into its child sitemaps. The page count updates as parsing finishes.

Step 5: First scan begins automatically

Once pages are discovered, the first scan starts on the next scheduled interval (typically within 1 minute for new domains). The scan:

1. Fetches each page using a standard HTTP client
2. Follows up to 5 redirect hops and records the chain
3. Parses the HTML to extract all on-page elements
4. Measures load time (ms) and page size (bytes)
5. Assigns a score and generates issues

The scan status is visible on the domain page and scans list. Completed scans show the number of pages processed.

What you should see: a status of "scanning", then "completed" with a processed page count that matches what was discovered. Pages that could not be fetched keep a status code of 0 and are listed separately.

Step 6: View your report

After the scan completes, go to your domain page and click Full Report. You'll see:

• An overall score and issue summary
• Issues grouped by type (e.g., "Missing meta description on 5 pages")
• A per-page table with individual scores and findings
• A score trend chart as more scans accumulate data
• Links to the Executive Summary and CSV export

What you should see: an average score for the domain, a severity breakdown above the issue list, and every issue linked to the pages it affects. If the average looks low on a new site, open the per-page table before assuming something is broken.

Troubleshooting

"I added my domain but no pages appeared"

Check these in order:

1. Verify your robots.txt is accessible at https://yourdomain.com/robots.txt. If it returns a 404, SEO by Daniel can't auto-discover sitemaps.
2. Check that your robots.txt contains Sitemap: directives. Without them, add sitemaps manually.
3. Add sitemaps manually from the domain page if auto-discovery fails.
4. Check the sitemap URL in a browser - if it times out or returns an error, the sitemap may be unreachable.
5. For password-protected or IP-restricted sites, SEO by Daniel won't be able to access your pages. Use a publicly accessible staging environment or whitelist the SEO by Daniel user agent.

"My site has 500 pages but only 12 were found"

Check your sitemaps: are all your pages listed? Some CMS platforms generate partial sitemaps for newly published pages only. Check your sitemap directly in a browser to see the URL count. Also check for nested sitemap indexes that may not be discoverable from your main sitemap.xml.

"Scans are stuck or not running"

Scans are queued and run in batches based on your per-hour rate limit. If you have multiple domains, they scan sequentially, not simultaneously. Check the domain's rate limit settings - if you set Pages Per Hour to a low value, scans process slowly.

"My pages all score 0"

A score of 0 usually means the pages couldn't be fetched (they returned a non-200 status or timed out). Check the page status codes on the domain report - they should be 200. If pages return 0 as the status code, the HTTP client couldn't reach them at all.

FAQ

Does SEO by Daniel need access to my server or code?

No. It requests your public pages over HTTP, the same as a browser. There is nothing to install.

Can I audit a staging or password-protected domain?

Only if it is reachable without authentication. If the site is behind a login or an IP allowlist, the crawl gets a 401 or times out. Use a publicly reachable staging URL, or allowlist the SEO by Daniel user agent.

My site is on www but I entered the bare domain. Does that matter?

The bare domain is fine. SEO by Daniel follows the redirect to www. If internal links point at a different hostname, set the Internal URL Prefix so links are classified correctly.

How do I re-scan after fixing something?

Scans run automatically on your configured frequency. You can also trigger a scan from the domain settings if you want to verify a fix immediately.