Week 9: Using AI to decode competitor websites

The Experiment 

I needed to understand what makes competitor websites tick before helping a client build theirs. Instead of spending weeks manually analyzing search data and user behavior, I wondered: could AI help me quickly extract meaningful insights from competitor search data? The goal was simple - understand what drives customers, what content matters, and how to structure a new site effectively.

The Process

Here's how I tackled it:

1. Gathering the Raw Data

   • Started with a Similar Web trial (there are alternatives like SEMrush, Ahrefs, or Uber Suggest if you prefer)

   • Pulled data on organic search terms, ad keywords, and visitor behaviour

   • Downloaded sitemap.xml files to understand content structure

2. Letting AI Do the Heavy Lifting

   • Fed the data into ChatGPT for analysis

   • Asked it to identify patterns in consumer priorities

   • Had it analyse CPC (Cost Per Click) and search volumes

   • Requested themes and motivations behind search behaviour

3. Connecting the Dots

   • Combined high-volume terms with high CPC to find "sweet spots" for navigation, merchandising and content

   • Used website crossover (i.e. which other websites they had visited) data to build customer journey profiles

   • Analysed ad copy themes to understand value propositions

   • Mapped content hierarchies based on search patterns

The Outcome

Unlike my Python finance adventure (which nearly broke my spirit), this experiment was surprisingly smooth. In just a few hours, I had:

• Clear insights into customer priorities and search behaviour

• Data-backed recommendations for site navigation

• Understanding of high-value content areas (where high search volume meets high CPC)

• Quick overview of customer profiles based on cross-site behaviour

The AI analysis was straightforward and effective - no complex prompts needed, just clear questions about patterns and priorities. While it's not a deep-dive analysis, it's perfect for quick, actionable insights.

Key Takeaway

Combining search data tools with AI analysis is a fast, cost-effective way to understand market positioning and customer behaviour. It's not exhaustive, but it gives you enough insight to make informed decisions about website structure and content priorities.

Pro Tips for Beginners:

1. Focus on Patterns: Ask AI to identify themes in search terms and ad copy - it's great at spotting recurring messages and priorities

2. Follow the Money: High CPC + high volume = areas worth prioritizing in your navigation and content strategy

3. Cross-Reference Everything: Look at visitor behavior across sites to understand customer journeys and preferences

4. Don't Overthink It: Simple, clear prompts work fine - no need for complex AI engineering.

Want to Try It Yourself?

• Start with a Similar Web trial (or alternatives like SEMrush/Ahrefs)

• Feed the data into ChatGPT or your preferred AI tool

• Focus on quick insights rather than exhaustive analysis

• Use the findings to inform navigation, content hierarchy, and value propositions

Next time, maybe I'll try using AI to analyze the actual website content and see how that compares to search insights. But for now, this quick-and-dirty approach has given me plenty to work with.