Evaluating Referral Traffic in GA4: AI Tools + Privacy Framework

I guarantee you it’s already in your GA4 – it’s just hiding. During the webinar, I demonstrated how AI traffic is currently misclassified in your referral traffic reports and can also show up in that frustrating “unassigned” channel.

Why does this happen? ChatGPT and Perplexity are sending visitors to your website, but they’re not properly setting the UTM medium parameter that GA4 needs. So instead of showing up as a distinct traffic source, this referral traffic gets lumped in with everything else or worse – shows up as chatgpt/”not set” in the unassigned channel.

The Hidden Quality Story

What I’m also seeing regularly is that AI referral traffic is quality traffic. Visitors coming from AI tools show higher engagement rates, spend longer on the website, and view more pages than your average visitor.

This isn’t just a measurement exercise anymore – it’s an optimization opportunity hiding in your referral and unassigned traffic channels.

I put together a risk framework for my upcoming LinkedIn Learning course that I shared during the webinar. Here’s what I tell clients they can safely work with:

Low Risk Data for AI Analysis

• Aggregated metrics like sessions and page views
• Basic page paths (like /products or /services)
• Device categories and broad channel groups such as organic search
• General geographic data at city or region level

Medium Risk Data Requiring Caution

• On-site search queries (may contain personal information)
• Form field selections and detailed behavior data
• Specific location data beyond general regions

High Risk Data – Never Upload to External AI Tools

• Any personal information or user IDs
• Page titles or paths behind user logins
• Custom dimensions based on individual user data
• Customer lists or email addresses

I’ve seen too many companies get excited about AI analysis and upload everything without thinking about the consequences. In my experience working with agencies, I never want to see your HubSpot contacts. I never want you to send me your customer lists! That’s private information and I don’t want to see it.

Legal consultation timeline reality check: I can’t give you a time estimate for dealing with legal teams about AI use – sorry! In my experience, this process can take anywhere from a few days to several weeks. Some EU companies I work with need comprehensive data processing agreements (DPAs) before they can use any AI tools for data analysis.

If your organization operates under GDPR or similar privacy regulations, you’ll need to verify that your AI tools have proper DPAs in place. Not all AI tools include these by default, and GDPR fines are serious money – just don’t mess with them.

While I won’t repeat the detailed setup instructions here since I already have comprehensive guides on both topics, I want to give you the strategic framework I shared during the webinar.

Step 1: Create Your AI Tools Custom Channel Group

This takes about 5 to 10 minutes of hands-on time. I have a complete AI traffic tracking guide that walks through the exact regex pattern and setup process.

Create an audience immediately. Once you have the channel group set up, create an audience of these AI tool visitors. This lets you see if they return to your site later, use them in remarketing, and compare their long-term engagement patterns to other traffic sources.

Step 2: Track AI Overviews and Text Highlighting

For this advanced tracking method, I have a detailed guide for tracking AIO, featured snippets, and People Also Ask results that covers the Google Tag Manager implementation.

During the webinar, I demonstrated how AI Overview clicks often include text highlighting parameters in the URL. This method captures more than just AI Overviews – it also tracks People Also Ask and featured snippet clicks that include text highlighting.

Honest limitation: Not every AI Overview click includes highlighting, and there’s no way to distinguish between AIO versus people also ask versus featured snippets in the data. But we track what we can and try not to worry about what we can’t track.

Here’s where I see analytics professionals get into trouble. They export twenty reports from GA4, upload everything to ChatGPT or Claude, and expect magic. But LLMs just want to help! They don’t know when to say no, and they’ll try to find patterns in garbage data whether you like it or not.

My 3-Step AI Analysis Framework

Step 1: Clean and Prepare Your Data

I always start with an exploration in GA4 that only has the dimensions and metrics I actually need for my analysis question. If something’s not relevant to what I’m trying to figure out, I don’t include it.
Filter out low volume data – for example, exclude pages with less than 50 views so that you can avoid having the AI focusing on outliers. Also check for translated page titles or other weird artifacts and exclude those as well.

Before uploading to any AI tool, remove those metadata rows at the top of GA4 exports. LLMs really struggle with that XML schema information, and it throws off the entire analysis.

Step 2: Context Setting

This is where I see the biggest failures. You can’t just upload data and say “analyze this.” You need to provide business context:

• What type of business this is (B2B SaaS, moving company, pest control, etc.)
• Important timing considerations (6-month sales cycles, seasonal patterns, holiday impacts)
• Specific data you’re providing (content performance for top 25 pages, Q4 2024 excluding holiday periods)
• Exact questions you want answered (phrase everything as a question – I call this the Jeopardy method)

I use Claude mostly – it’s my tool of choice, though you can use whatever you prefer. The key is giving it enough context to understand your business without overwhelming it with irrelevant information.

Step 3: Quality Control

Cross-reference any output with your business knowledge. Does what the AI is telling you actually make sense? Don’t act on small sample sizes – if it comes back with insights based on a tiny amount of visitors, that’s way too small to make confident decisions.

Look for confounding variables. If traffic suddenly dropped, maybe there was an ice storm that shut down operations for a week, but you forgot to mention that context to the AI.

Always test recommendations on a small scale before making site-wide changes. Document your process and outcomes – trust me on this one. I learned early in my career as a technical writer that future you will thank present you for good documentation.

For more structured approaches to analytics planning and measurement strategy, I recommend checking out our comprehensive guide to creating good marketing reports and dashboards which covers many of the same principles I use for AI analysis preparation.

During the webinar, I got some practical questions that I think will help everyone understand how to communicate these insights effectively.

• Amy asked about using LLMs to analyze content complexity. This is actually an excellent use case. I recommend creating an instruction set that defines exactly what you mean by “complicated” – are you looking for reading grade level, technical terminology density, or something else? The more specific your criteria, the better results you’ll get.
• Jeff wanted to know how to communicate AI tracking to clients and stakeholders. I focus on two key points: engagement quality and directional data. Show them that AI referral traffic typically has above-average engagement rates, and explain that we’re looking at this as directional information rather than absolute truth. Nothing in analytics is perfect anymore, so we’re measuring what we can and using it to understand trends.
• Someone (they didn’t give a name) asked about PII in AI analysis. Please don’t. I can’t stress this enough – never upload personal information to external AI tools.

During the webinar, several attendees asked about getting legal approval for AI data analysis. Here’s what I’ve learned from working with companies of different sizes:

• Small businesses: Usually can make decisions quickly, but may not have privacy policies that cover AI tool usage for data analysis.
• Mid-size companies: Technical implementation is usually feasible, but privacy compliance may require legal review that takes weeks.
• Enterprise organizations: Best positioned for implementation but often have the longest approval processes and most restrictive AI policies.
• For agencies: Client education and approval processes add complexity. You need clear policies about what data you will and won’t access, and what you will and won’t upload to AI tools.

The key is starting these conversations early. Don’t wait until you’re excited about implementing everything to discover your legal team has a blanket ban on AI tool usage.

The growth rate of AI referral traffic suggests this isn’t a trend – it’s a fundamental shift in how people discover content. While the current volume may seem small at 0.15% of global traffic, that 527% growth rate means early implementation gives you a significant advantage in understanding this traffic source.

Start with tracking implementation right away. You don’t need permission from legal to do this. Meanwhile, you can start the privacy consultation with your legal team, and you’ll already be collecting data while working through organizational approvals for AI analysis.

Most importantly, remember that better prompts plus better data equals better insights. The framework I shared during the webinar works because it acknowledges the limitations of LLMs while leveraging their pattern recognition strengths.

Your business context is what turns AI-identified patterns into actual strategic insights. The technology is the assistant – you’re still the expert driving the decisions.

For more insights like this and to stay updated on AI and analytics trends, join my newsletter or explore my comprehensive analytics training courses.

Have questions about implementing these techniques? Join the discussion in my YouTube community where I regularly respond to comments and questions from webinar attendees.